An image target localization method based on custom feature template matching
By using a custom feature template matching method, the problems of poor template construction flexibility and low computational efficiency in existing technologies are solved, achieving high accuracy and efficient computation for image target localization, and enhancing anti-occlusion ability and matching robustness.
Patent Information
- Application Number
- CN202611285183.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-24
- Publication Date
- 2026-11-10
AI Technical Summary
Existing template matching methods for image target localization suffer from problems such as poor template construction flexibility, disconnect between feature extraction and matching calculation, single matching degree calculation method, and the requirement that the template must use the complete appearance of the target, resulting in high computational load, weak anti-occlusion ability, and low matching accuracy.
A custom feature template matching method is adopted, which selects local feature graphics from the image to form a feature template, and performs preprocessing and multiple matching degree calculation methods according to the feature type. The calculation is only performed in the area covered by the feature graphics, and rotation angle matching is supported.
It improves the flexibility and computational efficiency of template construction, enhances the ability to resist occlusion, improves matching accuracy and robustness, and supports multi-layer feature fusion and flexible accuracy adjustment.
Smart Images

Figure CN123071684A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of image processing and computer vision technology, and specifically to an image target localization method based on custom feature template matching. Background Technology
[0002] Image target localization is a fundamental problem in the field of computer vision, with wide applications in industrial inspection, robot vision, autonomous driving, and security monitoring. Template matching-based target localization methods have long been widely used due to their intuitive principles and ease of implementation.
[0003] However, existing template matching methods suffer from the following shortcomings: First, template construction is simplistic and lacks flexibility. Traditional methods typically use a complete image of the target or a pre-extracted complete feature description. When the target has a complex shape or only some regions have significant features, using a complete template not only increases computation but may also lead to matching failure due to local occlusion. Users cannot flexibly select representative local regions of the target to construct templates according to their actual needs. Second, feature extraction and matching computation are disconnected. Existing methods typically employ a uniform preprocessing workflow for the entire image during template matching, such as uniformly converting it to grayscale before edge extraction. This approach does not differentiate for specific feature types—for example, when users are focused on the color features of the target, grayscale conversion may lose crucial matching information; when users are focused on texture features, simple edge detection is insufficient to enhance texture information. There is a lack of correspondence between preprocessing operations and the types of information required for subsequent matching. Third, the matching degree calculation method is not flexible enough. Existing methods typically use a fixed similarity metric when calculating the matching degree between the template and the image, making it difficult to balance computational efficiency and anti-interference capabilities in different application scenarios. For example, the sum of squared differences is simple to calculate but sensitive to changes in illumination, while normalized cross-correlation is robust to illumination but computationally expensive. Users cannot flexibly choose or combine different calculation methods based on the actual lighting conditions, accuracy requirements, and real-time requirements of the scene. Fourth, the template must use the complete appearance of the target. In traditional methods, the template must be a complete image or a complete feature description of the target; it is not possible to construct a template by selecting only a local representative region of the target. Even if the user only focuses on a few local features of the target, a complete template containing the entire target region must be used, leading to increased computational cost, and the participation of pixels from non-feature regions of the target in the matching calculation may introduce interference. Summary of the Invention
[0004] This invention provides an image target localization method based on custom feature template matching, addressing the problems of poor template construction flexibility, disconnect between feature extraction and matching calculation, single matching degree calculation method, and the requirement for the template to use the complete appearance of the target in existing technologies. The basic process of this invention is as follows: First, an image containing the target to be located is acquired. The user selects one or more distinctive local image blocks from the image as feature graphics. These feature graphics are arranged and combined according to their original spatial relationships to form a feature template. Then, the feature template is slid across the image, and the matching degree is calculated for each corresponding region of the feature template and the image, thereby generating a matching degree curve or surface plot, the dimension of which depends on the dimension of the template sliding across the image. Finally, the location of the maximum matching degree value is located in the curve or surface plot, which is determined as the target's position in the image.
[0005] This invention provides an image target localization method based on custom feature template matching, comprising the following steps: Acquire an image containing the target to be located; Extract at least one feature graphic containing local information of the target from the image; The extracted feature graphics are arranged according to their spatial relative positions in the target to form a feature template; The feature template is traversed across the image, and the correlation between the feature template and the image is calculated to obtain a matching curve or surface plot; and The position of the target in the image is determined based on the maximum matching degree in the curve or surface plot.
[0006] Furthermore, the feature graphic is an image patch defined according to the target, capable of representing the local shape, texture, or color information of the target. That is, the feature graphic is a specific image patch selected by the user from the image, an image region with specific image content, representing the local salient information of the target; the feature types of the feature graphic include: A complete or partial outer contour segment of the target profile; Through-holes, grooves, or raised areas inside the target; Printed patterns, text symbols, or specific texture blocks on the target surface; The local image patch consisting of the corners, edge intersections, and their neighborhoods of the target.
[0007] Furthermore, when the target has a complex shape, a feature discretization method is used to extract and combine the feature graphics, including the following steps: Selectively extract only some feature patterns (representative regions) from the complex shape of the target; and The extracted feature graphics are arranged while preserving their original spatial relationships, and then combined to form a feature template that contains only the selected feature graphics.
[0008] Further, extracting at least one feature graph containing local target information from the image includes: The image is separated into multiple single-color-channel images, and the feature pattern is extracted from at least one of the single-color-channel images; or The multiple color channels of the image are weighted and fused with different weights to generate a fused image, and the feature graphic is extracted from the fused image; wherein, the weight coefficients used in the weighted fusion with different weights are calculated based on the color features of the target, with the criterion of maximizing the contrast between the target area and the background area in the fused image, so that the layer where the feature graphic is located can highlight the difference between the target and the background.
[0009] Furthermore, the feature template is composed of one or more feature graphics combined according to their original spatial relationships. This "local selection, overall combination" template construction method allows users to flexibly select representative areas of the target according to actual needs, without having to use the entire target image as a template.
[0010] Furthermore, different feature templates are defined for different layers, and each feature template contains at least one feature graphic extracted from the same layer; Each feature template corresponds to a channel. The correlation between the feature template of each channel and the image is calculated. The matching degrees calculated for each channel are then weighted and fused to generate the matching degree curve or surface map. Matching degree calculations are performed on the pixel regions covered by the feature graphics in the feature templates to generate matching degree curves or matching degree surface maps.
[0011] Furthermore, the dimension of the curve or surface plot depends on the dimension by which the feature template slides on the image.
[0012] Furthermore, before calculating the correlation between the feature template and the image, the method further includes: Identify the feature types of feature images and perform image preprocessing based on the feature types, including: When the feature graphic represents contour or edge features, the preprocessing includes image sharpening, edge detection, or binarization of the image. When the feature image exhibits texture features, the preprocessing includes histogram equalization or contrast stretching of the image; and When the feature image originates from a specific color layer, the preprocessing includes weighted fusion processing of the corresponding color channels of the image. This deterministic mapping relationship between feature type and preprocessing operation ensures that preprocessing always serves the needs of subsequent matching.
[0013] Furthermore, the matching degree is calculated based on a preset similarity function or distance function, according to the pixel values of the feature template and the pixel values of the corresponding region of the image; wherein, the larger the similarity function value, the higher the matching degree, and the smaller the distance function value, the higher the matching degree.
[0014] Furthermore, the similarity function or distance function adopts at least one of the following: Normalized cross-correlation: Calculate the normalized cross-correlation coefficient between the pixel value of the feature template and the pixel value of the corresponding region of the image. The value range is [-1, 1]. A value of 1 indicates a perfect match, and a value of -1 indicates a perfect anti-correlation. Zero-mean normalized cross-correlation: Subtract the mean of the pixel values of the feature template, subtract the mean of the pixel values of the corresponding region of the image, and calculate the normalized cross-correlation coefficient of the two sets of values obtained after subtracting the mean. Sum of squared differences: Calculate the sum of squares of the differences between the pixel values of the feature template and the pixel values of the corresponding region of the image. The smaller the value, the higher the degree of matching. Sum of absolute differences: Calculate the sum of the absolute differences between the pixel values of the feature template and the pixel values of the corresponding region of the image. The smaller the value, the higher the degree of matching. Matrix inner product: The pixels of the feature template are arranged in order to form a template vector, and the pixels of the corresponding region of the image are arranged according to the same rule to form a feature vector. The inner product of the template vector and the feature vector is calculated; a larger value indicates a higher degree of matching. Since the feature template is only composed of a combination of feature graphics, the calculation of the similarity function or distance function is based on the pixel region covered by the feature template, thereby reducing unnecessary computation and avoiding interference from non-feature regions. When using matrix inner product, the vectors can be binarized first, simplifying multiplication to logical AND operations and addition to bit counting operations, significantly improving computational efficiency.
[0015] Furthermore, the calculation of the similarity function or distance function is based solely on the pixel region covered by the feature graphic in the feature template.
[0016] Furthermore, when using the matrix inner product, each element of the template vector and / or the feature vector is first binarized, and the pixel value is converted to 0 or 1 according to a preset threshold; when calculating the binarized vector inner product, the multiplication operation is replaced by the logical AND operation, and the addition operation is replaced by the bit counting operation.
[0017] Furthermore, when the target may undergo rotational changes, a template representation corresponding to multiple rotation angles is generated based on the feature template. When the feature template is slid across the image, the correlation calculation results under the multiple rotation angles are combined to generate a matching degree curve or surface map, and the optimal matching angle and position of the target are determined.
[0018] Furthermore, determining the position of the target in the image based on the maximum matching degree in the curve or surface plot involves combining the pixel coordinates corresponding to the maximum matching degree with the size of the feature template to determine the bounding box of the target, or performing subpixel interpolation on the position of the maximum matching degree to obtain the target position with subpixel precision, and outputting the position and rotation angle of the target in the image.
[0019] This invention allows users to select one or more local image patches as feature graphics from an image, combine these feature graphics according to their original spatial relationships to form a feature template, slide the feature template on the image, calculate the correlation between the area covered by the feature graphics and the corresponding area in the image, and generate a matching degree curve or surface map. The target location is determined by the position of the maximum matching degree. This invention decomposes the traditional complete template into two levels: feature graphics and feature template. Users can flexibly select local areas to construct templates. The use of feature discretization can form sparse templates, reducing computation and enhancing anti-occlusion capabilities. The corresponding preprocessing operations are determined according to the feature type of the feature graphics to ensure that preprocessing serves subsequent matching. Multiple matching degree calculation methods are provided for selection, and the calculation is limited to the area covered by the feature graphics. It has the advantages of wide applicability, high positioning accuracy, and adjustable computational efficiency.
[0020] The core idea of this invention lies in decomposing the traditional concept of a "complete template" into two levels: "feature graphics" and "feature templates." Feature graphics are specific image patches selected by the user from an image, representing the local salient information of the target; feature templates are composed of one or more feature graphics combined according to their original spatial relationships. This "local selection, overall combination" template construction method allows users to flexibly select representative areas of the target according to actual needs, without having to use the entire target image as a template.
[0021] In summary, the present invention improves upon the prior art in the following aspects: In terms of feature extraction, users can directly select image blocks of interest from an image as feature graphics. These feature graphics can be local fragments of the target contour, through-holes or grooves inside the target, printed patterns or texture blocks on the target surface, and local image blocks near corners or edge intersections. When the target has a complex shape, a feature discretization method is used to selectively extract representative regions from the complex shape, preserving their original spatial relationships and arranging them to form a feature template containing only the selected feature graphics.
[0022] Regarding the sources of feature graphics extraction, feature graphics can be extracted directly from a single color channel of the image, or from a fused image generated by weighted fusion of multiple color channels. The weighting coefficients for weighted fusion are determined based on maximizing the contrast between the target and background regions in the fused image, thus ensuring that the layer containing the feature graphics highlights the difference between the target and the background. Furthermore, feature graphics can be extracted from different layers separately, and an independent feature template can be constructed for each layer. After matching each template with the image, the matching results of each channel are fused.
[0023] In terms of preprocessing before matching, corresponding preprocessing operations are determined based on the feature type of the feature image to enhance the information in the image corresponding to the feature image. Specifically, when the feature image exhibits contour or edge features, image sharpening, edge detection, or binarization are performed; when the feature image exhibits texture features, histogram equalization or contrast stretching is performed; and when the feature image originates from a specific color layer, weighted fusion processing is applied to the corresponding color channels of the image. This deterministic mapping relationship between feature type and preprocessing operations ensures that preprocessing always serves the needs of subsequent matching.
[0024] Regarding the calculation of matching degree, the matching degree is calculated based on the pixel values of the feature template and the pixel values of the corresponding region in the image, using a preset similarity function or distance function. A higher similarity function value indicates a higher degree of matching, while a lower distance function value indicates a higher degree of matching. The similarity function or distance function can be selected from the following calculation methods: normalized cross-correlation, calculating the normalized cross-correlation coefficient between the template pixel values and the corresponding region pixel values; zero-mean normalized cross-correlation, subtracting the mean from the pixel values before calculating the normalized cross-correlation coefficient; sum of squared differences, calculating the sum of squared differences between the template pixel values and the corresponding region pixel values; sum of absolute differences, calculating the sum of absolute differences between the template pixel values and the corresponding region pixel values; and matrix inner product, arranging the template pixels into a vector and calculating the inner product with the vector of the corresponding region image. Since the feature template is only composed of a combination of feature graphics, the calculation of the similarity function or distance function is based on the pixel region covered by the feature template, thereby reducing unnecessary computation and avoiding interference from non-feature regions. When using matrix inner product, the vectors can be binarized first, simplifying multiplication to logical AND operation and addition to bit counting operation, which greatly improves computational efficiency.
[0025] When the target may rotate, a template representation corresponding to multiple rotation angles is generated based on the feature template. During the sliding matching process, the correlation calculation results of multiple angles are combined to determine the position of the target and the optimal matching angle.
[0026] Regarding target localization accuracy, the bounding box of the target can be determined by combining the pixel coordinates corresponding to the maximum matching degree with the size of the feature template. Alternatively, subpixel interpolation can be performed on the position of the maximum matching degree to obtain the target position with subpixel accuracy, and the position and rotation angle of the target in the image can be output.
[0027] The present invention has at least the following beneficial effects: 1) Flexible template construction. Users can freely select one or more local image blocks as feature graphics according to the actual characteristics of the target, and combine them to form a feature template. The feature template does not need to completely describe the entire appearance of the target, but only needs to contain local information with discriminative power. When the target has a complex shape, the feature discretization method is used to select only some representative areas to form a sparse template, which effectively reduces the amount of template data and subsequent computation. At the same time, since the template only contains local areas of the target, when the target is partially occluded, the unoccluded feature graphics can still provide effective matching information, enhancing the anti-occlusion ability. 2) Feature extraction and preprocessing correspond to each other. The preprocessing operation is determined according to the feature type of the feature graphics, forming a clear mapping relationship of "edge features → edge enhancement preprocessing", "texture features → texture enhancement preprocessing", and "color features → color channel enhancement preprocessing". This design ensures that the image processing always revolves around the information type carried by the feature graphics, avoiding the problem of preprocessing operations being disconnected from matching requirements in traditional methods. 3) Rich and selectable matching degree calculation methods. It provides multiple similarity or distance metrics, such as normalized cross-correlation, zero-mean normalized cross-correlation, sum of squared differences, sum of absolute differences, and matrix inner product, allowing users to flexibly choose according to the lighting conditions, accuracy requirements, and real-time requirements of the actual application scenario. Normalized cross-correlation and zero-mean normalized cross-correlation are robust to lighting changes and are suitable for scenarios with complex lighting conditions; sum of squared differences and sum of absolute differences are simple to calculate and suitable for industrial scenarios with stable lighting; matrix inner product can be combined with binarization processing to accelerate bit operations and is suitable for scenarios with high real-time requirements. 4) The feature template consists only of selected feature graphics, and the matching degree calculation is only performed on the pixel area covered by the feature graphics. Compared with traditional templates that use the entire target area, this invention effectively reduces the number of pixels involved in the calculation, reducing the computational load; at the same time, non-feature areas on the target that are not selected do not participate in the matching, avoiding the interference of random matching between these areas and the image background on the matching degree response, and improving the accuracy and stability of the matching. 5) Supports multi-layer feature fusion. Feature graphics can be extracted from different color channels or blending layers, and an independent feature template can be built for each layer. The results are then fused after matching each layer separately. This approach fully utilizes the differentiated information of the target in different layers. Each layer matches independently without interference, and the final result is a weighted fusion for comprehensive decision-making, improving the robustness and accuracy of the matching. 6) Flexible and adjustable matching accuracy. It supports both pixel-level and sub-pixel-level positioning output modes, allowing users to choose the appropriate accuracy level based on application requirements. Pixel-level positioning is used in scenarios requiring fast coarse positioning, while sub-pixel interpolation is used in scenarios with higher accuracy requirements to obtain a more precise target position. Rotation angle information can also be output simultaneously. Attached Figure Description
[0028] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the embodiments of the invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.
[0029] Figure 1 This diagram illustrates the overall flow of an image target localization method based on custom feature template matching in some embodiments of the present invention. Figure 2 The diagram illustrates the principle of feature pattern extraction and combination to form feature templates in some embodiments of the present invention; Figure 3 The diagram illustrates the principle of constructing sparse feature templates using the feature discretization method in some embodiments of the present invention. Figure 4 The diagram illustrates the principle of multi-layer independent construction of feature templates and weighted fusion of multi-channel matching results in some embodiments of the present invention. Detailed Implementation
[0030] It should be noted that the components in the accompanying drawings may be shown exaggerated for illustrative purposes and may not be to scale.
[0031] In this invention, the various embodiments are merely intended to illustrate the solutions of the invention and should not be construed as limiting.
[0032] In this invention, unless otherwise specified, the quantifiers “a” and “one” do not exclude scenarios involving multiple elements.
[0033] It should also be noted that, in the embodiments of the present invention, only a portion of the parts or components may be shown for clarity and simplicity. However, those skilled in the art will understand that, under the teachings of the present invention, the required parts or components can be added as needed for specific scenarios.
[0034] It should also be noted that within the scope of this invention, the terms "same", "equal", and "equal to" do not mean that the two values are absolutely equal, but allow for a certain reasonable error. In other words, the terms also cover "substantially the same", "substantially equal", and "substantially equal to".
[0035] It should also be noted that in the description of this invention, the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not explicitly or implicitly suggest that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0036] Furthermore, the embodiments of the present invention describe the process steps in a specific order. However, this is only for the convenience of distinguishing each step, and is not a limitation on the order of each step. In different embodiments of the present invention, the order of each step can be adjusted according to the process.
[0037] Before introducing specific embodiments, several core concepts in this invention will be explained first.
[0038] A feature graph is an image region with specific image content extracted by the user from an image. Unlike a traditional complete template, a feature graph is not a complete description of the entire target, but rather a capture of the target's local salient features. A feature graph can be a contour fragment, a via region, a printed symbol, a corner neighborhood, or any image region that the user considers distinctive.
[0039] A feature template is a whole composed of one or more feature graphics arranged according to their original spatial relative positions within the target. The feature graphics maintain their original spatial relationships within the feature template. In essence, a feature template is a combination of feature graphics—it consists only of these selected image patches and contains no other regions.
[0040] Feature discretization is a template construction strategy proposed in this invention. When the target has a complex shape, instead of requiring the user to extract the complete outline or all of the target's appearance, it allows the user to selectively extract only some representative regions as feature graphics. These feature graphics can be discretely distributed in space. The core idea of feature discretization is to "capture key local areas and ignore irrelevant regions," thereby significantly reducing the complexity of the template and the amount of matching computation while ensuring matching accuracy.
[0041] The following embodiments provide an image target localization method based on custom feature template matching. Figure 1 This diagram illustrates the overall flowchart of an image target localization method based on custom feature template matching. The method includes the following steps: Acquire an image containing the target to be located; Extract at least one feature graph containing local information about the target from the image; The extracted feature graphics are arranged according to their spatial relative positions in the target to form a feature template. For example, Figure 2 The diagram illustrates the principle of feature extraction and combination to form a feature template, that is, it shows the process of feature extraction and feature template combination. (a) is an image containing the target to be located, with three local regions A, B, and C marked in the image to be selected; (b) are feature graphics A, B, and C extracted from the image; (c) is the feature template formed after combining the feature graphics while maintaining their original spatial relative positions. Identify the feature types of the feature images and preprocess the images according to the feature types; The feature template is traversed across the image, and the correlation between the feature template and the image is calculated to obtain a matching curve or surface plot. The dimension of the curve or surface plot depends on the dimension of the feature template sliding across the image; and Based on the maximum matching degree in the curve or surface plot, the position of the target in the image is determined. This can be achieved by combining the pixel coordinates corresponding to the maximum matching degree with the size of the feature template to determine the bounding box of the target, or by performing subpixel interpolation on the position of the maximum matching degree to obtain the target position with subpixel precision, and outputting the position and rotation angle of the target in the image.
[0042] Feature graphics are image patches defined based on the target and capable of representing the local shape, texture, or color information of the target. In other words, a feature graphic is a specific image patch selected by the user from an image; it is an image region with specific image content, representing the local salient information of the target. The feature types of feature graphics include: A complete or partial outer contour segment of the target profile; Through-holes, grooves, or raised areas inside the target; Printed patterns, text symbols, or specific texture blocks on the target surface; The local image patch consisting of the corners, edge intersections, and their neighborhoods of the target.
[0043] When the target has a complex shape, a feature discretization method is used to extract and combine feature graphics, including the following steps: Selectively extract only some feature patterns (representative regions) from the complex shape of the target; and The extracted feature graphics are arranged while preserving their original spatial relationships, and then combined to form a feature template that contains only the selected feature graphics. Figure 3The diagram illustrates the principle of constructing a sparse feature template using the feature discretization method. The left side shows the original image containing a complex target, from which representative local feature graphics A, B, and C are selectively extracted. The right side shows the sparse feature template obtained after preserving the original spatial arrangement of each feature graphic.
[0044] Extracting at least one feature graph containing local information of a target from an image includes: Separate the image into multiple single-color-channel images, and extract feature patterns from at least one single-color-channel image; or Multiple color channels of an image are weighted and fused with different weights to generate a fused image. Feature graphics are then extracted from the fused image. The weight coefficients used for weighted fusion with different weights are calculated based on the color characteristics of the target and with the criterion of maximizing the contrast between the target area and the background area in the fused image, so that the layer containing the feature graphics can highlight the difference between the target and the background.
[0045] Different feature templates are defined for different layers, and each feature template contains at least one feature graphic extracted from the same layer; Each feature template corresponds to one channel. The correlation between the feature template of each channel and the image is calculated. Then, the matching degrees calculated for each channel are weighted and fused to generate a matching degree curve or surface map. The matching degree is calculated for the pixel region covered by the feature graphics in the feature template to generate a matching degree curve or matching degree surface map. Figure 4 This diagram illustrates the principle of independently constructing feature templates across multiple layers and weighted fusion of multi-channel matching results. Specifically, it shows the process of extracting feature graphics from different layers and constructing multi-channel feature templates. The top section shows the processing flow for the first layer (color fusion layer): multi-channel weighted fusion of the original RGB image yields a fused image; feature graphics A and B are extracted to construct the first feature template; and the first channel matching is completed to obtain the first channel matching degree. The bottom section shows the processing flow for the second layer (R channel layer): the R channel image is separated from the original RGB image; feature graphic C is extracted to construct the second feature template; the second channel matching is completed to obtain the second channel matching degree; and the first and second channel matching degrees are weighted and fused to obtain the final matching degree curve or surface plot.
[0046] Identifying the feature types of feature images and preprocessing the image according to the feature types includes: When the feature graphics are represented as contours or edge features, preprocessing includes image sharpening, edge detection, or binarization. When the feature image exhibits texture features, preprocessing includes histogram equalization or contrast stretching of the image; and When the feature image originates from a specific color layer, preprocessing includes weighted fusion of the corresponding color channels of the image. This deterministic mapping between feature type and preprocessing operation ensures that preprocessing always serves the needs of subsequent matching.
[0047] The matching degree is calculated based on the pixel values of the feature template and the pixel values of the corresponding region in the image, using a preset similarity function or distance function. A higher similarity function value indicates a higher degree of matching, while a lower distance function value also indicates a higher degree of matching. The similarity function or distance function is calculated only based on the pixel region covered by the feature graphic in the feature template, and at least one of the following similarity functions or distance functions may be used: Normalized cross-correlation: Calculate the normalized cross-correlation coefficient between the pixel value of the feature template and the pixel value of the corresponding region in the image. Its value ranges from [-1, 1]. A value of 1 indicates a perfect match, and a value of -1 indicates a perfect anti-correlation. Zero-mean normalized cross-correlation: Subtract the mean of the pixel values of the feature template and the mean of the pixel values of the corresponding region of the image, and calculate the normalized cross-correlation coefficient of the two sets of values obtained after subtracting the mean. Sum of Squared Differences: Calculates the sum of squares of the differences between the pixel values of the feature template and the pixel values of the corresponding regions in the image. The smaller the value, the higher the degree of matching. Sum of absolute differences: Calculate the sum of the absolute differences between the pixel values of the feature template and the pixel values of the corresponding region in the image. The smaller the value, the higher the degree of matching. Matrix dot product: The pixels of the feature template are arranged in order to form a template vector, and the pixels of the corresponding regions of the image are arranged according to the same rules to form a feature vector. The dot product of the template vector and the feature vector is calculated; a larger value indicates a higher degree of matching. Since the feature template is only composed of a combination of feature graphics, the calculation of the similarity function or distance function is based on the pixel region covered by the feature template, thereby reducing invalid computation and avoiding interference from non-feature regions. When using matrix dot product, the vectors can be binarized first, simplifying multiplication to logical AND operations and addition to bit counting operations, greatly improving computational efficiency. When using matrix dot product, each element of the template vector and / or feature vector is first binarized, converting pixel values to 0 or 1 according to a preset threshold; when calculating the dot product of the binarized vectors, multiplication is replaced by logical AND operations, and addition is replaced by bit counting operations.
[0048] When the target may rotate, a template representation corresponding to multiple rotation angles is generated based on the feature template. When the feature template is slid across the image, the correlation calculation results under multiple rotation angles are combined to generate a matching degree curve or surface map, and the optimal matching angle and position of the target are determined.
[0049] The following three specific embodiments comprehensively illustrate the implementation process and technical effects of the method of the present invention from the perspectives of different target types, different feature combinations, different calculation methods, and different accuracy requirements.
[0050] Example 1 – Positioning of a rectangular transparent glass plate in a chip mounter This embodiment uses the positioning of a rectangular transparent glass sheet in a pick-and-place machine as an example. The transparent glass sheet is visually characterized by low contrast and high light transmittance, with its most prominent feature being its edge contour. In this embodiment, the complete contour of the glass sheet is used as the feature image, focusing on demonstrating the template construction based on the complete contour feature, edge enhancement preprocessing, and correlation calculation process, as well as the pose optimization method combining rotation matching and iterative search.
[0051] Step S1: Acquire the image.
[0052] An industrial camera on the pick-and-place machine captures an image of the transparent glass sheet to be mounted, obtaining a color image in RGB format. This image contains the entire transparent glass sheet and its background area.
[0053] Step S2: Extract feature graphics and construct feature templates.
[0054] The transparent glass sheet is rectangular, and its most prominent feature is its edge contour. The complete rectangular contour of the glass sheet is extracted from the image as a feature graphic. Specifically, the contour range is determined based on the actual physical dimensions of the glass sheet: the outer frame has a long side of 1200 pixels and a short side of 690 pixels. Based on the expected width of the contour, the inner frame has a long side of 1150 pixels and a short side of 620 pixels. The annular region inside the outer frame and outside the inner frame is used as the feature graphic. This feature graphic is a complete outer contour segment of the target contour.
[0055] The extracted feature graphics are arranged according to their original spatial relationships within the target to form a feature template. Since this embodiment uses only one feature graphic, the feature template is the feature graphic itself.
[0056] To further improve matching robustness, the feature image can be transformed into an edge distance field. First, a distance transformation is performed on the feature image, the distance from each pixel to the nearest boundary is calculated, and Gaussian smoothing is applied to pixels near the boundary to generate a floating-point feature template with edge weights.
[0057] Step S3: Preprocess the image.
[0058] Since the feature graphics are represented by contour edge features, edge enhancement preprocessing is performed on the image based on the correspondence between feature type and preprocessing operation.
[0059] First, the color image is decomposed into three monochrome channels: R, G, and B. Based on the spectral reflectance characteristics of the glass slide outline edge and the background, weighting coefficients are determined for each color channel to maximize the grayscale difference between the target rectangle outline edge and the background. For example, k_R = 1.1, k_G = 0.3, and k_B = -0.4. Channel fusion is then performed using the linear weighting formula I = k_R•R + k_G•G + k_B•B.
[0060] The weighted and fused image is then subjected to median filtering to remove potential salt-and-pepper noise while preserving edge sharpness. Binarization is then performed, setting a grayscale threshold (e.g., TH=190). Pixels with grayscale values greater than or equal to the threshold are assigned a value of 255, while pixels with grayscale values less than the threshold are assigned a value of 0. A logical inversion operation is then performed, making the contour edge regions white foreground and other regions black background, resulting in a binarized feature image.
[0061] Step S4: Perform correlation calculations to generate a matching degree curve or surface plot.
[0062] In this embodiment, since the target may undergo translation and rotation changes, the correlation calculation adopts a combination of rotation matching and iterative search.
[0063] First, set the initial pose parameters: place the center of the feature template at the center of the image, set the initial rotation angle to 0°, and the initial translation offset to (0,0).
[0064] Perform a translation search: Set the translation search range (e.g., ±100 pixels) and step size (e.g., 10 pixels), and cyclically move the feature template in the X and Y directions. For each translation position, calculate the normalized overlap matching score between the template and the feature image to form a score matrix. Perform edge-preserving filtering and smoothing (e.g., guided filtering) on the score matrix, and extract the position corresponding to the maximum value from the smoothed score matrix as the optimal translation offset.
[0065] Perform rotation search: Fix the above optimal translation offset, and set the rotation angle search range (e.g., ±2°) and step size (e.g., 0.5°). Generate rotation templates with different angles based on the feature templates, calculate the score of each rotation template with the feature image, and take the angle corresponding to the maximum value as the optimal rotation angle.
[0066] To obtain higher accuracy pose parameters, an iterative optimization strategy is adopted: the currently obtained optimal translation offset and optimal rotation angle are used as new initial values, the translation search range is reduced (e.g., ±10 pixels), and the angle search range is reduced (e.g., ±0.5°), and the translation and rotation searches are repeated. After multiple iterations (e.g., 3 to 5 times), the pose parameters converge to the preset accuracy. The matching degree response of each iteration is combined to obtain the final matching degree curve or surface plot.
[0067] Optionally, the roundness of the outer contour can be calculated before positioning begins. If the roundness is below a preset threshold, it indicates that the target is a non-rotationally symmetric shape, and a complete translation and rotation search is performed; if the roundness is above the preset threshold, it indicates that the target is nearly circular, and the rotation search can be skipped, and the translation search result can be directly output to improve computational efficiency.
[0068] Step S5: Determine the target location.
[0069] After the above iterative optimization, the maximum value in the matching curve or surface plot corresponds to the optimal matching position of the target. The pixel coordinates corresponding to this maximum value are extracted as the center coordinates of the target. Combined with the size of the feature template, the target bounding box is determined, and the corresponding rotation angle is output. The center coordinates and rotation angle are superimposed and plotted on the original image to display the matching position of the feature template and the center marker, completing the visualization output.
[0070] Example 2 – IC Chip Positioning in a Pick and Place Machine This embodiment uses the positioning of an IC chip in a pick-and-place machine as an example. Unlike the transparent glass sheet in Embodiment 1, the surface of an IC chip has abundant local features, such as surface silkscreen markings and pin pads. These features are distributed at different locations on the chip surface, making them naturally suitable for selection using the feature discretization method. This embodiment focuses on demonstrating the application of the feature discretization method and the advantages of matching based solely on feature patterns in reducing computational load.
[0071] Step S1: Acquire the image.
[0072] The industrial camera on the pick-and-place machine captures images of the IC chip to be mounted, obtaining color images in RGB format.
[0073] Step S2: Extract feature graphics and construct feature templates.
[0074] The surface of an IC chip has many significant features. In this embodiment, the feature discretization method is used to selectively extract a portion of representative areas from the complex surface features of the chip as feature patterns.
[0075] Feature A: The surface silkscreen marking area in the upper left corner of the chip. This silkscreen marking is a white printed symbol, creating high contrast against the dark background of the chip package surface. This feature is a printed pattern on the target surface.
[0076] Feature B: A pin pad area in the lower right corner of the chip. The pad has a distinct metallic luster and a regular geometric shape. This feature is a local outer contour segment of the target profile.
[0077] Feature Pattern C: A specific texture block in the middle of the chip. This region contains minute texture structures on the chip surface. The type of this feature pattern is a specific texture block of the target surface.
[0078] The three feature patterns are arranged according to their original spatial relationships on the chip. Feature pattern A is located in the upper left of the template, feature pattern B is located in the lower right of the template, and feature pattern C is located in the center of the template. After combination, a feature template containing only three feature patterns is formed.
[0079] Step S3: Preprocess the image.
[0080] Since the three feature images represent different feature types, differential preprocessing is performed on the image based on the correspondence between feature types and preprocessing operations.
[0081] For feature graphic A (printed pattern, which is a texture feature), histogram equalization is performed on the image to enhance the contrast between the silkscreen symbol on the chip surface and the background, making the outline of the white silkscreen mark clearer.
[0082] For feature graphic B (pad edge, represented by contour feature), image sharpening processing is performed to enhance the gradient response of the pad edge, making the geometric contour of the pad more prominent.
[0083] For the feature graphic C (surface texture, represented by texture features), the image is subjected to contrast stretching to enhance the local contrast of the texture area and make the micro-texture structure more obvious.
[0084] After the above preprocessing, a comprehensive preprocessed feature image is obtained, in which silkscreen markings, pad edges and surface texture information are all enhanced in a targeted manner.
[0085] Step S4: Perform correlation calculations to generate a matching degree curve or surface plot.
[0086] The feature template is slid across the preprocessed feature image. Since the feature template consists of only three feature patterns, the matching degree is calculated based on the pixel regions covered by these three feature patterns. The matching degree is calculated using normalized cross-correlation; the template is slid pixel by pixel, and the normalized cross-correlation coefficient between the pixel values of the regions covered by the feature patterns and the corresponding pixel values of the feature image is calculated. Compared to using the entire chip region as the template, the number of pixels involved in the calculation is significantly reduced, while avoiding interference from accidental matches between unselected areas of the target and the background image on the matching degree response.
[0087] When the IC chip may rotate, template representations corresponding to multiple rotation angles are generated based on the feature template. When the feature template is slid across the image, the correlation calculation results under multiple rotation angles are combined to generate a matching degree curve or surface plot, and the optimal matching angle of the target is determined at the same time.
[0088] Step S5: Determine the target location.
[0089] Find the maximum matching degree value in the matching degree curve or surface plot. The pixel coordinates corresponding to this maximum value are the center position of the IC chip. Combined with the overall size of the feature template, determine the chip's bounding box and output the corresponding rotation angle. For the high precision requirements of the pick-and-place machine, perform sub-pixel interpolation on the neighborhood near the maximum value position in the matching degree curve or surface plot to obtain the target position with sub-pixel precision.
[0090] This embodiment adopts... Figure 3 The feature discretization method shown constructs a sparse feature template, selecting only three representative local regions and not using the complete chip image as a template, effectively reducing the number of pixels involved in the calculation.
[0091] Example 3 – Rapid and High-Precision Positioning of PCB Boards This embodiment uses the positioning of a PCB board in a pick-and-place machine as an example. PCB boards are typically made of green substrate, with copper pads and traces on the surface, and white silkscreen markings. In mass production scenarios, PCB boards of the same model have fixed features, requiring high positioning accuracy and speed. This embodiment comprehensively demonstrates the complete application of feature discretization, multi-layer independent template construction, multi-channel matching fusion, and binarized matrix inner product acceleration.
[0092] Step S1: Acquire the image.
[0093] The industrial camera on the pick-and-place machine captures images of the PCB board to be placed, obtaining a color image in RGB format. Since the PCB board is fixed in the pick-and-place machine fixture by positioning pins, the rotational deviation is extremely small (usually less than 0.5°). This embodiment can skip the rotation search step and only perform a translation search to improve positioning speed.
[0094] Step S2: Extract feature images and construct multi-channel feature templates.
[0095] The PCB board has several distinct features of different types. The copper pads on the green substrate are golden yellow, and the white silkscreen markings contrast sharply with the green substrate. Positioning holes are also distributed across the board. In this embodiment, feature graphics are extracted and independent feature templates are constructed based on different layers.
[0096] Feature graphics A and B in the first layer (color blending layer): These feature graphics perform weighted blending of multiple color channels of the image. The weighting coefficients are determined based on the maximization of contrast between the green substrate, copper pads, and white silkscreen (e.g., R: 0.3, G: 0.8, B: 0.2), generating a blended image. In this blended image, the response values of the copper pads (golden yellow) and the white silkscreen markings are significantly higher than those of the green substrate background. Feature graphics A, extracted from the blended image, represents a copper pad area in the upper right corner of the PCB board, and feature graphics B represents a white silkscreen marking area in the lower left corner of the PCB board.
[0097] Feature graphic C in the second layer (R channel image): After separating the image into the R monochrome channel image, feature graphic C is extracted as a positioning hole area in the middle of the PCB board. This positioning hole is a circular through hole, which appears as a dark circle in the R channel image, forming a clear contrast with the surrounding green substrate.
[0098] Feature graphics A and B are arranged according to their original spatial relationship on the PCB board to form a first feature template, which contains two feature graphics extracted from the color blending layer. Feature graphics C is arranged according to its original spatial position to form a second feature template, which contains one feature graphic extracted from the R channel layer. The first and second feature templates correspond to the first and second channels, respectively.
[0099] Step S3: Preprocess the image.
[0100] The feature patterns A (copper pads) and B (silk screen markings) corresponding to the first channel are both color features. Based on the correspondence between feature types and preprocessing operations, the image undergoes weighted fusion processing of the corresponding color channels (using the same weighting coefficients R:0.3, G:0.8, B:0.2 as in step S2) to generate the first channel preprocessed image. In this image, the pads and silk screen areas obtain higher response values.
[0101] The feature pattern C (positioning hole) corresponding to the second channel is represented by contour features. Edge enhancement preprocessing is performed on the R channel of the image. Gaussian filtering (5×5 window, standard deviation σ=1.0) is applied to the R channel image to suppress image acquisition noise. Then, Canny edge detection (low threshold 50, high threshold 150) is performed to extract the circular edge of the positioning hole and generate the preprocessed image of the second channel.
[0102] Step S4: Perform correlation calculations to generate a matching degree curve or surface plot.
[0103] The first feature template is slid across the first channel preprocessed image, and the second feature template is slid across the second channel preprocessed image, and correlation calculations are performed respectively.
[0104] Due to the high speed requirements of PCB mass production, the matching degree is calculated using the matrix inner product method. Pixels in the area covered by the feature image are arranged sequentially to form a template vector, and pixels in the corresponding area of the image are arranged according to the same rules to form a feature vector. The inner product of the template vector and the feature vector is then calculated. Before calculating the matrix inner product, each element of the template vector and the feature vector is binarized, converting pixel values to 0 or 1 according to a preset threshold. When calculating the binarized vector inner product, multiplication is replaced by logical AND operations, and addition is replaced by bit counting operations, significantly improving computational efficiency compared to floating-point multiplication.
[0105] After obtaining the matching degree responses of the first and second channels respectively, the matching degrees calculated from the two channels are weighted and fused (e.g., the first channel has a weight of 0.5 and the second channel has a weight of 0.5) to generate the final matching degree curve or surface plot.
[0106] Step S5: Determine the target location.
[0107] The maximum matching degree value is found in the matching degree curve or surface map, and the pixel coordinates corresponding to the maximum value are obtained. Since the pick-and-place machine requires sub-pixel accuracy, a quadratic surface fitting is performed on the 3×3 neighborhood near the maximum value position in the matching degree curve or surface map to obtain the target position with sub-pixel accuracy. Finally, the sub-pixel accuracy target position is output, and combined with the overall size of the feature template, the bounding box of the PCB board is determined for the pick-and-place machine to perform subsequent placement operations.
[0108] This embodiment adopts Figure 4 The multi-layer independent template construction scheme shown constructs a first feature template corresponding to the color fusion layer and a second feature template corresponding to the R channel layer. The two templates independently complete the correlation calculation and then the matching degree is weighted and fused.
[0109] Comparison and summary of the three embodiments The above three embodiments illustrate the implementation methods and technical effects of the present invention from different perspectives: Regarding the selection of feature graphics, Example 1 uses the complete outline as the feature graphics, which is suitable for scenarios where the target has a single significant outline feature; Example 2 uses the feature discretization method to select three different types of local feature graphics from the surface of the IC chip, demonstrating the construction process of the sparse template; Example 3 extracts feature graphics from different layers (fusion layer and R channel layer) to construct multi-channel independent feature templates, demonstrating the complete process of multi-layer independent template construction and multi-channel matching and fusion.
[0110] In terms of preprocessing, Example 1 performs channel-weighted fusion and edge enhancement on contour edge features; Example 2 performs histogram equalization, image sharpening, and contrast stretching on different types of feature graphics (texture, contour, texture); Example 3 performs color channel fusion and edge extraction preprocessing on different channel feature graphic types (color, contour). All three examples demonstrate a clear correspondence between feature types and preprocessing operations.
[0111] Regarding correlation calculation, Example 1 uses a combination of rotation matching and iterative search, gradually converging to the accurate pose through translation search, rotation search, and iterative optimization, and using roundness judgment to optimize the search strategy; Example 2 only calculates the matching degree of the feature image coverage area, combined with normalized cross-correlation, to reduce the amount of computation and avoid interference from non-feature areas; Example 3 only calculates the matching degree of the feature image coverage area, combined with binarized matrix inner product and multi-channel fusion, which greatly improves the calculation speed and is suitable for the high real-time requirements of mass production.
[0112] In terms of accuracy, Example 1 obtains accurate pose parameters through iterative optimization, and outputs the target's position and rotation angle at the same time; Examples 2 and 3 obtain the target position with sub-pixel accuracy through sub-pixel interpolation (neighborhood quadratic surface fitting), which meets the high-precision placement requirements of the pick-and-place machine.
[0113] Therefore, the method of the present invention can flexibly select the feature image extraction method, preprocessing operation and matching degree calculation method according to different target types, different feature combinations, different accuracy requirements and different speed requirements, and has wide applicability and good technical effect.
[0114] While some embodiments of the present invention have been described in this application, those skilled in the art will understand that these embodiments are merely illustrative. Numerous variations, alternatives, and improvements will arise in those skilled in the art under the teachings of this invention without departing from its scope. The appended claims are intended to define the scope of the invention and thereby cover methods and structures within the scope of the claims themselves and their equivalents.
Claims
1. An image target localization method based on custom feature template matching, characterized in that, Includes the following steps: Acquire an image containing the target to be located; Extract at least one feature graphic containing local information of the target from the image; The extracted feature graphics are arranged according to their spatial relative positions in the target to form a feature template; The feature template is traversed across the image, and the correlation between the feature template and the image is calculated to obtain a matching curve or surface plot. as well as The position of the target in the image is determined based on the maximum matching degree in the curve or surface plot.
2. The image target localization method based on custom feature template matching according to claim 1, characterized in that, The feature graphic is an image patch defined according to the target, capable of representing the local shape, texture, or color information of the target; the feature types of the feature graphic include: A complete or partial outer contour segment of the target profile; Through-holes, grooves, or raised areas inside the target; Printed patterns, text symbols, or specific texture blocks on the target surface; The local image patch consisting of the corners, edge intersections, and their neighborhoods of the target.
3. The image target localization method based on custom feature template matching according to claim 1, characterized in that, When the target has a complex shape, a feature discretization method is used to extract and combine the feature graphics, including the following steps: Selectively extract partial feature patterns from the complex shape of the target; and The extracted feature graphics are arranged while preserving their original spatial relationships, and then combined to form a feature template that contains only the selected feature graphics.
4. The image target localization method based on custom feature template matching according to claim 1, characterized in that, Extracting at least one feature graph containing local information of the target from the image includes: The image is separated into multiple single-color-channel images, and the feature pattern is extracted from at least one of the single-color-channel images; or The multiple color channels of the image are weighted and fused with different weights to generate a fused image, and the feature graphic is extracted from the fused image; wherein, the weight coefficients used in the weighted fusion with different weights are calculated based on the color features of the target, with the criterion of maximizing the contrast between the target area and the background area in the fused image.
5. The image target localization method based on custom feature template matching according to claim 1, characterized in that, Different feature templates are defined for different layers, and each feature template contains at least one feature graphic extracted from the same layer; Each feature template corresponds to a channel. The feature template of each channel is correlated with the image, and the matching degree calculated by the channel is weighted and fused to generate the matching degree curve or surface map.
6. The image target localization method based on custom feature template matching according to claim 1, characterized in that, Before calculating the correlation between the feature template and the image, the method further includes: Identifying the feature type of the feature image and preprocessing the image according to the feature type includes: When the feature graphic represents contour or edge features, the preprocessing includes image sharpening, edge detection, or binarization of the image. When the feature image exhibits texture features, the preprocessing includes histogram equalization or contrast stretching of the image; and When the feature image originates from a color layer, the preprocessing includes weighted fusion processing of the corresponding color channels of the image.
7. The image target localization method based on custom feature template matching according to claim 1, characterized in that, The matching degree is calculated based on the pixel values of the feature template and the pixel values of the corresponding region of the image, using a preset similarity function or distance function; wherein, a larger similarity function value indicates a higher degree of matching, and a smaller distance function value indicates a higher degree of matching.
8. The image target localization method based on custom feature template matching according to claim 7, characterized in that, The similarity function or distance function adopts at least one of the following: Normalized cross-correlation: Calculate the normalized cross-correlation coefficient between the pixel value of the feature template and the pixel value of the corresponding region of the image. The value range is [-1, 1]. A value of 1 indicates a perfect match, and a value of -1 indicates a perfect anti-correlation. Zero-mean normalized cross-correlation: Subtract the mean of the pixel values of the feature template, subtract the mean of the pixel values of the corresponding region of the image, and calculate the normalized cross-correlation coefficient of the two sets of values obtained after subtracting the mean. Sum of squared differences: Calculate the sum of squares of the differences between the pixel values of the feature template and the pixel values of the corresponding region of the image. The smaller the value, the higher the degree of matching. Sum of absolute differences: Calculate the sum of the absolute differences between the pixel values of the feature template and the pixel values of the corresponding region of the image. The smaller the value, the higher the degree of matching. Inner product of matrices: The pixels of the feature template are arranged in order to form a template vector, and the pixels of the corresponding region of the image are arranged according to the same rule to form a feature vector. The inner product of the template vector and the feature vector is calculated. The larger the value, the higher the degree of matching.
9. The image target localization method based on custom feature template matching according to claim 1, characterized in that, When the target may rotate, a template representation corresponding to multiple rotation angles is generated based on the feature template. When the feature template is slid on the image, the correlation calculation results under the multiple rotation angles are combined to generate a matching degree curve or surface map, and the optimal matching angle and position of the target are determined.
10. The image target localization method based on custom feature template matching according to claim 1, characterized in that, The step of determining the position of the target in the image based on the maximum matching degree in the curve or surface plot involves combining the pixel coordinates corresponding to the maximum matching degree with the size of the feature template to determine the bounding box of the target, or performing subpixel interpolation on the position of the maximum matching degree to obtain the target position with subpixel precision, and outputting the position and rotation angle of the target in the image.
