A real-time template image matching method using image gradient

Abstract

The invention discloses a real-time template image matching method using image gradients. The method comprises the following steps: calculating the gradient amplitude and gradient direction of an input image, quantizing the gradient direction, retaining only the pixel points whose gradient amplitude exceeds a certain threshold as feature points, using a logical OR operation to represent the existence of the gradient direction in a neighborhood for each quantized gradient direction, thereby encoding the gradient information in the neighborhood into a binary string; calculating the similarity between each template and all gradient directions in the neighborhood of a current pixel, using a pre-calculated similarity lookup table to quickly obtain the maximum similarity value of each direction, accumulating the maximum values ​​to obtain the response strength of the current pixel point to the template, and quickly calculating the matching score of each template in the entire image according to a response graph and pre-calculated gradient information, and identifying the position of the object by setting a threshold. The invention can significantly improve the matching speed.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and pattern recognition, and specifically relates to a real-time template image matching method using image gradients, which is particularly suitable for application scenarios such as robot vision, security monitoring, augmented reality and self-driving cars, and is used in various real-time template image matching based on visible light images. Background Art

[0002] As the cornerstone of computer vision, image matching plays an irreplaceable role in achieving a deep understanding of the visual world by machines. It involves not only extracting information from images, but also comparing this information with existing data to identify and locate specific objects or scenes in the image. Image matching technology plays a key role in many industries, including but not limited to robot navigation, augmented reality, medical imaging analysis, security monitoring, and self-driving cars. In robot navigation, image matching helps robots understand their surroundings and achieve accurate path planning and obstacle avoidance. In augmented reality (AR), accurate image matching ensures that virtual images can be seamlessly superimposed with the real world to provide an immersive experience. In medical imaging analysis, image matching technology assists doctors in evaluating changes in disease or treatment effects by comparing medical images at different time points. Therefore, the development of image matching technology is of great significance to promoting technological progress in related fields.

[0003] The development of image matching algorithms has gone through several stages, from early feature-based matching methods such as SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features) to region-based matching methods such as mean shift and direct methods. Each method has its own specific application scenarios and advantages. SIFT and SURF algorithms achieve a robust description of image content by extracting key points in the image and calculating their feature vectors, and can perform effective matching even under scale changes, rotation or lighting conditions. Region-based matching methods focus on the entire area of ​​the image and match by calculating the similarity between regions. For example, the mean shift algorithm uses the color distribution information of the image block for matching, while the direct method tracks objects by directly calculating the optical flow field on the image.

[0004] However, these classic methods often encounter challenges when dealing with textureless or less textured objects. Textureless objects lack obvious feature points, making it difficult for feature-based methods to extract effective descriptors. In addition, region-based matching methods may be affected by illumination changes, occlusion, and image noise, resulting in reduced matching accuracy. More importantly, the real-time limitations of these methods make it difficult for them to meet application scenarios that require fast feedback. For example, although SIFT and SURF algorithms are powerful, they have high computational costs and are not suitable for scenarios that require real-time processing of large amounts of image data. Mean shift and direct methods also have limitations in processing speed, especially when the image resolution is high or the image content is complex, the calculation time will increase significantly. Summary of the invention

[0005] In view of this, the present invention provides a real-time template image matching method using image gradients, which can quickly skip a large number of pixels that do not contain useful information, thereby significantly improving the matching speed.

[0006] The technical solution for implementing the present invention is as follows:

[0007] A real-time template image matching method using image gradients calculates the gradient magnitude and gradient direction of an input image, quantizes the gradient direction, and only retains pixels whose gradient magnitude exceeds a certain threshold as feature points. For each quantized gradient direction, a logical OR operation is used to indicate the existence of the gradient direction in a neighborhood, thereby encoding the gradient information in the neighborhood into a binary string. For each template, its similarity with all gradient directions in the neighborhood of a current pixel is calculated, and a pre-calculated similarity lookup table is used to quickly obtain the maximum similarity value of each direction. These maximum values ​​are accumulated to obtain the response strength of the current pixel to the template. For each template, its matching score in the entire image is quickly calculated based on a response map and pre-calculated gradient information, and the position of the object is identified by setting a threshold.

[0008] Furthermore, for RGB color images, the following steps are used to determine the gradient direction:

[0009] 1. For each pixel in the image, calculate its gradient magnitude and direction in the three color channels of red (R), green (G), and blue (B).

[0010] 2. By comparing the gradient amplitudes of the three channels, select the channel with the largest gradient amplitude.

[0011] 3. Use the gradient direction of this channel as the final gradient direction of the pixel.

[0012] Furthermore, when extracting feature points, non-maximum suppression (NMS) is applied to remove duplicate feature points in the neighborhood to avoid excessive concentration of feature points in the local area, and a minimum distance between feature points is specified to increase the distribution uniformity of feature points.

[0013] Furthermore, the memory layout of the response graph is optimized to make it more suitable for the cache mechanism of modern CPUs. The specific measures include:

[0014] The response map is stored linearly in memory by row, ensuring that the data of adjacent pixels are stored in adjacent memory locations;

[0015] The SIMD (Single Instruction Multiple Data) instruction set is used to process data of multiple pixels in parallel to further improve the calculation speed.

[0016] Furthermore, the point or connected area with the highest matching score is identified as the location of the object.

[0017] Beneficial effects:

[0018] This technical solution has the following advantages in implementation:

[0019] 1. Real-time performance: Through gradient expansion and linearized memory technology, the speed of the matching process is greatly improved to meet the needs of real-time processing.

[0020] 2. Robustness: The quantized gradient direction and neighborhood statistics strategies make the algorithm robust to changes in noise and image content.

[0021] 3. Generalization capability: By extracting and matching image gradients, the technical solution can generalize to a variety of different objects and scenes, including objects with no texture or simple texture.

[0022] 4. Resource efficiency: Pre-calculating similarities and using lookup tables reduces the amount of computation at runtime, allowing the algorithm to run on resource-constrained devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0024] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0025] The present invention provides a template image real-time matching method using image gradient. In order to compare the image to be matched with the template image, it is necessary to propose a quantitative similarity measurement index to illustrate the similarity between the template image and the real-time image. In the early stage, Steger proposed the following similarity measurement function:

[0026] ∈Steger(I,T,c)=r∈P∑∣cos(ori(O,r)-ori(I,c+r))∣

[0027] In the formula, ori(O,r) represents the direction of the gradient at position r in the template image O, and ori(I,c+r) represents the direction of the gradient at position r and offset c in the image I to be tested. cos represents the cosine value of the difference between the two directions. T is represented as a template, which includes the image O itself and the selected area P to be matched. P is selected from the most representative feature points in the image O. The matching similarity is calculated for all offsets c, and the offset c corresponding to the larger value is the matching target.

[0028] Among the optimization measures, the metric measures are extended to the neighborhood of each pixel to reduce the impact of noise and slight deformation and improve robustness:

[0029] E(I,T,c)=r∈P∑(maxt∈R(c+r)|cos(ori(O,r)-ori(I,c+r))|)where:

[0030] R(c+r)=[c+r-2T,c+r+2T]×[c+r-2T,c+r+2T]

[0031] like Figure 1 As shown, the present invention specifically comprises the following steps:

[0032] 1. Calculate image gradient

[0033] In image processing, gradient is a measure of the change in image intensity and is crucial for the detection of textureless objects. We first calculate the gradient magnitude and gradient direction of the input image. For RGB color images, we use the following steps to determine the gradient direction:

[0034] 1.1 For each pixel in the image, calculate its gradient magnitude and direction in the three color channels of red (R), green (G), and blue (B).

[0035] 1.2 By comparing the gradient amplitudes of the three channels, select the channel with the largest gradient amplitude.

[0036] 1.3 Use the gradient direction of this channel as the final gradient direction of the pixel.

[0037] This approach ensures that the contour information of objects in the image can be effectively captured even under changing lighting and complex backgrounds.

[0038] 2. Quantify the gradient direction

[0039] In order to make the representation of the gradient direction more compact and easier to process, we quantize it into a certain number of directions. The quantization strategy is as follows:

[0040] 2.1 Quantize the gradient direction into n0 discrete directions according to different angle ranges.

[0041] 2.2 In order to improve the robustness to noise, for the gradient direction of each pixel position, we use the direction with the largest number of statistical gradient directions in the 3×3 neighborhood. For example, if the number of directions “↑” is the largest in the 3×3 neighborhood, the gradient direction of the pixel is quantized to “↑”.

[0042] This quantization method not only simplifies the representation of gradients, but also enhances the algorithm's resistance to noise.

[0043] 3. Extract feature points

[0044] In an image, not all pixels contain feature information that is helpful for matching. Therefore, we only retain pixels whose gradient magnitude exceeds a certain threshold as feature points. In addition, in order to further reduce the amount of calculation, the following additional conditions are introduced: non-maximum suppression (NMS) is applied to remove duplicate feature points in the neighborhood to avoid excessive concentration of feature points in the local area. The minimum distance between feature points is specified to increase the uniformity of the distribution of feature points.

[0045] Through these strategies, we are able to extract the most representative and discriminative feature points from images.

[0046] 4. Gradient expansion

[0047] After gradient quantization, we expand the gradient information through a binary-like expression. This step involves expanding the gradient of pixels in the neighborhood, and the specific operation is as follows:

[0048] 4.1 For each quantized gradient direction, a logical OR operation is used to indicate whether the direction exists in the neighborhood.

[0049] 4.2 In this way, we can encode the gradient information in the neighborhood into a binary string, laying the foundation for subsequent fast matching.

[0050] The process of gradient expansion not only improves the expressiveness of gradient information, but also makes it possible to quickly calculate similarity.

[0051] 5. Calculate the response graph

[0052] The calculation of the response map is the core of this technical solution. Through gradient expansion, we can calculate the maximum value of the gradient direction similarity of all positions in the neighborhood of a point, which is equivalent to calculating the maximum value of the gradient direction similarity after the point is expanded. The calculation process is as follows:

[0053] 5.1 For each template, calculate its similarity with all gradient directions in the neighborhood of the current pixel.

[0054] 5.2 Use the pre-calculated similarity lookup table to quickly obtain the maximum similarity value in each direction.

[0055] 5.3 Add up these maximum values ​​to get the response intensity of the current pixel to the template.

[0056] The computation of the response map provides an efficient and robust similarity measure for template matching.

[0057] 6. Linearized Memory

[0058] In order to speed up the final matching calculation, we optimize the memory layout of the response graph to make it more suitable for the cache mechanism of modern CPUs. Specific measures include:

[0059] 6.1 The response map is stored linearly in memory by row, ensuring that the data of adjacent pixels are stored in adjacent memory locations.

[0060] 6.2 Use SIMD (Single Instruction Multiple Data) instruction set to process data of multiple pixels in parallel to further improve the calculation speed.

[0061] The linearization of memory and the application of SIMD instructions significantly improve the efficiency of matching calculations.

[0062] 7. Matching calculation

[0063] The final step is to use the linearized response map to perform template matching calculations. This step involves:

[0064] 7.1 For each template, quickly calculate its matching score in the entire image based on the linearized stored response map and the pre-computed gradient expansion information.

[0065] 7.2 By setting a threshold, determine which positions have a high enough matching score to identify the location of the object.

[0066] The efficiency of matching calculation is the key to achieve real-time detection.

[0067] Through the above technical solution, we can achieve fast and accurate real-time detection of textureless objects, which is especially suitable for application scenarios in complex backgrounds and dynamic environments. This solution not only improves computational efficiency, but also enhances the robustness of the algorithm through gradient quantization and feature point extraction.

[0068] Example 1

[0069] Step 1: Image gradient calculation

[0070] First, the gradient of the input RGB image is calculated. For each color channel, Sobel or similar gradient operators are used to calculate the gradient magnitude and direction of the image in the horizontal and vertical directions. Then, the direction of the color channel with the largest gradient magnitude is selected as the final gradient direction.

[0071] Sobel operator horizontal and vertical filters

[0072] Step 2: Gradient direction quantization

[0073] The calculated gradient direction is quantized into n0 discrete directions. In order to improve robustness, for each pixel, the gradient directions of all pixels in its 3×3 neighborhood are considered, and the direction with the most occurrences is selected as the quantized direction of the pixel.

[0074] Step 3: Feature point extraction

[0075] Extract feature points from the quantized image. Feature points are those points whose gradient magnitude exceeds a threshold. In order to reduce the amount of computation, non-maximum suppression (NMS) can be applied to remove local minimum or non-significant gradient points.

[0076] Step 4: Diffusion representation of gradient direction

[0077] Create a binary image representation J where the value of each pixel is determined by the set of gradient directions in its neighborhood. For each gradient direction in the neighborhood, if that direction appears in the 3×3 neighborhood, the corresponding bit in J is set to 1.

[0078] Step 5: Precompute the response map

[0079] For each template, precompute a response map S that represents the maximum similarity between each gradient direction in the template and the gradient direction at every possible location in the image. Use a lookup table to speed up this process.

[0080] Step 6: Linearize memory layout

[0081] In order to take advantage of the cache mechanism and SIMD instructions of modern CPUs, the precomputed response map SS is stored linearly, which means that the data of physically adjacent pixels in the image are also stored continuously in the memory.

[0082] Step 7: Matching Template

[0083] The linearized response map and the binary image representation J are used to quickly calculate the match between the template and the image. For each template, its matching score on the entire image is calculated by a simple addition operation.

[0084] Step 8: Determine the object position

[0085] The location of the object is determined based on the matching score. Usually, the point or connected area with the highest matching score is identified as the location of the object. You can set a threshold to filter out matches with lower scores to reduce false detections.

[0086] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A template image real-time matching method using image gradient, characterized in that: Calculate the gradient magnitude and gradient direction of the input image, quantize the gradient direction, and only retain the pixels whose gradient magnitude exceeds a certain threshold as feature points. For each quantized gradient direction, use the logical OR operation to indicate the existence of the gradient direction in the neighborhood, thereby encoding the gradient information in the neighborhood into a binary string. For each template, calculate its similarity with all gradient directions in the neighborhood of the current pixel, and use the pre-calculated similarity lookup table to quickly obtain the maximum similarity value of each direction. These maximum values ​​are accumulated to obtain the response strength of the current pixel to the template. For each template, quickly calculate its matching score in the entire image based on the response map and the pre-calculated gradient information, and identify the position of the object by setting a threshold.

2. The template image real-time matching method according to claim 1, characterized in that: For RGB color images, the following steps are used to determine the gradient direction:

1. For each pixel in the image, calculate its gradient amplitude and gradient direction in the three color channels of red, green and blue respectively; 2. By comparing the gradient amplitudes of the three channels, select the channel with the largest gradient amplitude; 3. Use the gradient direction of this channel as the final gradient direction of the pixel.

3. The template image real-time matching method according to claim 2, characterized in that: When extracting feature points, non-maximum suppression is applied to remove duplicate feature points in the neighborhood to avoid excessive concentration of feature points in the local area, and the minimum distance between feature points is specified to increase the uniformity of feature point distribution.

4. The template image real-time matching method according to claim 2 or 3, characterized in that: The memory layout of the response graph is optimized to make it more suitable for the cache mechanism of modern CPUs, including: The response map is stored linearly in memory by row, ensuring that the data of adjacent pixels are stored in adjacent memory locations; The SIMD instruction set is used to process data of multiple pixels in parallel to improve the calculation speed.

5. The template image real-time matching method according to claim 1, characterized in that: The point or connected region with the highest matching score is identified as the location of the object.

Images