Method for positioning image containing any target and background
The described image positioning method addresses sensitivity to illumination and transformation issues by scaling and correlating depth features, enhancing accuracy and reducing training time, thus improving image recognition across diverse conditions.
Patent Information
- Application Number
- CN202411287160.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-13
AI Technical Summary
Existing image positioning methods are sensitive to illumination changes, fail under angular rotation and affine transformation, and require lengthy training with poor generalization, leading to high matching errors and overfitting.
An image positioning method that scales template and search images using interpolation, extracts depth features with feature backbone networks, calculates correlation using Euclidean distance or other methods, and employs non-maximum suppression to achieve accurate matching results across arbitrary targets and backgrounds.
The method achieves robust image recognition under varying illumination and transformations, improves matching accuracy through multi-scale feature usage, and reduces training time by avoiding network model training.
Smart Images

Figure CN119090963A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image positioning, and in particular to an image positioning method including arbitrary targets and backgrounds. Background Art
[0002] Among the existing image positioning methods, there are the following methods: Method 1: Chinese patent application number 202111255618.9 provides a heterogeneous image matching and positioning method and system based on a twin network and supervised training. The method traverses the template image T to be matched in the search image I, compares pixel features, and then calculates correlation. However, this method fails if the illumination of the template image to be matched and the search image changes. Method 2: Chinese patent application number 202410202593.3 provides an image retrieval method and related device. It extracts features from template images T of multiple resolutions to obtain their respective multi-scale features FT. It then extracts features from the search image to obtain the search feature FI. The search feature FI is combined with the multi-scale feature FT for image matching to obtain the search result, thereby locating the position of the template image in the search image. However, in this method, the search image only outputs a single search feature, and the obtained search feature is single, resulting in a large matching error. Method 3: Chinese patent application number 202310359338.5 provides a heterogeneous image matching method based on a twin neural network, including (1) extracting features from the template image T to be matched and the search image I to obtain corresponding feature maps FT and FI; (2) traversing the obtained template feature FT on the search feature map FI, performing a correlation operation on each traversed position, and obtaining a matching mutual relationship map Mgen; (3) building a regression network Netreg, performing position regression on the matching correlation map Mgen, and outputting the relative position of the template image T to be matched in the search image I. Estimation; (4) constructing an ideal cross-correlation map Mide according to the actual position information as supervision information for matching the cross-correlation map; (5) building a discriminant network Netdis, constructing the discriminant loss and the overall generation loss function, and training the network; however, this method uses a regressive twin network for training, and the trained model has poor generalization for unknown images, resulting in overfitting; the template feature to be matched is only matched with a single search feature, and the single search feature does not contain rich semantic information, making it difficult for the network model to fit, and the model accuracy is low; and it takes a long time to train the model. Summary of the Invention
[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an image positioning method including arbitrary targets and backgrounds.
[0004] The object of the present invention is achieved through the following technical solutions: The present invention provides an image positioning method containing arbitrary targets and backgrounds, comprising: S1. Scale the template image by interpolation to obtain a first template image. The size of the original template image is , the size of the first template image is ; S2. Use the first model to infer the first template image to obtain the depth feature of the first template image. The size of the depth feature of the first template image is ; S3. Scale the search image by interpolation to obtain a first search image. The size of the original search image is , the size of the first search image is ; S4. Use the first model to infer the first search image to obtain the depth feature of the first search image. The size of the depth feature of the first search image is ; S5. Calculate the correlation between the depth feature of the first template image and the depth feature of the first search image, and select the result with the greatest correlation as the matching result. The calculation includes: The depth feature of the first template image is moved from left to right or from top to bottom in the depth feature of the first search image. During the movement, the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image is calculated. , selecting the depth feature of the first template image of the result with the greatest correlation as the first matching result, and mapping the coordinates of the depth feature of the first search image corresponding to the first matching result to the image coordinates of the first search image; S6. Repeat steps S1-S5 N times, replacing the first model with the Nth model in sequence, to obtain the Nth matching result; S7. Use non-maximum suppression to obtain the best matching result.
[0005] Furthermore, the interpolation method includes nearest neighbor interpolation, bilinear interpolation or bicubic interpolation.
[0006] Furthermore, the first model to the Nth model are feature backbone extraction networks trained with open source data or feature backbone networks for target detection and target segmentation trained with specific data.
[0007] Furthermore, the method of using the first model to infer the first search image or the first template image includes a combination of any one or more of a convolution operator, a residual operator, a multi-layer perceptron operator, a fully connected network operator or a Transformer operator.
[0008] Furthermore, the method for calculating the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image includes Euclidean distance: in an n-dimensional real vector space, A and B are two points in the n-dimensional real vector space, and the coordinates of point A are , the coordinates of point B are , the Euclidean distance between point A and point B is defined as: ; Wherein, d(A, B) represents the Euclidean distance between point A and point B, and the calculated Euclidean distance is used as the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image.
[0009] Furthermore, the method for calculating the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image also includes: dot product, cosine similarity or absolute value error.
[0010] The beneficial effects of the present invention are: 1) The present invention solves the problem of insensitivity to illumination and can still successfully recognize images even when they undergo angular rotation and affine transformation; 2) The present invention uses search image features at multiple scales to achieve greater matching accuracy; 3) The present invention does not require training of network models, has better generalization, and saves time costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 This is a flow chart of a method for locating an image containing arbitrary targets and backgrounds according to an embodiment of the present invention; Figure 2 A template image 1 of an image localization method including an arbitrary target and background according to an embodiment of the present invention; Figure 3 A search image 1 of an image positioning method including an arbitrary target and background according to an embodiment of the present invention; Figure 4 This is a second search image of a method for locating an image containing an arbitrary target and background according to an embodiment of the present invention; Figure 5 This is a second template image of an image localization method including arbitrary targets and backgrounds according to an embodiment of the present invention. DETAILED DESCRIPTION
[0012] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.
[0013] Illustratively, the flow chart of the present invention is as follows Figure 1 As shown, the present invention includes the following steps: a step of obtaining the depth features of the first template image, a step of obtaining the depth features of the first search image, and a step of obtaining the best matching result. In the present invention, there is no order of priority between the step of obtaining the depth features of the first template image and the step of obtaining the depth features of the first search image.
[0014] Specifically, we can first Figure 2 The template image 1 shown in FIG is processed to obtain the depth feature of the first template image of the template image 1, and then the template image 1 is processed. Figure 3 The search image 1 shown in FIG is processed to obtain the depth feature of the first search image of the search image 1, and finally the step of obtaining the best matching result is performed; the step of obtaining the depth feature of the first search image can also be performed first, and the step of obtaining the depth feature of the first search image is performed. Figure 4 The search image 2 shown in FIG is processed to obtain the depth feature of the first search image of the search image 2, and then the step of obtaining the depth feature of the first template image is performed. Figure 5 The template image 2 shown is processed to obtain the depth feature of the first template image of the template image 2, and finally the step of obtaining the best matching result is performed.
[0015] Specifically, the step of obtaining the depth feature of the first template image includes the following sub-steps: scaling the template image by an interpolation method, wherein the interpolation method includes nearest neighbor interpolation, bilinear interpolation or bicubic interpolation, etc., and any one of them can be selected. The size of the original template image is ; Corresponding to the width, height and number of layers of the original template image, the scaled first template image is obtained. The size of the first template image is , corresponds to the width, height and number of layers of the first template image, where the width Greater than 12, height is also greater than 12, and then the first template image is inferred through the first model. The first model can adopt a feature backbone extraction network trained with open-source data (such as: ResNet, Vit, DeiT, and Unet), or a feature extraction backbone network for object detection and object segmentation trained with specific data. Specific data training, for example: infrared and visible light. In time and coordinates, infrared and visible light correspond to data such as vehicles, airplanes, or houses. In this way, when training the model, both infrared and visible light are involved in training at the same time. The inference method can be a combination of one or more of the following operators: convolution operator, residual operator, multi-layer perceptron operator, fully connected network operator, or Transformer operator. Each open-source model contains different operators, so the calculation processes included are also different, thereby obtaining the depth features of the first template image. The size of the depth features of the first template image is , corresponding to the width, height, and number of layers of the depth features of the first template image.
[0016] Specifically, the steps for obtaining the depth features of the first search image include the following sub-steps: The search image is scaled through interpolation. The interpolation methods include nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation, etc. Any one of them can be selected. The size of the original search image is , corresponding to the width, height, and number of layers of the original search image, obtaining the scaled first search image. The size of the first search image is , corresponding to the width, height, and number of layers of the first search image, where the width is greater than 12, the height is also greater than 12. The size difference between the first template image and the first search image is X times, where 1 < X < the resolution of the original search image ; then the first search image is inferred through the first model. The first model can adopt a feature backbone extraction network trained with open-source data (such as: ResNet, Vit, DeiT, and Unet), or a feature extraction backbone network for object detection and object segmentation trained with specific data. Specific data training, for example: infrared and visible light. In time and coordinates, infrared and visible light correspond to data such as vehicles, airplanes, or houses. In this way, when training the model, both infrared and visible light are involved in training at the same time. The inference method can be a combination of one or more of the following operators: convolution operator, residual operator, multi-layer perceptron operator, fully connected network operator, or Transformer operator. Each open-source model contains different operators, so the calculation processes included are also different, thereby obtaining the depth features of the first search image. The size of the depth features of the first search image is , corresponding to the width, height, and number of layers of the depth features of the first search image.
[0017] Specifically, the step of obtaining the best matching result includes the following sub-steps: moving the depth feature of the first template image in the depth feature of the first search image, and the moving method can be from left to right or from top to bottom. During the moving process, the correlation result between the depth feature of the first template image and the depth feature of the first search image at the corresponding position is calculated, wherein the calculation method includes Euclidean distance, dot product, cosine similarity or absolute value error, etc., according to the width and height of the depth feature of the first template image. , select the depth feature of the first template image with the largest correlation as the first matching result, and map the coordinates of the depth feature of the first search image corresponding to the first matching result to the image coordinates of the first search image; then repeat the above steps, and change the first model to the Nth model, and finally obtain the Nth matching result; finally, obtain the best matching result through non-maximum suppression.
[0018] The non-maximum suppression method is: 1. Sorting: First, sort each bounding box in descending order based on its confidence (usually a comprehensive indicator of classification probability and positioning accuracy). The bounding box with the highest confidence is considered to be the most likely to have correctly detected the object.
[0019] 2. Selection: Select the bounding box with the highest confidence from the sorted list, mark it as selected, and add it to the final detection results list.
[0020] 3. Calculate IOU: For each remaining bounding box, calculate its IOU with the selected bounding box.
[0021] 4. Comparison and elimination: If the IOU between a bounding box and the selected box exceeds a preset threshold (such as 0.5 or 0.7), the two boxes are considered to represent the same target. Therefore, based on the principle of lower confidence, the low-confidence bounding box is eliminated.
[0022] Repeat steps 2-4: Continue to select the highest confidence among the remaining bounding boxes, repeat the IOU calculation and elimination process until all bounding boxes have been checked.
[0023] Exemplarily, the search image is interpolated and scaled by any interpolation method to obtain a first search image, and the size of the first search image is: 1920*1080*3, corresponding to the width*height*number of layers of the first search image. After reasoning with the first model (calculation of the first model), the depth feature of the first search image is obtained, and the size of the depth feature of the first search image is: 960*540*50; the template image is interpolated and scaled by any interpolation method to obtain a first template image, and the size of the first template image is: 50*50*3, corresponding to the width*height*number of layers of the first template image. After reasoning with the first model (calculation of the first model), the depth feature of the first template image is obtained, and the size of the depth feature of the first template image is: 25*25*50.
[0024] The depth feature of the first template image is slid from left to right and from top to bottom on the depth feature of the first search image. The size of the sliding window is 25*25, which is the width and height of the depth feature of the first template image. The sliding window feature values of the depth feature of the first search image are calculated in turn. For example, the Euclidean distance is used to calculate the same dimension values corresponding to the same point. In n-dimensional space (n-dimensional real vector space), point A and point B are two points in n-dimensional space. The coordinates of point A are , the coordinates of point B are , the Euclidean distance between point A and point B is defined as: ; Wherein, d(A, B) represents the Euclidean distance between point A and point B. The calculated Euclidean distance is used as the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image. The size of the sliding window is 25*25, which is the width and height of the depth feature size of the first template image. The depth feature of the first template image with the largest correlation is selected as the first matching result. The coordinates of the first matching result are: (w0, w1, h0, h1) = (100, 125, 100, 125), where (w0, w1, h0, h1) is the coordinate value relative to the depth feature of the first search image. The image coordinates mapped to the first search image (1920*1080*3) are (W0, W1, H0, H1). The calculation method of the image coordinates mapped to the first search image (1920*1080*3) is: w0:W0=960:1920, so W0=w0*1920 / 960, Use the same method to calculate W1; h0:H0=540:1080, so H0=h0*1080 / 540, use the same method to calculate H1.
[0025] Repeat the above steps, change the first model to the Nth model in turn, and finally obtain the Nth matching result. Finally, obtain the best matching result through non-maximum suppression The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.
Claims
1. A method for locating an image containing an arbitrary target and background, characterized in that: include: S1. Scale the template image by interpolation to obtain a first template image. The size of the original template image is , the size of the first template image is ; S2. Use the first model to infer the first template image to obtain the depth feature of the first template image. The size of the depth feature of the first template image is ; S3, scaling the search image by interpolation to obtain a first search image. The size of the original search image is , the size of the first search image is ; S4. Use the first model to infer the first search image to obtain the depth feature of the first search image. The size of the depth feature of the first search image is ; S5, calculating the correlation result between the depth feature of the first template image and the depth feature of the first search image, and selecting the result with the greatest correlation as the matching result, the calculation includes: The depth feature of the first template image is moved from left to right or from top to bottom on the depth feature of the first search image. During the movement, the correlation result between the depth feature of the first template image and the corresponding depth feature of the first search image is calculated. , selecting the depth feature of the first template image of the result with the greatest correlation as the first matching result, and mapping the coordinates of the depth feature of the first search image corresponding to the first matching result to the image coordinates of the first search image; S6, repeat steps S1-S5 N times, replace the first model with the Nth model in sequence, and obtain the Nth matching result; S7. Use non-maximum suppression to obtain the best matching result.
2. The image positioning method including arbitrary targets and backgrounds according to claim 1, characterized in that: The interpolation method includes nearest neighbor interpolation, bilinear interpolation or bicubic interpolation.
3. The image positioning method including arbitrary targets and backgrounds according to claim 1, characterized in that: The first model to the Nth model are feature backbone extraction networks trained with open source data or feature backbone networks for target detection and target segmentation trained with specific data.
4. The image positioning method including arbitrary targets and backgrounds according to claim 1, characterized in that: The method of using the first model to infer the first search image or the first template image includes any one or more combinations of a convolution operator, a residual operator, a multi-layer perceptron operator, a fully connected network operator or a Transformer operator.
5. The image positioning method including arbitrary targets and backgrounds according to claim 1, characterized in that: The method for calculating the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image includes the Euclidean distance: in an n-dimensional real vector space, A and B are two points in the n-dimensional real vector space, and the coordinates of point A are , the coordinates of point B are , the Euclidean distance between point A and point B is defined as: ; Wherein, d(A, B) represents the Euclidean distance between point A and point B, and the calculated Euclidean distance is used as the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image.
6. The image positioning method including arbitrary targets and backgrounds according to claim 1, characterized in that: The method for calculating the correlation result between the depth feature of the first template image and the depth feature of the corresponding first search image also includes: dot product, cosine similarity or absolute value error.
Citation Information
Patent Citations
Fast and dense image matching method and system
CN113033708A
Image matching method based on deep learning
CN113283525A
Target tracking method based on residual channel attention and multilevel classification regression
CN113706581A
Embedded twin network real-time tracking method applied to mobile platform
CN113850189A
End-to-end template matching method based on Transform network
CN116758162A
