Heterogenous image matching method based on multi-scale HOG (Histogram of Oriented Gradient) fusion
Through the multi-scale HOG fusion method, the problem of low matching accuracy of heterologous image in weak texture scenes is solved, and more accurate matching results and more efficient calculation process are achieved, which is suitable for heterologous image scenes with weak textures.
Patent Information
- Application Number
- CN202411445319.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-01-10
AI Technical Summary
In weak texture scenes, the matching accuracy of heterologous images is not high, and the existing single-scale HOG features cannot effectively describe the scale information of scene features, resulting in deviations in the matching results.
The multi-scale HOG fusion method is adopted to achieve a matching process from coarse to fine by fusion of HOG costs of multiple scale layers, reducing the calculation amount and avoiding dependence on a specific scale layer. The specific steps include multi-scale decomposition of the template image and the target image, HOG coarse matching cost calculation of the multi-scale target image, linear weighted fusion to obtain the coarse positioning results, and fine matching near the positioning point.
It improves the accuracy of the matching results, reduces the computational complexity, and achieves more efficient heterologous image matching, especially in weak texture scenarios, which significantly improves the matching effect.
Smart Images
Figure CN119295784A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of heterogeneous image matching, and in particular to a heterogeneous image matching method based on multi-scale HOG fusion. Background Art
[0002] Heterogeneous image matching is widely used in many fields such as medicine, remote sensing, image fusion, and scene matching. Since heterogeneous images differ significantly in brightness, contrast, and texture details, there has been no effective solution for heterogeneous image matching. Heterogeneous image matching methods are mainly divided into grayscale-based matching and feature-based matching. Grayscale-based matching methods directly or indirectly use grayscale information for matching, such as mutual information-based and self-similarity-based matching methods. Due to different imaging mechanisms, there are large differences in grayscale information of heterogeneous images of the same scene. Therefore, the performance of such methods is significantly reduced when applied to heterogeneous image matching. Feature-based matching methods generate feature descriptors by statistically analyzing the feature information of local areas, and use distance or correlation functions as matching measures for feature matching. They can better achieve stable matching between heterogeneous images. How to find and extract stable and reliable common features between heterogeneous images is the key to achieving accurate matching.
[0003] At present, the method based on HOG feature, i.e. gradient histogram, can achieve relatively good heterogeneous matching effect. However, if the heterogeneous matching scene has the characteristic of weak texture, the extracted single-scale HOG feature cannot well describe the scale information of the scene feature, thus causing a certain deviation in the matching result. Therefore, the present invention proposes a heterogeneous image matching method based on multi-scale HOG fusion. Summary of the invention
[0004] The technical problem to be solved by the present invention is: how to solve the problem of low accuracy of heterogeneous image matching in weak texture scenes, and provide a heterogeneous image matching method based on multi-scale HOG fusion. By fusing the HOG costs of multiple scale layers, the matching is not dependent on a specific scale layer, and the obtained matching result is more accurate. In addition, the matching process from coarse to fine is realized by sampling matching, which effectively reduces the calculation amount of matching.
[0005] The present invention solves the above technical problems through the following technical solutions, and the present invention comprises the following steps:
[0006] S1: Multi-scale decomposition of template image and target image
[0007] Performing Gaussian filtering and downsampling operations on the template image and the target image at multiple scales to obtain corresponding template images at multiple scales and target images at multiple scales;
[0008] S2: HOG coarse matching cost calculation of target images at multiple scales
[0009] Perform equal-interval sampling on multiple scale target images to obtain multiple scale target sub-images, and extract HOG (gradient histogram) features on multiple scale target sub-images and multiple scale template images at the same time, and calculate the HOG coarse matching cost of multiple scale target images by using correlation matching;
[0010] S3: Coarse positioning of matching results
[0011] The HOG coarse matching costs of target images of multiple scales are fused in a linear weighted manner to obtain the fused coarse matching cost of the target image, and the point position corresponding to the maximum value is found. This position is the coarse positioning matching result, and the corresponding point is the coarse positioning point of the target image.
[0012] S4: Precise positioning of matching results
[0013] A rectangular search area is selected near the rough positioning point of the target image, and each point in the rectangular search area is traversed to generate a corresponding search sub-image. The HOG features of each search sub-image are extracted, and the correlation matching calculation is performed with the HOG features of the template image to obtain the corresponding precise matching cost of each point. The point position corresponding to the maximum precise matching cost is found, and this position is the precise positioning matching result.
[0014] Furthermore, in step S1, the specific processing process is as follows: T 、w T The template image T1 is subjected to N-1 Gaussian filtering and downsampling operations in sequence, and the obtained N-1 images are denoted as T2, ..., T N , T1 and T2, ..., T N It forms a multi-scale template image; similarly, the height and width are h R 、w R The target image R1 is subjected to N-1 Gaussian filtering and downsampling operations in sequence, and the obtained N-1 images are denoted as R2, ..., R N , R1 and R2, ..., R N Composed of multi-scale target images.
[0015] Furthermore, in step S2, the specific processing process is as follows:
[0016] S21: If the sampling interval of the first scale layer is S, then the sampling interval of the second scale layer is S / 2, and so on, the sampling interval of the kth scale layer is S / 2. k-1 , where k = 1, 2, ..., N, and then obtain the target sub-image M(i, j, k) corresponding to the i-th column and j-th row in the k-th scale layer;
[0017] S22: Extract the HOG (histograph of gradient) feature of the target sub-image M(i,j,k), and record the obtained feature result as f(M(i,j,k)). Similarly, extract the HOG feature of the template image T of the kth scale layer. k Perform HOG feature extraction, and the feature result is recorded as f(T k ), and calculate the HOG coarse matching cost C(i, j, k) of target images of multiple scales according to the formula.
[0018] Furthermore, in step S21, the target sub-image M(i,j,k) is obtained as follows: intercept the target image R of the kth scale layer k 1+(i-1)*S / 2 k-1 Column to w T +(i-1)*S / 2 k-1 Column and 1+(j-1)*S / 2 k-1 Go to h T +(j-1)*S / 2 k-1 The image composed of rows is M(i,j,k), where i∈[1,S*(w R / 2 k-1 -w T ) / 2 k-1 +1]∩Z,j∈[1,S*(h R / 2 k-1 -h T ) / 2 k-1 +1]∩Z, where Z is the set of integers.
[0019] Furthermore, in step S22, the calculation formula of the HOG coarse matching cost C(i, j, k) of the target images of multiple scales is as follows:
[0020] C(i,j,k)=corr(f(M(i,j,k)),f(T k ))
[0021] Among them, corr represents the Pearson correlation operation.
[0022] Furthermore, in step S3, the specific processing process is as follows:
[0023] S31: According to the linear weighted fusion formula, the coarse matching costs C(i, j, k) at different scales are fused to obtain the fused coarse matching cost C of the target image sum (i,j);
[0024] S32: Obtaining the rough matching cost C sum The maximum point (i * ,j *), calculate the coordinates (x0, y0) of the rough positioning point p of the target image:
[0025] x0=1+(i * -1)*S;
[0026] y0=1+(j * -1)*S.
[0027] Furthermore, in step S31, the linear weighted fusion formula is as follows:
[0028]
[0029] Among them, α k is a non-negative weighting coefficient that satisfies
[0030] Furthermore, in step S4, the specific processing process is as follows:
[0031] S41: With the target image coarse positioning point p as the center, select a rectangular search area with a length and width of U, where U=2S+1, and traverse each point in the rectangular search area to generate a search sub-image Q(x,y) corresponding to the xth column and yth row;
[0032] S42: Extract HOG (Histogram of Gradients) features from the search sub-image Q(x, y), and record the obtained feature result as f(Q(x, y)), and calculate the exact matching cost D(x, y) according to the formula;
[0033] S43: Calculate the maximum value point (x) of the exact matching cost D(x,y) * ,y * ) is the precise positioning matching result.
[0034] Furthermore, in step S41, the search sub-image Q(x, y) is generated as follows: intercept the xth column to the wth column of R1 T -1+x columns and yth to hth rows T The image composed of -1+y rows is taken as Q(x,y), where x∈[x0-S,x0+S]∩Z and y∈[y0-S,y0+S]∩Z.
[0035] Furthermore, in step S42, the calculation formula of the precise matching cost D(x, y) is as follows:
[0036] D(x,y)=corr(f(Q(x,y)),f(T1)).
[0037] Compared with the prior art, the present invention has the following advantages: the heterogeneous image matching method based on multi-scale HOG fusion realizes the fusion of feature information of multiple scales, so that the matching result does not depend on a specific scale layer; although feature calculations of multiple scales are performed, the computational complexity is still O(N), which is the same as the traditional matching method based on HOG features, but the amount of calculation required for the coarse-to-fine matching method is much lower than that of the traditional method, and the computational efficiency is higher. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a flow chart of a heterogeneous image matching method based on multi-scale HOG fusion in an embodiment of the present invention;
[0039] Figure 2 3 is a comparison diagram of heterogeneous image matching effects in an embodiment of the present invention, wherein (a) is the traditional heterogeneous matching effect based on HOG features (the red frame is the matching result), and (b) is the heterogeneous matching effect based on multi-scale HOG fusion of the present invention (the red frame is the matching result). DETAILED DESCRIPTION
[0040] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.
[0041] like Figure 1 As shown, this embodiment provides a technical solution: a heterogeneous image matching method based on multi-scale HOG fusion, comprising the following steps:
[0042] (1) Multi-scale decomposition of template image and target image
[0043] Performing Gaussian filtering and downsampling operations on the template image and the target image at multiple scales to obtain corresponding template images at multiple scales and target images at multiple scales;
[0044] (2) HOG coarse matching cost calculation of target images at multiple scales
[0045] Perform equal-interval sampling on multiple scale target images to obtain multiple scale target sub-images, and extract HOG (gradient histogram) features on multiple scale target sub-images and multiple scale template images at the same time, and calculate the HOG coarse matching cost of multiple scale target images by using correlation matching;
[0046] (3) Rough positioning of matching results
[0047] The HOG coarse matching costs of target images of multiple scales are fused in a linear weighted manner to obtain the fused coarse matching cost of the target image, and the point position corresponding to the maximum value is found. This position is the coarse positioning matching result, and the corresponding point is the coarse positioning point of the target image.
[0048] (4) Precise positioning of matching results
[0049] A rectangular search area is selected near the rough positioning point of the target image, and each point in the rectangular search area is traversed to generate a corresponding search sub-image. The HOG features of each search sub-image are extracted, and the correlation matching calculation is performed with the HOG features of the template image to obtain the corresponding precise matching cost of each point. The point position corresponding to the maximum precise matching cost is found, and this position is the precise positioning matching result.
[0050] In this embodiment, in step (1), the multi-scale decomposition of the template image and the target image is specifically as follows: T 、w T The template image T1 is subjected to N-1 Gaussian filtering and downsampling operations in sequence, and the obtained N-1 images are denoted as T2, ..., T N , T1 and T2, ..., T N It forms a multi-scale template image; similarly, the height and width are h R 、w R The target image R1 is subjected to N-1 Gaussian filtering and downsampling operations in sequence, and the obtained N-1 images are denoted as R2, ..., R N , R1 and R2, ..., R N Composed of multi-scale target images; Figure 2 In the embodiment shown, h T =512,w T =512,h R =800,w R =800, N is 3;
[0051] In this embodiment, in step (2), the HOG coarse matching cost of the target images of the multiple scales is calculated as follows: assuming that the sampling interval of the first scale layer is S, the sampling interval of the second scale layer is S / 2, and so on, the sampling interval of the kth scale layer is S / 2. k-1 , where k = 1, 2, ..., N, the target sub-image M(i, j, k) corresponding to the i-th column and j-th row in the k-th scale layer is obtained as follows: intercept the target image R of the k-th scale layer k 1+(i-1)*S / 2 k-1 Column to w T +(i-1)*S / 2 k-1 Column and 1+(j-1)*S / 2 k-1Go to h T +(j-1)*S / 2 k-1 The image composed of rows is M(i,j,k), where i∈[1,S*(w R / 2 k-1 -w T ) / 2 k-1 +1]∩Z,j∈[1,S*(h R / 2 k-1 -h T ) / 2 k-1 +1]∩Z, Z is the set of integers; Figure 2 In the embodiment shown, S takes a value of 8;
[0052] The target sub-image M(i,j,k) is subjected to HOG (histograph of gradient) feature extraction, and the feature result is recorded as f(M(i,j,k)). Similarly, the template image T of the kth scale layer is extracted. k Perform HOG feature extraction, and the feature result is recorded as f(T k ), calculate the HOG coarse matching cost C(i,j,k) of target images of multiple scales according to the following formula:
[0053] C(i,j,k)=corr(f(M(i,j,k)),f(T k ))
[0054] Among them, corr represents the Pearson correlation operation.
[0055] In this embodiment, in step (3), the rough positioning of the matching result is specifically as follows:
[0056] According to the following formula, the coarse matching costs C(i, j, k) at different scales are fused to obtain the fused coarse matching cost C of the target image sum (i,j):
[0057]
[0058] Among them, α k is a non-negative weighting coefficient that satisfies exist Figure 2 In the embodiment shown, the values of α1, α2, and α3 are 0.6, 0.3, and 0.1, respectively;
[0059] In order to obtain the rough matching cost C sum The maximum point (i * ,j * ), the coordinates (x0, y0) of the rough positioning point p of the target image are calculated according to the following formula:
[0060] x0=1+(i *-1)*S;
[0061] y0=1+(j * -1)*S;
[0062] In this embodiment, in step (4), the precise positioning of the matching result is specifically as follows:
[0063] With the target image coarse positioning point p as the center, select a rectangular search area with length and width U, where U = 2S + 1, and traverse each point in the search area to generate the search sub-image Q(x, y) corresponding to the xth column and yth row as follows: intercept the xth column to the wth row of R1 T -1+x columns and yth to hth rows T The image composed of -1+y rows is Q(x,y), where x∈[x0-S,x0+S]∩Z and y∈[y0-S,y0+S]∩Z;
[0064] The HOG (Histogram of Gradients) feature is extracted from the search sub-image Q(x, y), and the feature result is recorded as f(Q(x, y)). The exact matching cost D(x, y) is calculated according to the following formula:
[0065] D(x,y)=corr(f(Q(x,y)),f(T1))
[0066] Calculate the maximum value point (x) of the exact matching cost D(x,y) * ,y * ) is the precise positioning matching result.
[0067] like Figure 2 As shown in (b), it is the heterogeneous image matching effect. The template image (on the right) is a SAR image, and the target image (on the left) is an optical image. There are many weak texture areas in this scene. Figure 2 As shown in (a), the traditional HOG feature-based matching method only uses single-scale information for matching, and the matching effect has a large deviation. However, the method of the present invention after fusing multi-scale information can achieve a more accurate matching effect.
[0068] In summary, the heterogeneous image matching method based on multi-scale HOG fusion in the above embodiment makes the matching independent of a specific scale layer by fusing the HOG costs of multiple scale layers, and the obtained matching result is more accurate. In addition, the matching process from coarse to fine is realized by sampling matching, which effectively reduces the calculation amount of matching.
[0069] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A heterogeneous image matching method based on multi-scale HOG fusion, characterized in that: The following steps are involved: S1: Multi-scale decomposition of template image and target image Performing Gaussian filtering and downsampling operations on the template image and the target image at multiple scales to obtain corresponding template images at multiple scales and target images at multiple scales; S2: HOG coarse matching cost calculation of target images at multiple scales Perform equal-interval sampling on multiple scale target images to obtain multiple scale target sub-images, and extract HOG (gradient histogram) features on multiple scale target sub-images and multiple scale template images at the same time, and calculate the HOG coarse matching cost of multiple scale target images by using correlation matching; S3: Coarse positioning of matching results The HOG coarse matching costs of target images of multiple scales are fused in a linear weighted manner to obtain the fused coarse matching cost of the target image, and the point position corresponding to the maximum value is found. This position is the coarse positioning matching result, and the corresponding point is the coarse positioning point of the target image. S4: Precise positioning of matching results A rectangular search area is selected near the rough positioning point of the target image, and each point in the rectangular search area is traversed to generate a corresponding search sub-image. The HOG features of each search sub-image are extracted, and the correlation matching calculation is performed with the HOG features of the template image to obtain the corresponding precise matching cost of each point. The point position corresponding to the maximum precise matching cost is found, and this position is the precise positioning matching result.
2. According to claim 1, a heterogeneous image matching method based on multi-scale HOG fusion is characterized in that: In step S1, the specific processing process is as follows: for the height and width respectively h T 、w T The template image T1 is subjected to N-1 Gaussian filtering and downsampling operations in sequence, and the obtained N-1 images are denoted as T2, ..., T N , T1 and T2, ..., T N It forms a multi-scale template image; similarly, the height and width are h R 、w R The target image R1 is subjected to N-1 Gaussian filtering and downsampling operations in sequence, and the obtained N-1 images are denoted as R2, ..., R N , R1 and R2, ..., R N Composed of multi-scale target images.
3. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 2 is characterized in that: In step S2, the specific processing process is as follows: S21: If the sampling interval of the first scale layer is S, then the sampling interval of the second scale layer is S / 2, and so on, the sampling interval of the kth scale layer is S / 2. k-1 , where k = 1, 2, ..., N, and then obtain the target sub-image M(i, j, k) corresponding to the i-th column and j-th row in the k-th scale layer; S22: Extract the HOG (histograph of gradient) feature of the target sub-image M(i,j,k), and record the obtained feature result as f(M(i,j,k)). Similarly, extract the HOG feature of the template image T of the kth scale layer. k Perform HOG feature extraction, and the feature result is recorded as f(T k ), and calculate the HOG coarse matching cost C(i, j, k) of target images of multiple scales according to the formula.
4. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 3 is characterized in that: In step S21, the target sub-image M(i,j,k) is obtained as follows: intercept the target image R of the kth scale layer k 1+(i-1)*S / 2 k-1 Column to w T +(i-1)*S / 2 k-1 Column and 1+(j-1)*S / 2 k-1 Go to h T +(j-1)*S / 2 k-1 The image composed of rows is M(i,j,k), where i∈[1,S*(w R / 2 k-1 -w T ) / 2 k-1 +1]∩Z,j∈[1,S*(h R / 2 k-1 -h T ) / 2 k-1 +1]∩Z, where Z is the set of integers.
5. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 3 is characterized in that: In step S22, the calculation formula of the HOG coarse matching cost C(i, j, k) of the target images of multiple scales is as follows: C(i,j,k)=corr(f(M(i,j,k)),f(T k )) Among them, corr represents the Pearson correlation operation.
6. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 3 is characterized in that: In step S3, the specific processing process is as follows: S31: According to the linear weighted fusion formula, the coarse matching costs C(i, j, k) at different scales are fused to obtain the fused coarse matching cost C of the target image sum (i,j); S32: Obtaining the rough matching cost C sum The maximum point (i * ,j * ), calculate the coordinates (x0, y0) of the rough positioning point p of the target image: x0=1+(i * -1)*S; y0=1+(j * -1)*S。 7. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 6, characterized in that: In step S31, the linear weighted fusion formula is as follows: Among them, α k is a non-negative weighting coefficient that satisfies 8. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 6, characterized in that: In step S4, the specific processing process is as follows: S41: With the target image coarse positioning point p as the center, select a rectangular search area with a length and width of U, where U=2S+1, and traverse each point in the rectangular search area to generate a search sub-image Q(x,y) corresponding to the xth column and yth row; S42: Extract HOG (Histogram of Gradients) features from the search sub-image Q(x, y), and record the obtained feature result as f(Q(x, y)), and calculate the exact matching cost D(x, y) according to the formula; S43: Calculate the maximum value point (x) of the exact matching cost D(x,y) * ,y * ) is the precise positioning matching result.
9. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 8, characterized in that: In step S41, the search sub-image Q(x, y) is generated as follows: intercept the xth column to the wth column of R1 T -1+x columns and yth to hth rows T The image composed of -1+y rows is taken as Q(x,y), where x∈[x0-S,x0+S]∩Z and y∈[y0-S,y0+S]∩Z.
10. The heterogeneous image matching method based on multi-scale HOG fusion according to claim 8, characterized in that: In step S42, the calculation formula of the precise matching cost D(x, y) is as follows: D(x,y)=corr(f(Q(x,y)),f(T1)).