A method for underwater terrain image matching based on saliency superpixels

Abstract

The present invention belongs to the technical field of underwater terrain image matching, and specifically relates to a method for underwater terrain image matching based on saliency superpixels. The present invention preprocesses underwater terrain images to reduce the impact of noise and terrain resolution differences on image appearance, and generates a set of superpixels in the terrain image based on image aggregation technology; sorts superpixels by calculating the saliency value of each superpixel, and selects several superpixel blocks with high saliency values ​​as items to be matched; constructs a maximum index map based on the Log‑Gabor filter with the neighborhood of the superpixel as a unit, and describes the superpixel together with the terrain information parameters of the area. The present invention replaces the process of extracting feature points by selecting saliency areas in the image to improve the stability of the distribution of feature points, and reduces the sensitivity of the maximum index map to rotation by rotating the neighborhood.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater terrain image matching, and in particular relates to an underwater terrain image matching method based on saliency superpixels. Background Art

[0002] High-precision underwater positioning and navigation are key technologies for autonomous underwater operations by intelligent underwater vehicles (AUVs). With the development of AUV technology and the increasing demand for marine exploration and detection missions, using terrain information measured during the AUV's voyage to assist positioning and navigation has become a reliable solution for long-term, large-scale underwater operations. Underwater terrain matching navigation is similar to terrain matching for aircraft. When an AUV traverses a terrain-matching area, it uses acoustic altimeters, Doppler velocimeters, multi-beam sonars, and other depth-sensing sensors in conjunction with pressure sensors to obtain the terrain elevation value at its location. This is then matched with reference terrain elevation data to obtain the AUV's real-time position. Furthermore, by gridding and interpolating the discrete elevation data and projecting it into a two-dimensional image on a plane, terrain matching navigation can be transformed into a positioning solution based on image matching. Therefore, it is possible to consider applying current image matching technology to AUV positioning solutions.

[0003] Image matching technology has been widely studied in the past few decades. Current image matching methods mainly include region-based methods and feature-based methods. Region-based matching methods are also called template matching, such as NCC, BBS, QATM, etc. They usually regard the template image as a whole area, calculate the similarity of the target image subwindow by sliding window, and regard the subwindow with the highest similarity score as the matching area. This type of method is suitable for image pairs that only contain translation changes. When faced with complex changes such as rotation and perspective changes, template matching is still a difficult task. During the underwater navigation of AUV, the heading sensor is affected by its own accuracy and latitude, and it is inevitable that there will be certain deviations. When operating in shallow waters, it will also be affected by ocean currents and cause heading drift, resulting in rotation changes between the acquired real-time terrain data and the reference terrain data. Therefore, the robustness of the matching method to rotation changes should be fully considered.

[0004] Compared to region-based methods, feature-based matching methods are more robust to planar rotations. Currently, most feature-based methods typically detect and describe salient points in an image. Their main steps include feature point detection, feature description, feature matching, and rejection of mismatched point pairs. The stability of feature points extracted by the detector is directly related to matching performance. Traditional feature point extraction methods, such as SIFT, Fast, and Harris detectors, detect feature points by calculating the intensity or gradient difference of local regions. Due to their robustness to changes in illumination, scale, and viewpoint, they have been widely used in optical image processing. However, underwater terrain images are measured in a water environment using acoustic sensors. Because acoustic sensors have much lower accuracy than optical sensors, the measurement data often contain significant errors. Furthermore, the inevitable environmental noise in underwater environments also affects measurement accuracy. Furthermore, different types of acoustic sensors and different sensor depths result in different terrain resolutions in elevation data acquired from the same area. The effects of noise and differences in terrain resolution are key factors leading to nonlinear intensity differences between pairs of underwater terrain images. These factors pose greater challenges to the stability of feature point extraction. The relatively flat nature of the local terrain further increases the difficulty of feature point extraction and feature description. Learning-based feature detectors, such as LIFT, MagicPoint, and Superpoint, are based on the design concept of traditional detectors. They improve the stability of feature points and their robustness to illumination changes by learning in samples. LoFTR is a detector-free local feature matching method that uses self-attention layers and cross-attention layers in Transformer to improve matching results in areas with less texture. However, these methods are designed and trained for specific types of images, and their generalization ability is poor when applied to underwater terrain images. In summary, it can be seen that directly applying the existing image matching methods to terrain images is difficult to meet the actual application requirements of positioning tasks for accuracy. Summary of the Invention

[0005] The purpose of the present invention is to provide an underwater terrain image matching method based on saliency superpixels. By selecting salient areas in the image instead of extracting feature points, the stability of the feature point distribution can be improved. The saliency value can quantify the richness of terrain information in the local area of ​​the superpixel, thereby improving the expressive power of the feature descriptor.

[0006] A method for underwater terrain image matching based on saliency superpixels comprises the following steps:

[0007] Step 1: For the underwater terrain reference image set U and the target image set V to be matched, all underwater terrain images in the two sets are subjected to Gaussian filtering for noise reduction. Then, PCA is used to further remove noise and extract the boundary lines of terrain changes.

[0008] Step 2: Use the linear iterative clustering (SLIC) method to segment the underwater terrain image into superpixel blocks s, and obtain the superpixel set U of the underwater terrain reference image. s and the superpixel set V of the target image s ;

[0009] Step 3: For the superpixel set U of the underwater terrain reference image s For each superpixel block s in the image, calculate the saliency value Sallency(s), sort the superpixel blocks according to the saliency value, select several superpixel blocks with high saliency values ​​as the items to be matched, and obtain the saliency superpixel set U of the underwater terrain reference image. t ;

[0010] Saliency(s)=(Norm(E(s))+Norm(R(s)))*N

[0011] Where E(s) represents the terrain information entropy in the neighborhood of superpixel block s; R(s) represents the terrain roughness in the neighborhood of superpixel block s; N represents the number of superpixel members in the neighborhood of superpixel block s; Norm represents normalization processing;

[0012] Step 4: For the salient superpixel set U of the underwater terrain reference image t For each superpixel block in the , the feature is extracted based on the adjacent domain of the superpixel block and used as the feature descriptor of the central superpixel block;

[0013] Step 5: For the salient superpixel set U of the underwater terrain reference image t The superpixel set V of the labeled image s , for any u∈U t ,v∈V s , let F(u) and F(v) represent the eigenvectors of u and v respectively, regard u and v as a node in the graph, let S(u,v) represent the association score of node u and node v, and use the cosine distance cosin to describe the similarity between the eigenvectors, S(u,v) = cosin(F(u),F(v)); the association scores of all nodes generate a score matrix, and then the optimal match is obtained by the Hungarian algorithm, and the wrong matches are eliminated based on the RANSAC algorithm.

[0014] Furthermore, the superpixel block s in step 2 is represented by pixel labels, eigenvalues ​​and centroid coordinates; in the grayscale terrain image, the grayscale value represents the elevation value of the location, and each superpixel block s is regarded as an equal-height area from a physical perspective.

[0015] Furthermore, the step 4 is specifically as follows: for the salient superpixel set U of the underwater terrain reference imaget For each superpixel block in the image, the connecting domain of the superpixel block is first extracted in the form of a mask, and then the circumscribed rectangular boundary of the connecting domain is calculated and intercepted to generate the corresponding sub-image; the texture information in each sub-image is extracted using a Log-Gabor filter, including the image texture in the connecting domain and the boundary information of the connecting domain structure; for the connecting domain of each superpixel block, the ellipse with the same second-order moment as the region is first solved, and then the angle θ between the major axis of the ellipse and the 0th axis is used as the rotation angle to rotate the connecting domain. At the same time, the connecting domain is rotated by (θ+180)° and then MIMs are constructed respectively. The two MIMs are then added to construct the final MIM descriptor;

[0016] A two-dimensional Log-Gabor filter is represented as the product of a radial filter and an angular filter in polar coordinates:

[0017]

[0018] Where r represents the radial coordinate; θ represents the angular coordinate; f0 and θ0 represent the center frequency and directional angle of 2D-LGF respectively; σ r and σ θ denote the bandwidth of r and θ respectively.

[0019] The beneficial effects of the present invention are:

[0020] The present invention preprocesses underwater terrain images to reduce the effects of noise and terrain resolution differences on image appearance. Based on image aggregation technology, a set of superpixels is generated within the terrain image. Superpixels are sorted by calculating the saliency value of each superpixel, and several superpixel blocks with high saliency values ​​are selected as matching items. The saliency value quantifies the richness of terrain information in the local superpixel region, thereby improving the stability of feature extraction. A maximum index map is constructed based on a Log-Gabor filter, using the neighborhood of the superpixel as a unit, and the superpixels are described together with the terrain information parameters of the region. The present invention replaces the feature point extraction process with the selection of salient regions in the image to improve the stability of the feature point distribution. The sensitivity of the maximum index map to rotation is reduced by rotating the neighborhood. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a case diagram for noise sensitivity analysis.

[0022] Figure 2 This is a case diagram for rotation sensitivity analysis.

[0023] Figure 3 This is a sensitivity analysis diagram of terrain resolution.

[0024] Figure 4This is the superpixel segmentation result of the underwater terrain image.

[0025] Figure 5 Schematic diagram of superpixel saliency value calculation.

[0026] Figure 6 Build process maps for MIM. DETAILED DESCRIPTION

[0027] The present invention will be further described below with reference to the accompanying drawings.

[0028] The present invention provides an underwater terrain image matching method based on saliency superpixels. First, the underwater terrain image is preprocessed to reduce the impact of noise and terrain resolution differences on the image appearance, and a group of superpixels is generated in the terrain image based on image aggregation technology. Secondly, the superpixels are sorted by calculating the saliency value of each superpixel, and several superpixel blocks with high saliency values ​​are selected as items to be matched. The saliency value can quantify the richness of terrain information in the local area of ​​the superpixel, thereby improving the stability of feature extraction. Then, a maximum index map is constructed based on the Log-Gabor filter with the neighborhood of the superpixel as the unit, and the superpixel is described together with the terrain information parameters of the area. In addition, the sensitivity of the maximum index map to rotation is reduced by rotating the neighborhood.

[0029] The reference image and target image mentioned in the present invention represent the terrain map acquired by the AUV in real time and the pre-stored priori terrain map, respectively. The size of the reference image is much smaller than that of the target image.

[0030] 1. Preprocessing

[0031] As a preprocessing step for the algorithm, we first need to extract superpixels from the terrain image pair. Superpixel algorithms segment images by grouping similar pixels, making them highly sensitive to noise variations. Nonlinear noise in underwater terrain images can affect the superpixel algorithm's sensitivity to terrain boundaries, leading to significant deviations in the segmentation results. Therefore, we first need to perform noise reduction on the algorithm's input image to improve the stability of superpixel segmentation.

[0032] Since the noise V in underwater terrain images is usually considered to obey a Gaussian distribution with zero mean and variance δ, that is, Therefore, we primarily target Gaussian noise. In image denoising algorithms, filter-based methods employ filter design to denoise images. Due to their high execution speed and effectiveness against Gaussian noise, they are used as image preprocessing methods in various algorithms. The principal component analysis (PCA) algorithm assumes that the data is contaminated by small Gaussian noise and reconstructs the original data by preserving the principal components in the image, thereby achieving the goal of denoising. Therefore, before extracting superpixels, we first perform Gaussian filtering on the underwater terrain image to denoise it. Then, based on PCA, we further remove noise and extract the boundaries of terrain changes. PCA can capture the hierarchical changes in terrain data by reducing the degree of correlation between the data.

[0033] Finally, we use the Simple Linear Iterative Clustering (SLIC) method to segment the terrain image into superpixels. SLIC has low computational cost and strong edge response, and the generated superpixels are relatively compact, regular, and of similar size. Each superpixel can be represented by its pixel label, eigenvalue, and centroid coordinates. In grayscale terrain images, the grayscale value represents the elevation value of that location. From a physical perspective, each superpixel generated in the terrain image can be treated as an area of ​​equal height. Using this as the basic processing unit significantly reduces the complexity of post-processing. Figure 4 This is an example of the effect of the target image after preprocessing and image segmentation.

[0034] 2. Extracting salient superpixels

[0035] When humans perform target matching, they first select the most conspicuous areas, actively searching for specific target features to increase the probability of successful matching. Inspired by this behavior, we mimic this process by selecting salient superpixels from images. Due to the unstructured nature of underwater terrain, there are no clear definitions of targets in underwater terrain images. Salient areas in topographic maps can be summarized as the richness of local terrain information. Therefore, we introduce terrain parameters to measure intrinsic qualities of the terrain distribution, such as the discreteness and smoothness, and use them to calculate the metric for selecting salient superpixels.

[0036] Specifically, for each superpixel s generated in the underwater terrain image, we calculate the terrain information entropy E(s) and terrain roughness R(s) within the superpixel's neighborhood, then normalize and add them together to form the local terrain attribute value of s. E(s) quantifies the amount of information about the local terrain; a larger E(s) indicates more dramatic terrain height variations, while R(s) reflects the complexity of the surface cells within s's neighborhood.

[0037] In addition, since the superpixels at the edge of the image lack some neighborhood information, which makes it impossible to fully record the local terrain attributes, the values ​​of these superpixels need to be penalized when calculating the saliency index. We finally define the saliency value of s as:

[0038] Saliency(s)=(Norm(E(s))+Norm(R(s)))*N

[0039] Where N represents the number of superpixel members in the neighborhood of s. Since superpixels at the edge of an image usually have fewer neighborhood members, this can be used as a penalty term. Norm represents normalization. Figure 5 A schematic diagram of the significance values ​​calculated by different parameters is shown.

[0040] It is worth noting that the process of extracting salient superpixels can be performed only in the reference image. We can also calculate and select superpixel sequences in the target image whose saliency value is greater than a fixed value (this value is related to the saliency value of the superpixel sequence in the reference image), thereby reducing the complexity of the subsequent matching process.

[0041] 3. Feature Description

[0042] After extracting the significant superpixels, each superpixel needs to be characterized to increase the distinction between features. Although the image is denoised in the preprocessing stage, there will still be differences in the pixel decomposition in each view, resulting in inaccurate correspondence, but the terrain and structural features in the neighborhood contain sufficient distinguishing information. For each significant superpixel block selected in the previous stage, the adjacent region formed by it and the adjacent superpixels can record the topological structure of the terrain around the superpixel. This structure is rotationally invariant and is more robust to nonlinear intensity changes than a single superpixel block. Based on the above principles, we extract features based on the adjacent region of the superpixel block and use it as the feature descriptor of the central superpixel block.

[0043] For each selected significant superpixel, we first extract its bounding box using a mask, then calculate the bounding rectangle of the bounding box and intercept it to generate the corresponding sub-image. We use the Log-Gabor filter to extract texture information from each sub-image, including the image texture within the bounding box and the boundary information of the bounding box structure. The Log-Gabor filter has shown good results in extracting texture features and is more robust to nonlinear intensity changes. A two-dimensional Log-Gabor filter (2D-LGF) can be expressed as the product of a radial filter and an angular filter in polar coordinates:

[0044]

[0045] Where r represents the radial coordinate, θ represents the angular coordinate, f0 and θ0 represent the center frequency and direction angle of 2D-LGF respectively, r and σ θ Then, θ represents the bandwidth of r and θ respectively. We construct a directional 2D-LGF with four angles (0°, 45°, 90°, and 135°) and select the largest filter response. Then, we divide each sub-image into 4×4 sub-grids and construct a distribution histogram to construct the Maximum Index Map (MIM) descriptor.

[0046] Although the neighborhood structure of superpixels has a certain stability to rotation changes, MIM is very sensitive to rotation. Therefore, if there is a rotation change in the image pair, the MIM descriptor will no longer be applicable. In order to ensure the robustness of the feature to rotation, the most direct method is to obtain the local gradient direction and rotate the region. However, the image gradient is very sensitive to nonlinear intensity changes, so the stability of the local region gradient cannot be guaranteed. We obtain the rotation angle based on the robustness of the superpixel neighborhood structure to noise changes. Specifically, for each superpixel's adjacent domain, we first solve the ellipse with the same second-order moment as the region, and then use the angle θ between the major axis of the ellipse and the 0th axis as the rotation angle to rotate the adjacent domain. In addition, since the major axis of the ellipse has no directionality, we also rotate the adjacent domain by (θ+180)° and construct MIMs separately. Then, the two MIMs are added to construct the final MIM descriptor to reduce the impact of rotation on the MIM value, thereby improving the rotation robustness of the feature descriptor. The construction process of MIM is as follows: Figure 6 Finally, we connect MIM with the above terrain parameters to construct the feature vector of superpixel.

[0047] 4. Matching Strategy

[0048] In this stage, we determine the correspondence between superpixels by performing similarity matching on the feature vectors of superpixels. t and the superpixel set V in the target image s , for any u∈U t ,v∈V s , let F(u) and F(v) represent the feature vectors of u and v respectively. Inspired by the graph attention network, we regard u and v as nodes in the graph, thereby constructing a bipartite graph matching problem. We let S(u,v) represent the association score between node u and node v, and use cosine distance cosin to describe the similarity between feature vectors, that is:

[0049] S(u,v)=cosin(F(u),F(v))

[0050] The association scores of all nodes generate a score matrix, and then the optimal matching items are obtained through the Hungarian algorithm, and the wrong matches are eliminated based on the RANSAC algorithm.

[0051] Figure 1 Figure 1 shows a noise sensitivity analysis example. From top to bottom, the first row shows the effect of noise variations on the appearance of the terrain image and the corresponding principal component plots. The second to last rows show the matching results of different methods. The red box indicates the true location of the reference image, the green box indicates the matching location, and the light blue line indicates the matching result based on feature point matching. (a) Noise variance = 0, (b) Noise variance = 3, (c) Noise variance = 5.

[0052] Figure 2 Figure 1 shows a case study of rotation sensitivity analysis. From top to bottom, the first row shows the terrain reference image sampled at different rotation angles and the corresponding principal component images. The second to last rows show the matching results of different methods. The red box indicates the true position of the reference image, the green box indicates the matching position, and the light blue line indicates the matching result based on feature point matching. (a) 30°, (b) 120°, (c) 210°.

[0053] Figure 3 This is a sensitivity analysis of terrain resolution. From top to bottom, the first row shows the appearance changes of terrain images under different TGRs and the corresponding principal component maps. The second to last rows show the matching results of different methods. The red box indicates the true position of the reference image, the green box indicates the matching position, and the light blue line indicates the matching result based on feature point matching. (a) Sampling step = 4, (b) Sampling step = 8, (c) Sampling step = 12.

[0054] Figure 4 (a) is the original underwater terrain map; (b) is the preprocessed underwater terrain image; (c) is the superpixel segmentation result.

[0055] Figure 5 Where (a) is E(s); (b) is R(s); and (c) is Saliency(s).

[0056] Figure 6 (a) shows the selected significant superpixel (represented by the dark color in the middle) and its neighboring superpixels (represented by the light color around it); (b) shows the calculation process of the rotation angle of the neighboring domain; (c) and (e) are schematic diagrams after the neighboring domain is rotated by θ° and (θ+180)°, respectively; (d) and (f) are the MIMs of (c) and (e), respectively; (g) is the MIM after adding (d) and (f).

[0057] Figures 1 to 3The following diagrams show the appearance changes of the reference image, preprocessing results, and matching results of the comparison methods in a sensitive analysis case. It can be seen that the template matching method based on region matching is robust to noise and terrain resolution differences, but is very sensitive to rotation changes. The feature point-based method cannot guarantee the stability of feature point acquisition and thus fails to match in most tests. In comparison, the present invention achieves remarkably high accuracy in all cases and maintains sufficient stability, thus confirming its effectiveness for matching underwater terrain images.

[0058] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for underwater terrain image matching based on saliency superpixels, characterized in that: The following steps are involved: Step 1: For the underwater terrain reference image set U and the target image set V to be matched, all underwater terrain images in the two sets are subjected to Gaussian filtering for noise reduction. Then, PCA is used to further remove noise and extract the boundary lines of terrain changes. Step 2: Use the linear iterative clustering (SLIC) method to segment the underwater terrain image into superpixel blocks s, and obtain the superpixel set U of the underwater terrain reference image. s and the superpixel set V of the target image s ; Step 3: For the superpixel set U of the underwater terrain reference image s For each superpixel block s in the image, calculate the saliency value Saliency(s), sort the superpixel blocks according to the saliency value, select several superpixel blocks with high saliency values ​​as the items to be matched, and obtain the saliency superpixel set U of the underwater terrain reference image. t ; Saliency(s)=(Norm(E(s))+Norm(R(s)))*N Where E(s) represents the terrain information entropy in the neighborhood of superpixel block s; R(s) represents the terrain roughness in the neighborhood of superpixel block s; N represents the number of superpixel members in the neighborhood of superpixel block s; Norm represents normalization processing; Step 4: For the salient superpixel set U of the underwater terrain reference image t For each superpixel block in the , the feature is extracted based on the adjacent domain of the superpixel block and used as the feature descriptor of the central superpixel block; Step 5: For the salient superpixel set U of the underwater terrain reference image t The superpixel set V of the labeled image s , for any u∈U t , v∈V s , let F(u) and F(v) represent the feature vectors of u and v respectively, regard u and v as a node in the graph respectively, let S(u, v) represent the association score of node u and node v, and use the cosine distance cosin to describe the similarity between the feature vectors, S(u, v) = cosin(F(u), F(v)); the association scores of all nodes generate a score matrix, and then the optimal match is obtained by the Hungarian algorithm, and the wrong matches are eliminated based on the RANSAC algorithm.

2. The underwater terrain image matching method based on saliency superpixels according to claim 1, characterized in that: In step 2, the superpixel block s is represented by pixel labels, eigenvalues ​​and centroid coordinates; in the grayscale terrain image, the grayscale value represents the elevation value of the location, and each superpixel block s is regarded as an equal-height area from a physical perspective.

3. The underwater terrain image matching method based on saliency superpixels according to claim 1, characterized in that: The step 4 is specifically as follows: for the salient superpixel set U of the underwater terrain reference image t For each superpixel block in the image, the connecting domain of the superpixel block is first extracted in the form of a mask, and then the circumscribed rectangular boundary of the connecting domain is calculated and intercepted to generate the corresponding sub-image; the texture information in each sub-image is extracted using a Log-Gabor filter, including the image texture in the connecting domain and the boundary information of the connecting domain structure; for the connecting domain of each superpixel block, the ellipse with the same second-order moment as the region is first solved, and then the angle θ between the major axis of the ellipse and the 0th axis is used as the rotation angle to rotate the connecting domain. At the same time, the connecting domain is rotated by (θ+180)° and then MIMs are constructed respectively. The two MIMs are then added to construct the final MIM descriptor; A two-dimensional Log-Gabor filter is represented as the product of a radial filter and an angular filter in polar coordinates: Where r represents the radial coordinate; θ represents the angular coordinate; f0 and θ0 represent the center frequency and directional angle of 2D-LGF respectively; σ r and σ θ denote the bandwidth of r and θ respectively.

Images