Side-scan sonar image feature extraction method and system

Abstract

The invention belongs to the technical field of image processing, and discloses a side-scan sonar image feature extraction method and system, and the method employs an LF-Net network to carry out the feature point extraction of two adjacent side-scan sonar images, and employs a KNN algorithm to carry out the matching of the feature points extracted by the LF-Net. According to the method, a transfer learning mode is introduced based on the deep learning network LF-Net to solve the sonar image matching problem, side-scan sonar image feature point detection is carried out by constructing the pre-training actual measurement data matching model, and the problem that the generalization ability is poor due to the fact that the number of side-scan sonar image data is small is solved; the KNN algorithm is combined to realize accurate matching of acoustic image feature points, the matching result is subjected to statistical analysis to reduce the influence of mismatching, and the matching robustness is improved. According to the method, two adjacent side-scan sonar images can be accurately matched without too many side-scan sonar image data, and a brand new solution thought is provided for underwater sonar image matching in a complex scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a side-scan sonar image feature extraction method and system. Background Art

[0002] With the rapid development of various marine activities and the continued development of coastal areas, the demand for underwater topographic maps is increasing. Side-scan sonar (SSS) detection technology, based on ship-borne and submarine-borne platforms, has become a mainstream means of detecting seabed topography and underwater targets due to its high resolution and full coverage. In different measurement scenarios, due to the limited swath width, the data of side-scan sonar sometimes fails to fully cover the target in a single side-scan image (single survey line). This makes it difficult to fully construct some large underwater targets, making it difficult to restore the target's full image. This, in turn, hinders important fields such as seabed topography and mapping, seabed resource development, and AUV navigation. Therefore, accurate feature extraction and matching of side-scan sonar images is of great significance.

[0003] During operation, a side-scan sonar records the intensity of backscattered echoes from the seafloor and generates a grayscale image based on the intensity. This image is called a side-scan sonar image. Traditional methods based on manually matching side-scan sonar images of adjacent areas are inefficient and highly subjective. Therefore, many researchers have conducted research on feature extraction and matching of side-scan sonar images.

[0004] The literature "Sylvie Daniel, Fabrice Le L′eannec. Side-Scan Sonar Image Matching. IEEE Journal of Oceanic Engineering, 1998, 23(3): 245-259P" proposed an algorithm for side-scan sonar image matching based on acoustic shadow features, and matched two images through the geometric shapes of the shadow features and their corresponding key position information in two side-scan sonar images. The literature "KHATER H A, GAD A S, OMRAN E A, et al. Enhancement matching algorithms using fusion of multiple similarity metrics for sonar images[J]. World Applied Sciences Journal, 2009, 6(6): 759-763." proposed a side-scan sonar image matching strategy that fuses SUSAN and Harris corner features. Under ideal conditions where the feature performance is stable and evenly distributed, this method shows excellent matching performance. The literature "VANDRISH P, VARDYA, WALKER D, et al. Side-scan sonar image registration for AUV navigation[C] / / 2011 IEEE Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies. IEEE, 2011: 1-7." tried to apply the SIFT algorithm to the side-scan sonar image matching process collected by AUV. The results showed that the registration effect was better when the side-scan image features were obvious and the noise was less. The literature "Yang Fanlin. Multi-beam and side-scan sonar data fusion and its application in seabed sediment classification[D]. Wuhan University, 2003." proposed a registration method for homologous feature points based on isobaths and contour lines.The literature "Zhao J, Tao W, Zhang H, et al. Study on side scan sonar image matching based on the integration of SURF and similarity calculation of typical areas[C]. Oceans 2010 IEEE-Sydney. IEEE, 2010: 1-4P." proposes a side scan sonar image matching method based on the SURF algorithm and the similarity calculation of typical areas. First, it performs rough matching of adjacent strips through the SURF algorithm, and then performs geometric correction to make the matching image and the reference image have the same observation conditions. Finally, precise matching is completed by calculating the similarity of typical areas in the two corrected images. The above methods have all achieved feature extraction and matching of side scan sonar images through traditional methods to varying degrees, but they generally have poor generalization ability, and there will be more false matching points when there are many noise points, invalid features, and low signal-to-noise ratio in side scan sonar images, resulting in a significant deterioration of the matching effect.

[0005] In recent years, the booming development of deep learning has prompted scholars to explore the potential of convolutional neural networks in sonar image feature extraction, aiming to obtain deep and multi-dimensional features through deep learning to promote precise matching between images. However, the current research on the application of convolutional neural networks in the field of sonar image matching is still scarce.

[0006] The literature "VALDENEGRO-TORO M. Improving sonar image patch matching via deep learning[C] / / 2017 European Conference on Mobile Robots(ECMR). IEEE, 2017:1-6." first applied the idea of deep learning to the problem of sonar image matching and proposed a method to solve the sonar image matching problem using the idea of binary classification. The literature "YANG Haibo. Research on Side-scan Sonar Image Preprocessing and Matching Method[D]. Harbin Engineering University, 2020." proposed a side-scan sonar image template matching method based on the U-net (Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C] / / Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer International Publishing, 2015:234-241.) network, mainly for the side-scan sonar image matching and stitching scenario of the rough seabed.The literature "Xiaoteng Zhou. Research on Side-scan Sonar Image Matching and UUV-assisted Navigation Application Based on Deep Learning [D]. Harbin Institute of Technology, 2022. DOI: 10.27061 / d.cnki.ghgdu.2022.001205." explored the simple and complex environments involved in underwater navigation tasks, and investigated the feasibility of the combination of deep learning matching algorithms in which the performance of feature detectors in different scenarios was fused in AffNet (Chen X, Fu C, Tie M, et al. AFFNet: An attention-based feature-fused network for surface defect segmentation [J]. Applied Sciences, 2023, 13(11): 6428.) and HardNet (Chao P, Kao CY, Ruan YS, et al. Hardnet: A low memory traffic network [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2019: 3552-3561.). The above method trains the model in a data-driven manner and achieves the matching of side-scan sonar images. Although the above method achieves the matching of side-scan sonar images, it requires a large amount of side-scan sonar image data for model training, otherwise the matching effect is poor.

[0007] LF-Net is an end-to-end image matching network proposed by Yuki et al. in 2018 and is an unsupervised learning-based image matching network. The end-to-end architecture enables this method to eliminate the need for independent training of the feature point detector network and the descriptor network, thus avoiding the limitations of the previous step-by-step processing of feature point detectors and descriptors. In addition, it also differs from supervised learning networks that rely on SIFT detectors. For example, the LIFT network needs to pre-extract initial feature points during the self-descriptor training phase and enhance the uniqueness and discriminability of feature points synchronously during the detector training process. Similar to SIFT, LF-Net also selects a number of feature points on two images to be matched and then describes the selected feature points to finally achieve matching. During the training process, recursive precision evaluation is carried out using pre-given pose labels to optimize the learning algorithms of the feature point detector and the feature point descriptor. The core construction of LF-Net revolves around two key components: (1) a multi-scale fully convolutional module, which is responsible for generating the coordinate, direction, and scale information of feature points; (2) a feature point description module, which is used to output the feature vectors of feature points. These two are respectively called the detection module and the description module in LF-Net.

[0008] Through the above analysis, the problems and defects existing in the prior art are as follows:

[0009] (1) During the current side-scan sonar data acquisition process, restricted by the swath width, there is a problem that a single image cannot fully cover the target, resulting in difficulty in completely constructing some underwater targets with larger scales.

[0010] (2) The method based on traditional manual matching of adjacent area side-scan sonar images has problems of low efficiency and strong subjectivity; at the same time, this method has poor generalization ability, generates more false matching points, and the matching effect is poor.

[0011] (3) In the side-scan sonar image matching method based on deep learning, the number of datasets is small, while the model needs to use a large amount of side-scan sonar image data for training, otherwise its matching effect is poor. Summary of the Invention

[0012] To overcome the problems existing in the related art, the disclosed embodiments of the present invention provide a method and system for extracting side-scan sonar image features, especially a method and system for extracting side-scan sonar image features based on LF-Net. The technical solutions are as follows:

[0013] The present invention is implemented as follows. The method for extracting side-scan sonar image features includes the following steps:

[0014] S1, LF-Net training: Construct a network architecture based on a dual-branch, and use a dual-modal loss to train the detection network to construct a loss function;

[0015] S2, feature map generation: LF-Net uses the ResNet architecture to input the side-scan sonar image into the trained multi-scale fully convolutional network, and outputs an image o with feature points;

[0016] S3, feature point detection: Use LF-Net for feature point detection, and the detection network is a serial structure;

[0017] S4, feature description: Calculate the direction map of each feature point through the arctan function; construct a quadruple, and perform feature extraction operations on the image area to obtain a feature description map;

[0018] S5, KNN homologous point matching: For the detected feature points and the obtained feature description map, perform homologous point matching of two adjacent sonar images through the KNN algorithm.

[0019] In step S1, in constructing the network architecture based on a dual-branch, the two branches share the same set of parameters, and the input includes the images I i , I jand the depth and pose information obtained through the structured light algorithm; branch i undertakes differentiable operation tasks to backpropagate gradients and drive model optimization; branch j performs non-differentiable operations, and the non-differentiable operations include feature point extraction and generating supervision targets for the first branch.

[0020] In step S1, the loss function of LF-Net is divided into an image-level loss function and a patch-level loss function;

[0021] The image-level loss function is used to optimize the feature point detection network in LF-Net. By taking the score map S j calculated on the image I j transformed to the image I i as a training sample, after the rigid transformation ω process, the score map S j is obtained, and K feature points are extracted from the score map. The score map S j is Gaussian filtered with a Gaussian kernel of 0.5 to obtain a smoothed score map. The expression of the loss function is:

[0022] L im (S i , S j ) = |S i - g(ω(S j ))| 2

[0023] In the formula, S i is the score map of the i-th image, representing the probability of feature point response; S j is the score map of the j-th image; g is non-differentiable Gaussian filtering, which is applied with a Gaussian kernel at the feature points for cleaning; ω is the rigid transformation;

[0024] The patch-level loss function is used to optimize the local descriptors generated by the feature description network, including the matching feature point loss L pair , the geometric consistency loss L geom , and the triplet loss L tri ;

[0025] The matching feature point loss L pair minimizes the distance between the feature vectors of the same key point in two images, ensuring that the descriptors remain consistent after transformation; for the K feature points proposed in each iteration of branch i, using the mutual relationship between the current iteration number and the images I i , I j , the coordinates of the K feature points are transformed from the coordinate system of the image I i to the coordinate system of the image I j through the rigid transformation; after the coordinate transformation is completed, according to the scale and direction information obtained from branch j, the transformation results in the new coordinate system are fused, and from I jPrecisely select the image region to obtain two image patches and Extract features from the image patches respectively, and obtain features and k represents the number of pairs of feature point matches, is the feature descriptor of the k-th key point in the i-th image, is the feature descriptor corresponding to k after geometric transformation of the j-th image, and obtain the image patch-level loss function;

[0026]

[0027] LF-Net uses the geometric consistency loss L geom , to make the scale and direction of feature points consistent in different images, ensuring that LF-Net has scale invariance and rotation invariance; L geom consists of two parts: scale loss and direction loss: makes the directions of the matching key points consistent, makes the scales of the matching key points consistent; represents the scale of the k-th key point in the i-th image, represents the direction of the k-th key point in the i-th image; represents the scale of the corresponding key point after geometric transformation of the j-th image, represents the direction of the corresponding key point after geometric transformation of the j-th image, λ ori and λ scale are hyperparameters, which respectively control the influence of direction and scale loss. The loss function L geom is as follows;

[0028]

[0029] LF-Net uses the triplet loss function to optimize the feature descriptor. For non-corresponding image pairs, progressive mining strategy is used as negative samples for training, making the matching positive samples closer and the non-matching negative samples farther; a minimum margin is set through the hyperparameter C to ensure that the distance between matching points and non-matching points is large enough; represents the feature descriptor of the k-th key point in the i-th image, represents the positive sample of the matching key point after geometric transformation of the j-th image, represents a random negative sample in the j-th image, and C represents the Margin hyperparameter in the triplet loss, which controls the separation degree of positive and negative samples; the triplet loss L tri is expressed mathematically as follows;

[0030]

[0031] where k′≠k and C = 1;

[0032] After three loss functions, the loss functions for training the detection network and the description network are obtained as follows:

[0033] L det = L im + λ pair L pair + L geom

[0034] L desc = L tri .

[0035] In step S2, the ResNet architecture includes three modules, namely the convolutional module, the residual module, and the fully connected module. Each module embeds multiple groups of 5×5 convolutional kernels; a batch normalization step is implemented after each convolutional operation; Leaky-ReLU is used as the activation function, and the number of output channels of all convolutional layers is fixed at 16, and the output dimension is the same as the input; the convolutional module performs preliminary feature extraction, reduces the spatial size of the input data, and improves the computational efficiency; the residual module solves the problem of gradient disappearance in deep neural networks, enables information to flow efficiently between layers, and enables the network to more easily learn the residuals through identity mapping; the fully connected module reduces the number of parameters through global average pooling, improves the generalization ability, and makes the final classification decision.

[0036] In step S3, feature point detection includes:

[0037] LF-Net performs multi-scale detection based on the feature map o; the feature map o is scaled N times, and the scaling range is N = 5, The scaled feature maps are convolved with a 5×5 convolutional kernel to obtain N score maps h N (1 ≤ n ≤ N). Each of these N score maps corresponds to N score maps at different scales. The Softmax operator with a size of 15×15 is used to perform convolution operations on the N score maps respectively to achieve differentiable non-maximum suppression, and N sharpened score maps after the Softmax operator are obtained

[0038] LF-Net resizes each back to the size of the original image, and the resized score map is The score maps at the same scale are fused using an operator similar to softmax to fuse all to form the final score map in the scale space;

[0039]

[0040] In the formula, ⊙ represents the Hadamard product; after scale fusion, the K pixels with the highest scores are selected from the finally fused scale-invariant score map S as feature points, and local softargmax is applied to obtain sub-pixel accuracy.

[0041] Furthermore, the Softmax operator is used to normalize the input values into a probability distribution such that the sum of the output values is 1;

[0042] The function definition of the Softmax operator is as follows, where softmax(z i ) represents the value of the i-th element after the Softmax transformation, representing a certain probability value in the probability distribution;

[0043]

[0044] represents applying an exponential transformation to the input z i to ensure non-negative values and amplify the influence of larger z i values; represents the sum of the exponential transformations of all input elements, which is used for normalization such that the sum of all outputs is 1, forming a probability distribution.

[0045] In step S4, feature description includes: by applying a 5×5 convolution to each pixel on the feature map o, two values are output for each pixel as sine and cosine, and the orientation map θ of each feature point is calculated through the arctan function;

[0046] Descriptor extraction: by analyzing the score map S, K feature points with the highest scores are selected and coordinate data is obtained; the key points are combined with the scale and orientation information from the scale estimation map and the orientation estimation map to form a quadruple structure p k =(x, y, s, θ) k , and feature extraction operations are performed on the corresponding image regions.

[0047] Furthermore, the orientation map θ of each feature point is calculated through the arctan function. The direction calculated by arctan is a continuously varying angle value, without direction jumps, providing smooth direction estimation, being applicable to various scales, rotations, and perspective changes, improving the robustness of the feature points; and LF-Net needs to find the same key points in images with different perspectives and rotational changes. The arctan calculation of the direction angle can ensure that the feature points still point in the same direction after rotation transformation, improving rotational invariance; its calculation formula is as follows:

[0048]

[0049] where θ represents the direction angle of the key point, and f x represents the gradient of the feature point in the x direction, and f y represents the gradient of the feature point in the y direction.

[0050] In step S5, the KNN homologous point matching includes: for the detected feature points and the obtained feature description map, performing homologous point matching of two adjacent sonar images through the KNN algorithm;

[0051] Before matching, the feature descriptors output by LF-Net are processed in the way of L2 normalization;

[0052] For two side-scan sonar images to be matched, matching feature points are performed through the KNN algorithm, and the Euclidean distance is calculated for feature point matching. The KNN algorithm is used for image registration, target detection, and clustering tasks; in side-scan sonar image matching, this method accurately calculates the similarity of feature points in adjacent images and improves the accuracy of stitching and classification; K feature points and the corresponding descriptors D_A and D_B are extracted respectively; the descriptor set D_B is used as the search database, and the descriptor set D_A is used as the query set; for each descriptor obtained from any sonar image, K nearest neighbor descriptors are found among the descriptors obtained from the other sonar image;

[0053]

[0054] where is the coordinate of feature point A i on the X-axis, is the coordinate of feature point B j on the X-axis; is the coordinate of feature point A i on the Y-axis, is the coordinate of feature point B j on the Y-axis;

[0055] The nearest neighbor is determined by calculating the Euclidean distance between descriptors. Calculate the similarity between each query descriptor D_A i and all descriptors D_B j in the database, and select the two points with the highest similarity as homologous points for matching.

[0056] Another object of the present invention is to provide a side-scan sonar image feature extraction system, which is used to regulate the side-scan sonar image feature extraction method. The system includes:

[0057] The LF-Net training module is used to construct a network architecture based on a double-branch and train the detection network using a dual-modal loss;

[0058] The feature map generation module is used for LF-Net to input the side-scan sonar image into the trained multi-scale fully convolutional network using the ResNet architecture and output an image o with feature points.

[0059] The feature point detection module is used for feature point detection using LF-Net, and the detection network is a serial structure.

[0060] The feature description module is used to calculate the direction map of each feature point through the arctan function; construct a quadruple and perform feature extraction operations on the image region to obtain a feature description map.

[0061] The KNN homologous point matching module is used to perform homologous point matching of two adjacent sonar images on the detected feature points and the obtained feature description map through the KNN algorithm.

[0062] Combining all the above technical solutions, the beneficial effects of the present invention are as follows:

[0063] To solve the problem of few data sets and difficult training models in the side-scan sonar image matching method based on deep learning, the present invention proposes a method that uses the deep learning network LF-Net to introduce transfer learning, constructs a pre-trained measured data matching model for outdoor scenes and migrates it to side-scan sonar images, performs an automatic feature point extraction algorithm on side-scan sonar image data, and realizes accurate matching of feature points through the combined KNN matching algorithm. This method can accurately match two adjacent side-scan sonar images without too much side-scan sonar image data. The present invention verifies the feasibility of this method through experiments and compares it with traditional methods, confirming that the deep learning network based on LF-Net can better complete the side-scan sonar image matching task and provides a new solution idea for underwater sonar image matching in complex scenes.

[0064] To solve the problem that in the process of side-scan sonar data collection, due to the limitation of the scan width, the data has the problem that a single image cannot completely cover the target object, the present invention introduces a transfer learning method based on the deep learning network LF-Net to solve the sonar image matching problem. First, the present invention detects the feature points of side-scan sonar images by constructing a pre-trained measured data matching model, solving the problem of poor generalization ability caused by the small number of side-scan sonar image data; secondly, combines the KNN algorithm to achieve accurate matching of sonar image feature points, and reduces the influence of false matching by statistical analysis of the matching results, improving the robustness of the matching.

[0065] The present invention proposes a method for feature extraction and matching of side-scan sonar images based on LF-Net, which solves the problems of fewer datasets and difficult training models in the side-scan sonar image matching method based on deep learning. The present invention also proposes a method of introducing transfer learning using the deep learning network LF-Net, constructing a measured data matching model for a pre-trained outdoor scene and migrating it to side-scan sonar images, performing an automatic feature point extraction algorithm on side-scan sonar image data, and achieving accurate matching of feature points through a combined KNN matching algorithm, reducing the dependence on a large amount of training data in the deep learning method.

[0066] The present invention fully considers that in the current project, in the case of large-scale and high-resolution measurement scenarios, a single survey line is difficult to cover the entire target area, and it is necessary to match sonar data for two adjacent survey lines. However, the effects of existing mainstream algorithms including the ORB algorithm and the Harris algorithm are poor. Through this method, the matching quality of sonar images can be effectively improved.

[0067] The present invention reduces data dependence: there is no need to train a model with a large number of sonar images, reducing the costs of data acquisition and annotation; improves efficiency: applicable to complex underwater scenarios (such as high noise and low signal-to-noise ratio), improving the efficiency of applications such as seabed terrain mapping, underwater target detection, and AUV navigation; has market potential: can be applied to fields such as marine resource development, military detection, and sunken ship salvage, promoting the intelligent upgrade of underwater detection technology, and having high market transformation potential.

[0068] The current field of sonar image matching mainly relies on traditional algorithms (such as SIFT, ORB, Harris) or deep learning models that require a large amount of data training. This case introduces LF-Net combined with transfer learning into side-scan sonar image matching, solving the following gaps: small sample generalization ability: the performance of traditional methods drops sharply when the amount of data is small, while transfer learning uses a pre-trained model (such as the Outdoor scene) to adapt to sonar images, breaking through the data limitation; end-to-end optimization: the detection and description modules of LF-Net are integrated, avoiding the error accumulation problem of traditional step-by-step processing, and filling the application gap of unsupervised learning in sonar image matching.

[0069] Traditional methods face two major problems in side-scan sonar image matching: false matching in noise and low signal-to-noise ratio environments: Traditional algorithms (such as SIFT, Harris) rely on artificial features and have a high false matching rate in complex underwater scenarios; poor model generalization ability: Deep learning requires a large amount of labeled data, and the acquisition cost of sonar images is high, making it difficult to apply the model in practice. In this paper, the problem of model generalization with small samples is solved through transfer learning, and false matching is effectively suppressed by combining the KNN algorithm with statistical analysis. Experiments prove that its robustness is significantly better than traditional methods, overcoming the above-mentioned long-standing technical bottlenecks. The traditional view holds that deep learning depends on a large amount of data: Sonar image data is scarce, and it is difficult to directly apply deep learning; In this paper, the pre-trained model is adapted to sonar images through transfer learning, verifying the feasibility of deep learning training in the scenario of scarce sonar image samples and breaking through the technical prejudice. Brief Description of the Drawings

[0070] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure;

[0071] Figure 1 It is a flowchart of the method for extracting side-scan sonar image features provided by an embodiment of the present invention;

[0072] Figure 2 It is a structural diagram of LF-Net provided by an embodiment of the present invention;

[0073] Figure 3 It is a schematic diagram of side-scan sonar data 1 provided by an embodiment of the present invention. Figure (a) is the sonar image of the left survey line of adjacent survey lines, and Figure (b) is the sonar image of the right survey line of adjacent survey lines;

[0074] Figure 4 It is a schematic diagram of side-scan sonar data 2 provided by an embodiment of the present invention. Figure (a) is the sonar image of the left survey line of adjacent survey lines, and Figure (b) is the sonar image of the right survey line of adjacent survey lines;

[0075] Figure 5 It is a schematic diagram of the feature point detection result of side-scan sonar data 1 provided by an embodiment of the present invention. Figure (a) is the sonar image of the left survey line of adjacent survey lines, and Figure (b) is the sonar image of the right survey line of adjacent survey lines;

[0076] Figure 6 It is a schematic diagram of the feature point detection result of side-scan sonar data 2 provided by an embodiment of the present invention. Figure (a) is the sonar image of the left survey line of adjacent survey lines, and Figure (b) is the sonar image of the right survey line of adjacent survey lines;

[0077] Figure 7It is a diagram for describing the direction of side-scan sonar data 1 provided by an embodiment of the present invention. Figure (a) is a sonar image of the left survey line of adjacent survey lines, and Figure (b) is a sonar image of the right survey line of adjacent survey lines;

[0078] Figure 8 It is a diagram for describing the direction of side-scan sonar data 2 provided by an embodiment of the present invention. Figure (a) is a sonar image of the left survey line of adjacent survey lines, and Figure (b) is a sonar image of the right survey line of adjacent survey lines;

[0079] Figure 9 It is a diagram for describing the scale of side-scan sonar data 1 provided by an embodiment of the present invention. Figure (a) is a sonar image of the left survey line of adjacent survey lines, and Figure (b) is a sonar image of the right survey line of adjacent survey lines;

[0080] Figure 10 It is a diagram for describing the scale of side-scan sonar data 2 provided by an embodiment of the present invention. Figure (a) is a sonar image of the left survey line of adjacent survey lines, and Figure (b) is a sonar image of the right survey line of adjacent survey lines;

[0081] Figure 11 It is a matching result diagram of side-scan sonar data 1 provided by an embodiment of the present invention;

[0082] Figure 12 It is a matching result diagram of side-scan sonar data 2 provided by an embodiment of the present invention;

[0083] Figure 13 It is a matching result diagram of the ORB algorithm for side-scan sonar data 1 provided by an embodiment of the present invention;

[0084] Figure 14 It is a matching result diagram of the Harris algorithm for side-scan sonar data 1 provided by an embodiment of the present invention;

[0085] Figure 15 It is a matching result diagram of the ORB algorithm for side-scan sonar data 2 provided by an embodiment of the present invention;

[0086] Figure 16 It is a matching result diagram of the Harris algorithm for side-scan sonar data 2 provided by an embodiment of the present invention. Detailed implementation manners

[0087] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the detailed implementation manners of the present invention with reference to the accompanying drawings. Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific implementations disclosed below.

[0088] The innovation points of the side-scan sonar image feature extraction method and system provided by the embodiments of the present invention are as follows:

[0089] 1. Deep feature adaptation based on transfer learning: Combine LF-Net with transfer learning, and use the pre-trained natural scene model (Outdoor scene) to be transferred to underwater sonar images, which solves the problems of small side-scan sonar data volume and high annotation cost.

[0090] 2. End-to-end multi-scale feature fusion architecture: Adopt the end-to-end unsupervised architecture of LF-Net to realize the integrated optimization of feature detection and description, and break through the error accumulation problem of traditional step-by-step processing (such as SIFT phased detection and description).

[0091] To solve the problem of the small number of datasets and the difficulty of training models in the side-scan sonar image matching method based on deep learning, the present invention proposes a method that introduces transfer learning using the deep learning network LF-Net, constructs a measured data matching model of a pre-trained outdoor scene and transfers it to side-scan sonar images, performs an automatic feature point extraction algorithm on side-scan sonar image data, and realizes accurate matching of feature points through the combined KNN matching algorithm. The proposed method can accurately match two adjacent side-scan sonar images without too much side-scan sonar image data. The present invention verifies the feasibility of this method through experiments and compares it with traditional methods, and proves that the deep learning network based on LF-Net can better complete the side-scan sonar image matching task, providing a new solution idea for underwater sonar image matching in complex scenes.

[0092] To solve the problem that in the process of side-scan sonar data acquisition, due to the limitation of the scan width, the data has the problem that a single image cannot completely cover the target object, the present invention introduces transfer learning using the deep learning network LF-Net to solve the sonar image matching problem. First, detect the feature points of side-scan sonar images by constructing a pre-trained measured data matching model, which solves the problem of poor generalization ability caused by the small number of side-scan sonar image data; secondly, combine the KNN algorithm to realize the accurate matching of sonar image feature points, and reduce the influence of false matching by statistical analysis of the matching results, and improve the robustness of the matching.

[0093] In Embodiment 1, when the side-scan sonar measures an underwater target, there will be an overlapping part between the adjacent side-scan sonar images obtained by two adjacent scan lines, and the similar parts show the same image features. The present invention uses the LF-Net network to extract feature points from two adjacent side-scan sonar images and uses the KNN algorithm to match the feature points extracted by LF-Net. The specific feature matching method process is as Figure 1 shown.

[0094] The side-scan sonar image feature extraction method provided by the embodiments of the present invention includes the following steps:

[0095] S1, LF-Net Training: Construct a network architecture based on dual branches, and use dual-modal loss to train the detection network to construct a loss function.

[0096] As Figure 2 shown, LF-Net constructs a network architecture based on dual branches for effective training. The two branches share the same set of parameters, and their inputs include the images I i , I j and the depth and pose information obtained through the Structure from Motion (SfM) algorithm. Among them, branch i undertakes differentiable operation tasks to backpropagate gradients and drive model optimization; while branch j performs non-differentiable operations such as feature point extraction and generating supervision targets for the first branch.

[0097] The loss function of LF-Net is divided into two components: (1) the image-level loss function, and (2) the patch-level loss function. For the key point detection network, the image-level loss function is crucial, but it may indirectly affect the patch extraction ability during iterative training. Therefore, LF-Net strategically uses dual-modal loss to train the detection network. At the same time, the training of the description network is only limited to the patch-level loss function.

[0098] The image-level loss function is mainly used to optimize the feature point detection network in LF-Net to ensure the consistency and accuracy of feature points under different viewpoints and geometric transformations. It transforms the score map s j calculated on image I j to image I i as a training sample, where ω in the formula represents this rigid transformation. After the ω process, the transformed score map S j can be obtained, and K feature points are extracted from its score map. A smoothed score map is obtained by applying Gaussian filtering with a Gaussian kernel of 0.5 to the score map S j . This series of operations is called g, and the mathematical expression of this loss function is:

[0099]

[0100] In the formula, S i is the score map of the i-th image, representing the probability of feature point response; S j is the score map of the j-th image; g is non-differentiable Gaussian filtering, which is applied with a Gaussian kernel at the feature points for cleaning; ω is the rigid transformation;

[0101] The image block-level loss function focuses on optimizing the feature description network. Its main role is to optimize the local descriptors generated by the network, enhance the discriminative ability of the feature descriptors so that they can be accurately matched between different images. For the K feature points proposed in each iteration of branch i, the mutual relationship between the current iteration number and images I i , I j can be utilized. The coordinates of the K feature points are transformed from the coordinate system of image I i to the coordinate system of image I j through a rigid transformation. This process is similar to the transformation process in the image-level loss function, but the transformation direction is exactly opposite. After completing the coordinate transformation, according to the scale and direction information obtained from branch j, the transformation results in the new coordinate system are fused, and the image regions are accurately selected from I j to obtain two image blocks and Feature extraction is performed on them respectively, and features and can be extracted. Here, k represents the number of pairs of matching feature points, and the image block-level loss function can be obtained. Its mathematical expression is:

[0102]

[0103] By forcing the estimated direction to be the same as the direction of the transformed set of matching key points, LF-Net can obtain the loss function L geom , and its mathematical expression is as follows:

[0104]

[0105] In the formula, and are the scale and direction after the transformation of the key point direction respectively, and λ ori and λ scale are weight parameters.

[0106] Triplet loss function: To better perform the learning of the description network, LF-Net considers non-corresponding image pairs as negative samples for progressive mining strategy training, and introduces the triplet loss L tri , and its mathematical expression is as follows:

[0107]

[0108] Among them, k′≠k, C = 1;

[0109] Through the above three loss functions, the loss function for training the detection network and the description network can be deduced as follows:

[0110] L det = L im+λ pair L pair +L geom (5)

[0111] L desc =L tri (6)

[0112] S2, Feature map generation: LF-Net uses the ResNet architecture to input the sidescan sonar image into the trained multi-scale fully convolutional network, and outputs an image o with feature points;

[0113] As Figure 2 shown, the sidescan sonar image is input into the multi-scale fully convolutional network, i.e., the detection network mentioned above, and an image o with feature points, i.e., the feature map, can be output. LF-Net uses the ResNet architecture to perform this process. This architecture contains three modules, and each module is embedded with multiple groups of 5x5 convolutional kernels. After each convolution operation, a batch normalization step is immediately implemented. Leaky-ReLU is used as the activation function, and the number of output channels of all convolutional layers is fixed at 16, and its output dimension is consistent with the input, which can keep the resolution of the entire feature extraction part unchanged throughout the full image.

[0114] S3, Feature point detection: Use LF-Net for feature point detection, and the detection network is a serial structure;

[0115] Use LF-Net for feature point detection. The detection network structure is a serial structure, and the steps are as follows. To ensure the scale invariance of the feature points, LF-Net performs multi-scale detection based on the feature map o. They first scaled the feature map o N times, and the scaling range is N = 5, Then, perform convolution operations on the feature maps after N times of scaling using a 5*5 convolutional kernel, and N score maps h N (1 ≤ n ≤ N) can be obtained. Each of these N score maps corresponds to the score maps at N different scales. Respectively use a Softmax operator with a size of 15*15 on these N score maps and perform convolution operations on them to achieve differentiable non-maximum suppression, and N score maps sharpened by the Softmax operator can be obtained The Softmax operator normalizes the input values into a probability distribution such that the sum of the output values is 1. In keypoint detection, the Softmax operator can enhance the saliency of keypoints, making the local maxima more prominent while suppressing other response values. Through the Softmax operator, the network can more clearly distinguish keypoints from other non-keypoint regions, thereby improving the accuracy and robustness of keypoint detection. This ability to enhance saliency enables the network to better handle noise and interference in complex scenes and improve the reliability of keypoint detection. The function definition of the Softmax operator is as follows:

[0116]

[0117] Denotes the exponential transformation of the input z i to ensure non-negative values and amplify the influence of larger z i values; Denotes the sum of the exponential transformations of all input elements, which is used for normalization so that the sum of all outputs is 1, forming a probability distribution.

[0118] Since the result of non-maximum suppression is scale-dependent, LF-Net resizes each back to the size of the original image, and the resized score map is The score maps of the same scale are fused using an operator similar to softmax to fuse all to form the final score map in the scale space. The process is as follows:

[0119]

[0120] In the formula, ⊙ represents the Hadamard product; after scale fusion, the top K pixels with the highest scores are selected from the final fused scale-invariant score map S as keypoints and local softargmax is applied to obtain sub-pixel accuracy.

[0121] S4, Feature description: Calculate the orientation map of each keypoint through the arctan function; construct a quadruple and perform feature extraction operations on the image region to obtain the feature description map;

[0122] The feature description part mainly solves for the orientation. By applying a 5*5 convolution to each pixel on the feature map o, each pixel outputs two values, which can be considered as sine and cosine, and the orientation map θ of each keypoint is calculated through the arctan function.

[0123] θ(x,y) = arctan(sin(x,y),cos(x,y)) (9)

[0124] The method for extracting descriptors is to analyze the score map S, screen out K feature points with the highest scores, and obtain their coordinate data. Subsequently, these key points are combined with the scale and direction information from the scale estimation map and the direction estimation map to form a quadruple structure p k =(x, y, s, θ) k . Based on this quadruple, detailed feature extraction operations are performed on the corresponding image regions.

[0125] S5, KNN homologous point matching: For the detected feature points and the obtained feature description map, the KNN algorithm is used to perform homologous point matching between two adjacent sonar images;

[0126] For the feature points detected by S3 and the feature description map obtained by S4, the KNN algorithm is used to perform homologous point matching between two adjacent sonar images. The specific process is as follows. Before matching, the feature descriptors output by LF-Net are normalized. Here, we use L2 normalization for normalization to ensure the comparability of descriptors with different scales and directions and avoid matching deviations caused by dimensional differences.

[0127] For two side-scan sonar images to be matched, K feature points and their corresponding descriptors (D A and D B ) are extracted respectively. The descriptor set D B of the right image is used as the search database, and the descriptor set D_A of the left image is used as the query set. For each descriptor obtained from any sonar image, its K nearest neighbor descriptors are found among the descriptors obtained from the other sonar image.

[0128]

[0129] In the formula, is the coordinate of feature point A i on the X-axis, is the coordinate of feature point B j on the X-axis; is the coordinate of feature point A i on the Y-axis, is the coordinate of feature point B j on the Y-axis;

[0130] Next, the Euclidean distance between descriptors is calculated to determine the nearest neighbor, and the similarity between each query descriptor (D_A i ) and all descriptors (D_B j ) in the database is calculated. The two points with the highest similarity are selected as homologous points for matching.

[0131] Example 2, The side-scan sonar image feature extraction system provided by the embodiment of the present invention includes:

[0132] The LF-Net training module is used to construct a dual-branch network architecture, train the detection network using a dual-modal loss, and describe that the training of the network is only limited to the image patch-level loss function;

[0133] The feature map generation module is used for LF-Net to input the side-scan sonar image into the trained multi-scale fully convolutional network using the ResNet architecture and output an image O with feature points;

[0134] The feature point detection module is used to detect feature points using LF-Net, and the detection network is a serial structure;

[0135] The feature description module is used to calculate the orientation map of each feature point through the arctan function; construct a quadruple, perform feature extraction operations on the corresponding image region, and obtain a feature description map;

[0136] The KNN homologous point matching module is used to perform homologous point matching of two adjacent sonar images on the feature points obtained by the feature point detection module and the feature description map obtained by the feature description module through the KNN algorithm

[0137] In the above embodiments, the descriptions of each embodiment have their own emphases. For parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0138] To further prove the positive effects of the above embodiments, the present invention conducts the following experiments based on the above technical solutions.

[0139] The article adopts two groups of experiments. The target object types of Data 1 and Data 2 are both sunken ships. The sunken ship structure of Data 1 is relatively simple, while the sunken ship structure of Data 2 is more complex and has higher noise. Through experimental verification, it can be found that the side-scan sonar image feature extraction and matching method based on LF-Net proposed in this case has more significant advantages in high-noise and complex target scenarios (Data 2), verifying its end-to-end architecture and the generalization ability of transfer learning; while the traditional algorithm can be applied in simple scenarios (Data 1), but it cannot adapt to complex environments.

[0140] 1. Introduction to experimental data

[0141] The experimental data is as Figure 3 and Figure 4As shown in the figure, the data is sourced from the SeabedObjects (Ship and Airplane) side-scan sonar dataset publicly released by Hohai University. This dataset includes data from multiple sonar device suppliers such as L-3 Klein Associates, EdgeTech, Lcocean, Hydro-tech Marine, and Tritech. The dataset contains 385 ship images and 62 airplane images, and all images are directly taken from the original large-area side-scan sonar maps without any processing.

[0142] Both Data 1 and Data 2 are images of sunken ships obtained by side-scan sonar, specifically as shown in Figure 3 and Figure 4 shown.

[0143] In this experiment, the network was built using the Python language and the TensorFlow open-source machine learning platform on Pycharm. The computer configuration is Windows 10 operating system, 16G DDR5 memory, NVIDIA RTX 1650 graphics card, and i5-12490F CPU.

[0144] In this experiment, transfer learning was carried out through the pre-trained model Outdoor, and a measured data matching model for pre-trained outdoor scenes was constructed and transferred to underwater acoustic images. In particular, feature point detection was performed on data represented by side-scan sonar images. In this test, the value of K, the number of feature points selected for each image, was set to 500. By setting a smaller number of feature points, the running speed and memory occupancy of the algorithm can be improved.

[0145] 2. Experimental Results and Analysis

[0146] Figure 5 and Figure 6 are pictures of feature point detection for Data 1 and Data 2 respectively.

[0147] As shown in Table 1, after feature point detection on the two groups of data, direction description and scale description were also carried out on the two groups of data respectively, and feature description images were obtained using the description network. Figure 7 and Figure 8 are direction description images, Figure 9 and Figure 10 are scale description images.

[0148] Based on the feature point detection of side-scan sonar images by constructing a pre-trained measured data matching model, the matching of two groups of side-scan sonar images was achieved through the KNN matching algorithm, as shown in Figure 11 and Figure 12 shown.

[0149] Table 1 LF-Net Number of Matched Points

[0150] Data Number of matching points Data 1 166 Data 1 183

[0151] As Figure 5 and Figure 6 are the results of sonar image feature extraction implemented by LF-Net. It can be clearly observed that the features extracted by LF-Net are mainly concentrated in areas with obvious terrain changes and object edges, and are less distributed in flat terrain areas or sonar shadow areas. In non-feature areas, due to factors such as different reflection intensities of the seabed terrain, LF-Net also detects a large number of feature points in the background area, which are considered noise points during the registration process and will be removed during the subsequent matching process. As Figure 11 and Figure 12 shown, the KNN algorithm is performed on the generated feature points to match the feature points of two adjacent sidescan sonar images pairwise. It can be seen that the KNN algorithm removes most of the noise points, the matching point pairs are relatively dense, and the matching effect is good.

[0152] The matching effects of the SIFT algorithm and the ORB algorithm are respectively compared and analyzed with the proposed algorithm in the experiment, as Figures 13 - 16 shown. The same as the process of feature point detection by LF-Net, the value of the number K of feature points selected for each image is set to 500. Both algorithms perform the RANSAC algorithm to remove the mismatched point pairs after matching, and the distance threshold of RANSAC is set to 8.

[0153] It is found through Table 2 that by using the pre-trained model of LF-Net outdoor data for feature point search, without relying on additional sidescan sonar image data to train the model, its matching effect is improved by 23 pairs and 67 pairs respectively compared with the ORB algorithm and the Harris algorithm on Data 1, and by 66 pairs and 16 pairs respectively on Data 2. It can be proved that the combination of using the pre-trained model of LF-Net outdoor data to automatically extract feature points from sidescan sonar image data and the KNN matching algorithm can better achieve the sidescan sonar image matching task.

[0154] Table 2 Number of matching points of other methods

[0155] Data ORB Harris Data 1 143 99 Data 2 117 167

[0156] As mentioned above, only the relatively optimal specific implementation manners of the present invention are described, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be covered by the protection scope of the present invention.

Claims

1. A method for extracting side-scan sonar image features, characterized in that, The method includes the following steps: S1, LF-Net training: Construct a network architecture based on dual branches, and use a dual-modal loss to train the detection network to construct a loss function; S2, feature map generation: LF-Net uses the ResNet architecture to input the side-scan sonar image into the trained multi-scale fully convolutional network, and outputs an image o with feature points; S3, feature point detection: Use LF-Net for feature point detection, and the detection network is a serial structure; S4, feature description: Calculate the direction map of each feature point through the arctan function; construct a quadruple, and perform feature extraction operations on the image region to obtain a feature description map; S5, KNN homologous point matching: For the detected feature points and the obtained feature description map, perform homologous point matching of two adjacent sonar images through the KNN algorithm.

2. The side-scan sonar image feature extraction method according to claim 1, wherein In step S1, in the dual-branch network architecture, the two branches share the same set of parameters, and the input contains the image I to be paired. i ,I j As well as the depth and posture information obtained by the structured light algorithm; branch i undertakes the task of differentiable operation to reversely propagate gradients and drive model optimization; Branch j performs non-differentiable operations, and the non-differentiable operations include feature point extraction and generating a supervision target for the first branch.

3. The side-scan sonar image feature extraction method according to claim 1, characterized in that In step S1, the loss function of LF-Net is divided into an image-level loss function and an image patch-level loss function; The image-level loss function is used to optimize the feature point detection network in LF-Net. By taking the score map S j calculated on the image I j transformed to the image I i as the training sample, after the rigid transformation ω process, the score map S j is obtained, and K feature points are extracted from the score map. The score map S j is Gaussian filtered with a Gaussian kernel of 0.5 to obtain a smoothed score map. The expression of the loss function is: L im (S i ,S j ) = |S i - g(ω(S j ))| 2 where S i is the score map of the i-th image, S j is the score map of the j-th image, g is a non-differentiable Gaussian filter, and ω is a rigid transformation; The image block-level loss function is used to optimize the local descriptors generated by the feature description network, including the matching feature point loss L pair , the geometric consistency loss L geom , and the triplet loss L tri ; Matching feature point loss L pair Minimize the distance between the feature vectors of the same key point in two images to ensure that the descriptors remain consistent after transformation; for the K feature points proposed in each iteration of branch i, use the current iteration number to represent the relationship between images I i ,I j Transform the coordinates of the K feature points from the coordinate system of image I i to the coordinate system of image I j through a rigid transformation; after completing the coordinate transformation, according to the scale and direction information obtained from branch j, fuse the transformation results in the new coordinate system, and accurately select the image region from I j to obtain two image patches and Extract features from the image patches respectively, and extract features and k represents the number of pairs of feature point matches, is the feature descriptor of the k-th key point in the i-th image, is the feature descriptor corresponding to k in the j-th image after geometric transformation, and obtain the image patch-level loss function; LF-Net uses the geometric consistency loss L geom , which makes the scales and directions of feature points consistent across different images, ensuring that LF-Net has scale invariance and rotational invariance; L geom consists of two parts: scale loss and direction loss: which makes the directions of the matched key points consistent, and makes the scales of the matched key points consistent; represents the scale of the k-th key point in the i-th image, and represents the direction of the k-th key point in the i-th image; represents the scale of the corresponding key point in the j-th image after geometric transformation, and represents the direction of the corresponding key point in the j-th image after geometric transformation, λ ori and λ scale are hyperparameters that control the influence of direction and scale losses respectively. The loss function L geom is as follows; LF-Net uses a triplet loss function to optimize the feature descriptor. For non-corresponding image pairs, a progressive mining strategy is used as negative samples for training, making the matching positive samples closer and the non-matching negative samples farther apart. By setting a minimum margin C with a hyperparameter, it ensures that the distance between matching points and non-matching points is large enough. represents the feature descriptor of the k-th key point in the i-th image, represents the positive sample of the matching key point after geometric transformation of the j-th image, represents a random negative sample in the j-th image. C represents the Margin hyperparameter in the triplet loss, controlling the separation between positive and negative samples. The triplet loss L tri is expressed mathematically as follows; Among them, k′≠k, C = 1; After three loss functions, the loss function used to train the detection network and the description network is: L det = L im + λ pair L pair + L geom L desc = L tri .

4. The side-scan sonar image feature extraction method according to claim 1, wherein In step S2, the ResNet architecture includes three modules, namely a convolutional module, a residual module, and a fully connected module. Each module embeds multiple groups of 5×5 convolutional kernels; a batch normalization step is implemented after each convolution operation; Leaky-ReLU is used as the activation function, and the output channel number of all convolutional layers is fixed at 16, and the output dimension is consistent with the input; the convolutional module performs preliminary feature extraction, reduces the spatial size of the input data, and improves the calculation efficiency; the residual module solves the gradient vanishing problem in the deep neural network, enables information to flow efficiently between layers, and enables the network to learn residuals more easily through the identity mapping; the fully connected module reduces the number of parameters through global average pooling, improves the generalization ability, and makes the final classification decision.

5. The side-scan sonar image feature extraction method according to claim 1, wherein In step S3, feature point detection includes: LF-Net performs multi-scale detection based on the feature map o; the feature map o is scaled N times, and the scaling range is N = 5, The feature maps after N times of scaling are convolved with a 5×5 convolutional kernel to obtain N score maps h N (1 ≤ n ≤ N, each of the N score maps corresponds to the score maps at N different scales. The Softmax operator with a size of 15×15 is used to perform convolution operations on the N score maps respectively to achieve differentiable non-maximum suppression, and N score maps sharpened by the Softmax operator are obtained LF-Net resizes each to the size of the original image, and the resized score map is Using an operator similar to softmax on score maps of the same scale, all are fused to form the final score map in the scale space; In the formula, ⊙ represents the Hadamard product; after scale fusion, select the K pixels with the highest scores from the finally fused scale-invariant score map S as feature points and apply local softargmax to obtain sub-pixel accuracy.

6. The side-scan sonar image feature extraction method according to claim 5, wherein The Softmax operator is used to normalize the input values into a probability distribution so that the sum of the output values is 1; The function definition of the Softmax operator is as follows, where softmax(z i ) represents the value of the i-th element after the Softmax transformation, representing a certain probability value in the probability distribution; wherein, performs an exponential transformation on the input z i to ensure non - negative values and amplify the influence of larger z i values; is the sum of the exponential transformations of all input elements, used for normalization so that the sum of all outputs is 1, forming a probability distribution.

7. The side-scan sonar image feature extraction method according to claim 1, characterized in that In step S4, feature description includes: By applying a 5×5 convolution to each pixel on the feature map o, each pixel outputs two values as sine and cosine, and calculate the direction map θ of each feature point through the arctan function; Descriptor extraction: By analyzing the score map S, K feature points with the highest scores are selected, and the coordinate data is obtained; the key points are combined with the scale and orientation information from the scale estimation map and the orientation estimation map to form a quadruple structure p k =(x, y, s, θ) k , and perform feature extraction operations on the corresponding image regions.

8. The side-scan sonar image feature extraction method according to claim 7, characterized in that, Calculate the direction map θ of each feature point through the arctan function. The direction calculated by arctan is a continuously changing angle value, without direction jumps, providing smooth direction estimation, applicable to various scales, rotations, and perspective changes, improving the robustness of feature points. And LF-Net needs to find the same key points in images with different perspectives and rotational changes. The arctan calculation of the direction angle can ensure that the feature points still point in the same direction after rotation transformation, improving rotational invariance. The calculation formula is as follows: where θ is the direction angle of the key point, and f x is the gradient of the feature point in the x direction, and f y is the gradient of the feature point in the y direction.

9. The side-scan sonar image feature extraction method according to claim 1, wherein In step S5, the KNN homologous point matching includes: for the detected feature points and the obtained feature description maps, perform homologous point matching of two adjacent sonar images through the KNN algorithm; Before matching, process the feature descriptors output by LF-Net in the way of L2 normalization; For two side-scan sonar images to be matched, use the KNN algorithm to match feature points and calculate the Euclidean distance for feature point matching. The KNN algorithm is used for image registration, target detection, and clustering tasks. In side-scan sonar image matching, this method accurately calculates the similarity of feature points in adjacent images, improving the accuracy of stitching and classification. Extract K feature points and the corresponding descriptors D_A and D_B respectively; use the descriptor set D_B as the search database and the descriptor set D_A as the query set; for each descriptor obtained from any sonar image, find K nearest neighbor descriptors among each descriptor obtained from the other sonar image; Wherein, is the coordinate of feature point A i on the X-axis, is the coordinate of feature point B j on the X-axis; is the coordinate of feature point A i on the Y-axis, is the coordinate of feature point B j on the Y-axis; The nearest neighbor is determined by calculating the Euclidean distance between descriptors, and each query descriptor D_A i is compared with all descriptors D_B j in the database for similarity. The two points with the highest similarity are selected as the homonymous points for matching.

10. A side-scan sonar image feature extraction system, characterized in that, This system is used to regulate the side-scan sonar image feature extraction method described in any one of claims 1-9. This system includes: The LF-Net training module is used to construct a network architecture based on a dual-branch and train the detection network using a dual-modal loss; The feature map generation module is used for LF-Net to input the side-scan sonar image into the trained multi-scale fully convolutional network using the ResNet architecture and output an image o with feature points; The feature point detection module is used to detect feature points using LF-Net, and the detection network is in a serial structure; The feature description module is used to calculate the direction map of each feature point through the arctan function; construct a quadruple and perform feature extraction operations on the image region to obtain a feature description map; The KNN homologous point matching module is used to perform homologous point matching of two adjacent sonar images through the KNN algorithm for the detected feature points and the obtained feature description maps.

Images