Side scan sonar target recognition method based on hybrid feature fusion and ensemble learning

Abstract

The application particularly relates to a side scan sonar target recognition method based on mixed feature fusion and ensemble learning, a target region of interest is extracted through an optimized YOLOv7 model, a mixed feature set is constructed by combining deep learning features and artificial features, feature dimension reduction and fusion are carried out by using principal component analysis and variance analysis, finally, target classification is carried out through an ensemble classifier, and the classification accuracy is further improved through decision-level fusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a side scan sonar target recognition method based on mixed feature fusion and ensemble learning. BACKGROUND

[0002] In the related art, underwater automatic target recognition technology is widely used in underwater equipment detection, marine biology monitoring, underwater environment exploration and other civil and military fields. The sensing devices used mainly include optical and acoustic devices. Although optical devices can provide high-definition images, they are limited in scope due to water quality conditions; in contrast, acoustic imaging devices are more suitable for target recognition in complex underwater environments due to their long-range detection capabilities and stability in turbid water. Side scan sonar, as a cost-effective acoustic imaging device, is widely used in underwater detection tasks because it can generate large-scale images. However, the complex and variable underwater environment can cause the image features to be blurred, thereby increasing the difficulty of target recognition. Therefore, how to achieve high-precision automatic target recognition of side scan sonar in complex environments is still a key challenge.

[0003] Automatic target recognition (ATR) technology analyzes side scan sonar images through automated algorithms to achieve accurate identification and classification of potential targets, thereby significantly improving the efficiency and accuracy of underwater tasks. However, traditional machine learning-based ATR systems limit the system's ability to capture complex patterns, especially in scenes with high background complexity. With the rapid development of deep learning technology, more and more researchers have begun to use deep neural networks to try to solve the high complexity problem in sonar image target ATR, bringing new breakthroughs to this field. In addition, deep learning-based ATR systems, deep learning models usually rely on large-scale labeled data sets, and overfitting problems are prone to occur on small sample data sets. For side scan sonar image data sets, this problem is particularly prominent due to the scarcity of data, which severely limits the training effect of deep learning models, thereby causing a decrease in recognition accuracy.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The present application provides a side scan sonar target recognition method based on mixed feature fusion and ensemble learning, a computer program product, a readable storage medium, and an electronic device, which improves the recognition accuracy of side scan sonar targets and can overcome the defects in the prior art to some extent.

[0006] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the application.

[0007] According to a first aspect of the present application, there is provided a side scan sonar target recognition method based on mixed feature fusion and ensemble learning, comprising:

[0008] Obtaining a side scan sonar image to be processed, and performing region of interest extraction on the side scan sonar image to obtain a candidate region image;

[0009] Performing feature extraction on the candidate region image by using a convolution feature extraction model based on a ResNet-50 network to obtain image deep learning features;

[0010] Extracting pixel features, texture features and target shape features from the candidate region image, and performing feature fusion processing to obtain image attribute features;

[0011] Performing dimension reduction processing on the image deep learning features and the image attribute features respectively, and performing feature fusion processing on the dimension-reduced feature data to obtain a mixed feature vector; wherein the dimension-reduced image attribute features retain the original dimensions of the pixel features and the texture features;

[0012] Inputting the mixed feature vector into an ensemble classifier to enable a first sub-classifier and a second sub-classifier of the ensemble classifier to perform classification processing on the mixed feature vector respectively, and obtaining first prediction data and second prediction data; wherein the first sub-classifier is constructed based on a support vector machine, and the second sub-classifier is constructed based on a random forest;

[0013] Performing weighted fusion processing on the first prediction data and the second prediction data to obtain a fusion classification result; wherein the weight coefficients of the weighted fusion processing are determined by a cross-validation method.

[0014] According to a second aspect of the present application, there is provided a computer program product having a computer program stored thereon, wherein the computer program is executed by a processor to implement the above-mentioned side scan sonar target recognition method based on mixed feature fusion and ensemble learning.

[0015] According to a third aspect of the present application, there is provided a storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the above-mentioned side scan sonar target recognition method based on mixed feature fusion and ensemble learning.

[0016] According to a fourth aspect of the present application, there is provided an electronic device comprising:

[0017] A processor, a memory; wherein the memory is used to store executable instructions of the processor; the processor is configured to implement the above-mentioned side-scan sonar target recognition method based on hybrid feature fusion and ensemble learning by executing the executable instructions.

[0018] The side-scan sonar target recognition method based on hybrid feature fusion and ensemble learning provided by the embodiment of the application extracts the target region of interest from the side-scan sonar image by using the YOLOv7 model optimized by transfer learning, effectively filters background noise; at the same time, the hybrid feature set is constructed by combining the image deep learning features and the image attribute features to capture the multi-dimensional information of the target; then, the feature dimension reduction processing is performed, and the unified low-dimensional representation is formed through the feature fusion, so as to enhance the distinguishability of the features; finally, the classifier based on the ensemble learning is used to classify the target, and the accuracy and robustness of the classification are further improved through the decision-level fusion. The application fully integrates the advantages of deep learning and traditional machine learning, and provides an effective and suitable automatic target recognition solution for side-scan sonar images.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0020] The drawings incorporated into the specification and forming a part of the specification, show embodiments consistent with the application, and together with the specification, serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained from these drawings without creative labor for those skilled in the art.

[0021] Figure 1 The schematic diagram of the side-scan sonar target recognition method based on hybrid feature fusion and ensemble learning according to an exemplary embodiment of the application is shown schematically;

[0022] Figure 2 The schematic diagram of the flow of the side-scan sonar target recognition method based on hybrid feature fusion and ensemble learning according to an exemplary embodiment of the application is shown schematically;

[0023] Figure 3 The schematic diagram of the actual driving path of the unmanned surface vehicle during data collection according to an exemplary embodiment of the application is shown schematically;

[0024] Figure 4 The convergence curve diagram of the training loss of the ROI extraction model with respect to the number of iterations and the convergence curve of the training loss of the CNN feature extraction model in the classification stage with respect to the number of iterations according to an exemplary embodiment of the application are shown schematically;

[0025] Figure 5 A schematic diagram illustrating an evaluation result of an exemplary embodiment of the present application on a test set;

[0026] Figure 6 A schematic diagram illustrating a recognition result of an exemplary embodiment of the present application on a test set;

[0027] Figure 7 A schematic diagram illustrating a comparison of detection results of an ATR method and an advanced target detection and recognition algorithm of an exemplary embodiment of the present application on different target types;

[0028] Figure 8 A schematic diagram illustrating a comparison of ROI extraction results of an ROI extraction algorithm and an existing advanced ROI extraction algorithm of an exemplary embodiment of the present application on large targets and on small and medium-sized targets;

[0029] Figure 9 A schematic diagram illustrating an electronic device of an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0030] Example implementations are now described with reference to the drawings; however, these implementations are merely examples of implementations and are not intended to limit example implementations in any way. In fact, example implementations described herein can be used in any number of ways, and should not be limited to implementations set forth herein; rather, implementations can be practiced in any number of objects.

[0031] Furthermore, the drawings are not necessarily drawn to scale. Like reference numerals can be used to denote like parts throughout the various illustrations. The same or similar reference numerals can be used in different drawings to denote the same or similar parts. Descriptions of features, structures, or characteristics can be presented in terms of functional descriptions and / or structural descriptions. Illustrative examples of the present application are described in terms of one or more exemplary implementations. Description of features, structures, or characteristics in the singular can encompass the plural and vice versa, unless otherwise noted. Spatially relative terms, such as "above," "below," "left," "right," "in," "under," "above," "on," "below," "up," "down," "downward," "upward," "top," "bottom," "laterally," "vertical," "horizontal," and the like, can be used herein for ease of description to describe one element's or feature's relationship to another element(s) or feature(s) as illustrated in the figures. Spatially relative terms can be intended to encompass different orientations of the device in use or

[0032] In the related art, a typical ATR system consists of three main steps: region of interest extraction, feature extraction, and classification recognition. First, the region of interest where the target is located is extracted from the sonar image; then, the features of these regions are extracted; finally, the extracted features are classified and recognized by a classifier. Currently, ATR systems are mainly divided into two categories: traditional machine learning-based methods and deep learning-based algorithms, each of which has different advantages and limitations. The ATR system based on traditional machine learning usually relies on manually designed region of interest extraction algorithm, feature extractor and classifier to build the target detection and recognition process. This method of combining traditional image processing techniques with machine learning has improved the accuracy of sonar target recognition to some extent and can effectively train in the case of limited data. Although the ATR system based on traditional machine learning has certain advantages, such as lower demand for labeled data, small training sample size, and certain interpretability of feature design, it highly depends on manually designed features. This dependence limits the system's ability to capture complex patterns, especially in scenes with high background complexity. With the rapid development of deep learning technology, more and more researchers have begun to use deep neural networks to try to solve the high complexity problem in sonar image target ATR, bringing new breakthroughs to this field. In recent years, the target detection network based on deep learning has provided an end-to-end solution for ATR, which can complete the three key processes of region of interest extraction, feature extraction, and target classification through a single model, and realize joint optimization training. The ATR system based on deep learning has a significant advantage in the expression ability of target detection, especially in capturing the complex patterns, scales, and textures of objects. However, deep learning models usually rely on large-scale labeled data sets, and overfitting problems often occur on small sample data sets. For side-scan sonar image data sets, this problem is particularly prominent due to the scarcity of data, which severely limits the training effect of deep learning models and leads to a decrease in recognition accuracy.

[0033] In view of the shortcomings and deficiencies of the prior art, a side-scan sonar target recognition method based on hybrid feature fusion and ensemble learning is provided in the example embodiment. Referring to Figure 1 The method can include the following steps:

[0034] Step S11, acquiring a side-scan sonar image to be processed, and performing region of interest extraction on the side-scan sonar image to obtain a candidate region image;

[0035] Step S12, using a convolution feature extraction model based on a ResNet-50 network to extract features from the candidate region image to obtain image deep learning features;

[0036] Step S13, pixel features, texture features and target shape features are extracted from the candidate region image, and feature fusion processing is performed to obtain image attribute features;

[0037] Step S14, dimension reduction processing is performed on the image deep learning features and the image attribute features respectively, and feature fusion processing is performed on the dimension-reduced feature data to obtain a mixed feature vector; wherein the image attribute features after dimension reduction processing retain the original dimensions of the pixel features and the texture features;

[0038] Step S15, the mixed feature vector is input into the integrated classifier, so that the first sub-classifier and the second sub-classifier of the integrated classifier respectively perform classification processing on the mixed feature vector, and obtain first prediction data and second prediction data; wherein the first sub-classifier is constructed based on a support vector machine, and the second sub-classifier is constructed based on a random forest;

[0039] Step S16, the first prediction data and the second prediction data are subjected to weighted fusion processing to obtain a fusion classification result; wherein the weight coefficients of the weighted fusion processing are determined by cross-validation.

[0040] In the following, the various steps of the side-scan sonar target recognition method based on mixed feature fusion and ensemble learning in the example embodiment will be described in more detail in conjunction with the accompanying drawings and examples.

[0041] In the example embodiment, the side-scan sonar target recognition method based on mixed feature fusion and ensemble learning can be executed by an intelligent terminal device; or it can also be executed in cooperation between the intelligent terminal device and the server end. Wherein the trained infrared image generation model can be pre-arranged on the intelligent terminal device end or the server end. The user can select the current side-scan sonar image to be processed on the intelligent terminal device end, create a corresponding recognition task, and execute the task locally on the terminal device and output the recognition result data; or the recognition task can be sent to the server end, and the server end executes the side-scan sonar target recognition method based on mixed feature fusion and ensemble learning to generate the recognition result data and feedback to the terminal device.

[0042] In step S11, the side-scan sonar image to be processed is obtained, and the region of interest extraction is performed on the side-scan sonar image to obtain the candidate region image.

[0043] For example, the above step S11 can include:

[0044] Step S21, the region of interest extraction network based on YOLOv7 network is used to perform region of interest extraction on the side-scan sonar image, obtain a candidate region containing a target of interest, and determine the position information of the corresponding clipping frame;

[0045] In step S22, the clipping frame is expanded outward based on a preset pixel size to obtain a size-expanded clipping frame.

[0046] In step S23, the candidate region image is obtained by clipping based on the size-expanded clipping frame.

[0047] Specifically, in the ATR (Automatic Target Recognition) framework, the target region of interest needs to be extracted from the sonar image first. This process is the basis for subsequent target recognition and has high requirements for timeliness. The YOLO series network is an ideal choice for real-time detection of underwater sonar targets due to its advantages in real-time performance, global information integration, simple structure, and multi-scale feature capture. In particular, YOLOv7 improves the Anchor mechanism and introduces an adaptive Anchor generation strategy, enabling the network to better adapt to targets of different scales and shapes, thereby improving the ROI extraction accuracy. In addition, the multi-scale prediction capability of YOLOv7 further enhances the detection performance of targets of different sizes. Therefore, YOLOv7 is selected as the core network framework for ROI (region of interest) extraction in this step.

[0048] Specifically, after obtaining the actual position of the target region of interest, the coordinates of the target region of interest in the image are used to crop the image. To reduce the loss of edge information, the size of the cropping frame is expanded outward by 20 pixels, thereby obtaining new diagonal coordinates (i.e., the top-left corner and bottom-right corner coordinates). Finally, the cropped image is resized according to the input requirements of the recognition network, and the processed image of the region of interest is input into the feature extraction network for further target recognition.

[0049] Illustratively, the method further comprises pre-training a region of interest extraction network based on the YOLOv7 network, including:

[0050] In step S31, the YOLOv7 model pre-trained based on the COCO dataset is used as the base model.

[0051] In step S32, the first two layers of the backbone feature extraction network and the feature fusion network of the base model are frozen.

[0052] In step S33, the SGD optimizer is used to optimize and train the model hyperparameters of the last two layers of the backbone feature extraction network and the feature fusion network and the detection head part to obtain the region of interest extraction network. The initial learning rate of the SGD optimizer is set to 0.01, and the cosine annealing strategy is used for dynamic adjustment.

[0053] ​Specifically, since YOLOv7 lacks a pre-trained model on underwater sonar data, and the network needs a large number of samples to achieve good detection effect, the method of transfer learning is adopted for optimization. Specifically, the YOLOv7 model pre-trained based on the COCO dataset is used, and the parameters thereof are used as the initial parameters of the region of interest extraction network. During the training process, the first two layers of parameters of the backbone feature extraction network and the feature fusion network in the YOLOv7 network are frozen, and only the last two layers of the feature fusion network and the detection head part are fine-tuned. In addition, the target class parameter is modified to 1, so that the model focuses on the ROI extraction task. After training, the model will be saved as a.pt file, and real-time inference will be performed in the actual ATR process to extract the target region of interest in the sonar image.

[0054] In step S12, a convolution feature extraction model based on a ResNet-50 network is used to extract features from the candidate region image to obtain image deep learning features.

[0055] For example, the convolution feature extraction model based on the ResNet-50 network uses the ResNet-50 network as the backbone network. The method further comprises pre-training the convolution feature extraction model, comprising:

[0056] In step S41, the images in the CIFAR-10 dataset are subjected to grayscale processing, and the ResNet-50 network is preliminarily trained using the grayscale processed images to obtain initial weight data of the convolution feature extraction model.

[0057] In step S42, the ResNet-50 model is fine-tuned and trained using the side-scan sonar image dataset, comprising: freezing the first three residual stages of the ResNet-50 model; and optimizing and training the model parameters of the last three residual stages of the ResNet-50 model and the fully connected layer.

[0058] In step S43, after the training is completed, the fully connected layer is removed, the feature extraction part is retained, and it is packaged as a convolution feature extraction model.

[0059] Specifically, for the extraction of CNN features, ResNet-50 is selected as the backbone network for feature extraction. First, in order to adapt to the grayscale form of the sonar image, the images in the CIFAR-10 dataset are subjected to grayscale processing, and the ResNet-50 model is preliminarily trained using these grayscale images to obtain the initial weights of the model. Subsequently, the trained weights are used as the initial parameters for sonar image feature extraction. Then, in order to adapt to the sonar image data, the ResNet-50 is fine-tuned and trained using the collected side-scan sonar image dataset.

[0060] In the fine-tuning stage, the first three stages (Stage 1-3) of ResNet-50, i.e., conv1 and the first three residual blocks (ResBlock1-3), are frozen, and only the parameters of the last three stages (Stage 4-5), i.e., ResBlock4 and ResBlock5, and the fully connected layer are optimized. Specifically, ResNet-50 is composed of an initial convolutional layer (conv1), a BatchNormalization, and a max-pooling layer, followed by four residual blocks (ResBlock1-4) composed of multiple residual units, and finally global pooling by avgpool and classification by connecting the fully connected layer. In order to enhance the model's feature extraction ability for sonar images, only the high-level feature extraction part (Stage 4-5, i.e., ResBlock4 and ResBlock5) and the classification layer are trained, while the low-level features are frozen to retain their generalization characteristics on natural image data.

[0061] Alternatively, the ResNet-50 network is composed of 6 modules, including 1 input module, 4 residual modules, and an output module composed of a global average pooling layer and a fully connected layer. In the fine-tuning process, the parameters of the first 3 modules are fixed (i.e., the parameters pre-trained on the optical data set are retained), and only the parameters of the last 3 modules are trained and updated to adapt to the feature expression requirements of sonar images. Achieve more efficient feature learning.

[0062] After training is complete, the fully connected layer of ResNet-50 is removed, the feature extraction part is retained, and it is encapsulated as a dedicated convolutional feature extractor. Assuming that the input image is , the feature map is obtained through a series of convolution operations and residual connections. At each layer, the convolution operation extracts local and high-level semantic features through local perception, weight sharing, etc. The CNN feature can be represented as:

[0063]

[0064] where, is the input image, is the number of channels of the convolutional layer, is the weight parameter of each convolution kernel, is the feature map after the convolution operation on the th channel of the input image , is the bias term of the convolutional layer, represents the deep features extracted by the convolution operation, is the activation function.

[0065] In practical applications, the method inputs the region of interest output by the network into a convolution feature extractor to extract deep learning features of the side scan sonar image target. To enhance the normativeness and adaptability of the features, the extracted features are normalized and then input into subsequent modules.

[0066] In step S13, pixel features, texture features, and target shape features are extracted from the candidate region image, and feature fusion processing is performed to obtain image attribute features.

[0067] By way of example, step S13 described above can include:

[0068] In step S51, statistical moments corresponding to the candidate region image are extracted, and pixel features are calculated according to the statistical moments; wherein the statistical moments include a gray distribution histogram, a first-order moment, a second-order moment, and a third-order moment.

[0069] In step S52, a gray level co-occurrence matrix (GLCM) is constructed for the candidate region image, and statistical quantities are calculated to obtain the texture features; wherein the statistical quantities include energy, entropy, contrast, and correlation.

[0070] In step S53, gradient intensity and direction corresponding to the candidate region image are calculated; the candidate region image is divided into sub-regions, and gradient direction histograms corresponding to the sub-regions are determined based on the gradient intensity and direction of the candidate region image; and normalization processing is performed based on the gradient direction histograms to obtain the target shape features.

[0071] By way of example, the feature fusion processing to obtain the image attribute features from the extracted pixel features, texture features, and target shape features includes vertically splicing the pixel features, texture features, and target shape features, and performing normalization processing to obtain the image attribute features.

[0072] Specifically, this part aims to the image attribute features of the sonar target, extracts pixel features, texture features, and shape features from the global attributes and local characteristics of the image, to construct a more comprehensive feature expression. First, the global pixel information is obtained by extracting the gray distribution histogram and statistical moments of the sonar image, the intensity variation pattern of the target region is captured, and the target and the background are distinguished. Second, the local texture pattern is described by using the gray level co-occurrence matrix (GLCM) and its statistical quantities, including energy, entropy, contrast, and correlation, to supplement the deficiency of deep features in low resolution or high noise environment. Then, the geometric contour features of the target are extracted by using the histogram of oriented gradient (HOG) algorithm to enhance the target recognition ability. Finally, the pixel, texture, and shape features are vertically spliced, and the adaptability and stability of the features are improved through normalization processing.

[0073] Specifically, pixel features, as global features, reflect the pixel distribution and its statistical properties of the target region. The proposed method extracts the gray-level distribution histogram, first moment, second moment, third moment, and other statistical moments of the side-scan sonar image to obtain the global pixel information of the target. In the sonar image, the distribution of echo intensity is closely related to the material and shape of the target. These features help to capture the intensity variation pattern of the target region, thereby helping to distinguish the significant difference between the target and the background. The pixel features are expressed as follows:

[0074]

[0075]

[0076] wherein, is the pixel feature vector, is the pixel feature extraction function, is the pixel histogram feature extraction function, is the gray level, is the total number of pixels, is the gray value of the image at position , is the indicator function, which is 1 when , otherwise 0. is the first moment, is the second moment, is the third moment.

[0077] Specifically, texture features are the representation of the surface structure pattern of the image, which can describe the relationship and distribution of local pixels in the image. In the low resolution or high noise environment of the sonar image, the light and dark changes of the target surface texture can provide additional discriminative basis, thereby making up for the deficiency of depth features in capturing fine-grained texture. This step models the texture characteristics of the sonar target by constructing GLCM (Gray-Level Co-occurrence Matrix) and calculating its statistics, including energy, entropy, contrast, and correlation. The texture features are expressed as follows:

[0078]

[0079]

[0080] wherein, is the texture feature vector, is the texture feature extraction function, is the energy of the GLCM-based statistics, is the entropy, is the contrast, For correlation, is an image The GLCM corresponds to the pixel position and The element value of and is the row and column mean of GLCM, and is the standard deviation.

[0081] Specifically, shape features represent local geometric properties and can reflect the geometric outline and edge information of the target. In sonar images, the contrast between the target and the background is usually presented in the form of contour features, so shape features become an important clue for target recognition.

[0082] This method uses the HOG algorithm to extract the shape features of a target. Specifically, the image is divided into small cells by calculating the gradient strength and direction, and a gradient direction histogram is constructed for each cell. Block normalization is then performed to enhance the robustness of the features. Ultimately, the gradient histograms of all blocks are concatenated to form the HOG feature vector of the target. This method can explicitly model the local contour characteristics of the target and exhibits strong discriminative power in sonar images with significant shape information. The shape feature is expressed as follows:

[0083]

[0084] in, is the shape feature vector, is the shape feature extraction function, is the gradient direction histogram in each small unit, Indicates that the image is divided into a grid structure of multiple small units.

[0085] Specifically, to construct a more comprehensive artificial feature representation, namely, image attribute features, this step vertically concatenates pixel features, texture features, and shape features to form an artificial feature set for the sonar target. Subsequently, normalization is performed to unify the feature's numerical range, improving its adaptability and stability in subsequent model processing. The fused image attribute features are expressed as follows:

[0086]

[0087] in, For normalization, For vertical splicing.

[0088] In step S14, the image deep learning features and image attribute features are respectively subjected to dimensionality reduction processing, and the feature data after dimensionality reduction are subjected to feature fusion processing to obtain a mixed feature vector; wherein, the image attribute features after dimensionality reduction retain the original dimensions of pixel features and texture features.

[0089] Exemplarily, the step S14 can include:

[0090] In step S61, the image deep learning features are processed by dimension reduction using a principal component analysis (PCA) method.

[0091] In step S62, the target shape features are processed by dimension reduction using an analysis of variance (ANOVA) method.

[0092] In step S63, the pixel features and the texture features are kept in the original dimension, and the image deep learning features and the target shape features processed by dimension reduction are spliced based on the vertical direction to obtain a mixed feature vector after feature fusion.

[0093] Specifically, two types of features are extracted in the feature extraction part: implicit deep features based on CNN (i.e., image deep learning) and explicit features based on a convolution feature extractor (i.e., image attribute features). Since the dimensions of these two types of features are high, directly using them for classification will result in excessive computation, thereby affecting the performance of the sonar ATR system. Therefore, before target classification, this step uses feature dimension reduction and feature fusion to reduce the feature space and improve the classification efficiency and the robustness of the model.

[0094] Exemplarily, the image deep learning features are processed by dimension reduction using a principal component analysis (PCA) method, including:

[0095] In step S71, a covariance matrix is calculated using the CNN feature vectors extracted from the public side-scan sonar dataset.

[0096] In step S72, eigenvalues and their corresponding eigenvectors are obtained by eigenvalue decomposition of the covariance matrix.

[0097] In step S73, the principal components to be retained are selected according to the sizes of the eigenvalues.

[0098] In step S74, the selected principal component eigenvectors are combined into a projection matrix.

[0099] In step S75, the image deep learning features are mapped to a low-dimensional space using the projection matrix to realize dimension reduction of the image deep learning features.

[0100] Specifically, for the deep learning features extracted by CNN, this step uses PCA for dimension reduction due to the high dimension. PCA can effectively reduce the dimension of the features while retaining as much of the main information of the data as possible.

[0101] (1) PCA training process: Before performing PCA (principal components analysis) dimensionality reduction, first use the CNN features extracted from the public side-scan sonar dataset for PCA training. This process includes the following steps: 1) Calculate the covariance matrix: Based on the standardized CNN feature vectors, calculate the covariance matrix to capture the correlation between the features in the data.

[0102] 2) Eigenvalue decomposition: Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues and their corresponding eigenvectors.

[0103] 3) Select principal components: Select the retained principal components according to the size of the eigenvalues. Larger eigenvalues represent the direction with the largest variance in the data, so these eigenvectors are selected as principal components.

[0104] 4) Construct the projection matrix: Combine the selected principal component eigenvectors into the projection matrix.

[0105] (2) PCA dimensionality reduction application:

[0106] In the test phase, after extracting the CNN features of the ROI region from the side-scan sonar images collected from the experiment, we use the PCA projection matrix obtained in the training phase to map the CNN features in the test set to a low-dimensional space. Specifically, the standardized test CNN features are projected through the PCA projection matrix obtained in the training phase to obtain the dimensionality-reduced feature representation as follows:

[0107]

[0108] where, is the matrix of standardized test CNN features (matrix dimension , where is the number of test samples, is the original feature dimension), is the PCA projection matrix (matrix dimension , where is the reduced feature dimension). is the result matrix of the dimensionality-reduced test features (matrix dimension ).

[0109] Specifically, for the explicit features extracted by the feature extractor, this step uses ANOVA (Analysis of Variance) for feature selection to filter out the features most relevant to the classification task. It should be noted that the first moment, second moment, third moment and texture features (such as energy, entropy, contrast, correlation) in the pixel feature do not participate in dimension reduction, and their original dimensions are retained in order to better retain the semantic information of these features. ANOVA selects the features most informative to the classification task by calculating the difference between each feature and the category.

[0110] (1) ANOVA training process: Before performing ANOVA dimensionality reduction, first use the artificial explicit features extracted from the public side-scan sonar dataset for ANOVA training. The training process includes the following steps:

[0111] 1) Feature grouping: Group each feature according to the target category (different types of sonar targets). Each group contains sample features belonging to the same category.

[0112] 2) Calculate inter-group variance and intra-group variance: For each feature, calculate the inter-group variance (difference between categories) and intra-group variance (difference between samples within the same category). A larger inter-group variance means that the difference between different categories is significant, and a smaller intra-group variance indicates that samples within the same category are more similar.

[0113] 3) Calculate F value: Calculate the F value using the inter-group variance and intra-group variance. The larger the F value, the greater the contribution of the feature to distinguishing different categories.

[0114] 4) Hypothesis testing: Perform hypothesis testing on the F value to determine whether the feature significantly affects the classification of categories. By calculating the p-value of the F value, determine which features have significance to the classification task.

[0115] 5) Select features: According to the set significance threshold, select features with larger F values, which are considered to have the most information for target classification.

[0116] 6) Feature set mask: According to the set number of selected features, form a feature set mask containing the most significant features for subsequent classification tasks.

[0117] (2) ANOVA dimensionality reduction application: After extracting artificial features from the region of interest of the side-scan sonar image collected from the experiment, apply the same grouping method used in the training process to maintain consistency in feature organization. Then, use the feature set mask obtained during training to project the test features into the selected feature space, thereby reducing the dimension while retaining the features most relevant to classification. The formula is:

[0118]

[0119] wherein, is the standardized test artificial feature matrix (matrix dimension is wherein, is the number of test samples, is the original feature dimension), is the mask matrix (matrix dimension is wherein, is the dimension of the reduced feature), is the reduced test feature matrix (matrix dimension is ).

[0120] Specifically, after dimension reduction, the two kinds of features are spliced into a unified feature vector in the vertical direction. This fusion combines the implicit features based on deep learning with the explicit artificial features, enhancing the discriminability and expression ability of the features. The fused feature set is used as the input of the classification model, providing more comprehensive support for target classification. The formula of feature fusion is as follows:

[0121]

[0122] wherein, is the fused feature matrix (matrix size is wherein, is the number of test samples, is the dimension of the fused feature).

[0123] In step S15, the mixed feature vector is input into the integrated classifier, so that the first sub-classifier and the second sub-classifier of the integrated classifier respectively perform classification processing on the mixed feature vector, and first prediction data and second prediction data are obtained; wherein the first sub-classifier is constructed based on support vector machine, and the second sub-classifier is constructed based on random forest.

[0124] Exemplarily, the above step S15 can include:

[0125] Step S81, input the mixed feature vector into the first sub-classifier constructed based on support vector machine, use linear decision boundary for prediction, output class label and corresponding prediction probability;

[0126] Step S82, input the mixed feature vector into the second sub-classifier constructed based on random forest, combine the prediction results of multiple decision trees through Bagging and majority voting strategy to determine the class label and the corresponding prediction probability.

[0127] Specifically, after feature extraction, dimensionality reduction, and feature fusion, the present invention obtains a hybrid feature set that combines CNN deep learning features with artificial features. To fully capture the linear and nonlinear relationships between features and fully leverage the advantages of both in different feature patterns, this step uses an ensemble learning method, combining a linear SVM (Support Vector Machine) and an RF (Random Forest) classifier.

[0128] To ensure the generalization ability of the classifier, we first train the classifier using a hybrid feature set extracted from a side-scan sonar image dataset collected online. The specific steps are as follows:

[0129] (1) SVM classifier: 1) Linear kernel selection: The linear kernel SVM classifier is used, which is suitable for high-dimensional linearly separable data.

[0130] 2) Optimization objective: Determine the optimal hyperplane parameters by solving the following optimization problem:

[0131]

[0132]

[0133] in, is the normal vector of the hyperplane, is the bias, It is a sample The category label, It is The feature vector of the samples.

[0134] 3) Decision boundary determination: The decision boundary is determined by maximizing the margin between the support vector and the hyperplane, which will be used to divide the feature space.

[0135] (2) RF classifier: 1) Decision tree construction: Each decision tree in the random forest is constructed by randomly selecting a data subset and a feature subset. The information gain is calculated using the CART algorithm:

[0136]

[0137] in, It is a dataset The information entropy of It is feature-based The subset obtained by partitioning.

[0138] 2) Random Forest Construction: Random Forest uses Bagging technology to randomly extract multiple subsets from the original dataset through bootstrap sampling and train an independent decision tree on each subset.

[0139] 3) Bagging and Voting: The final prediction is obtained by majority voting strategy, and the class with the most votes is taken as the output result.

[0140] In the test phase, the mixed features after dimensionality reduction are input into the trained classifier for prediction. Specifically, in the single classifier prediction:

[0141] 1) SVM classifier: The test features are input into the trained SVM classifier, and the linear decision boundary is used for prediction, and the class label is output.

[0142] 2) Random Forest classifier: The test features are input into the trained Random Forest classifier, and the prediction results of multiple decision trees are combined by Bagging and majority voting strategy to determine the final class.

[0143] In the SVM classifier, a linear kernel function is selected to handle high-dimensional linearly separable data, ensuring that the model can efficiently construct the optimal hyperplane. By maximizing the interval between support vectors and the hyperplane, the feature space is accurately divided, thereby improving the separation degree between classes. In the RF classifier, each decision tree selects a subset of data and a subset of features at random, and uses the Classification and Regression Tree (CART) algorithm to calculate the information gain to optimize the node splitting strategy. Subsequently, the Bagging technique is used to randomly sample multiple subsets of samples from the original data set and train the decision trees independently. Finally, the majority voting mechanism is used to fuse the classification results, and the class with the highest votes is selected as the final output.

[0144] In step S16, the first prediction data and the second prediction data are weighted and fused to obtain a fused classification result; wherein the weight coefficient of the weighted fusion processing is determined by cross-validation.

[0145] For example, in order to make full use of the prediction information of the two classifiers, a decision-level fusion strategy is adopted: the prediction probabilities of the two classifiers are calculated, and the results are fused using a weighted average method:

[0146]

[0147] wherein, is the weight coefficient determined by cross-validation, which is set to 1 / 2 to ensure that the SVM and RF classifiers have equal contribution in the final decision. and represent the prediction probabilities of the SVM and RF classifiers, respectively.

[0148] Finally, the class with the highest prediction probability after fusion can be selected as the final classification result.

[0149] Exemplarily, the method described above can further include:

[0150] Step S91, data summary and experimental setting, evaluate the performance of the method through experimental results, and compare and analyze with the current advanced target recognition algorithm to verify the superiority and effectiveness of the method.

[0151] Step S92, ablation experiment setting, verify the effectiveness of the region of interest extraction algorithm, the region of interest classification algorithm, feature fusion, feature dimension reduction and ensemble learning.

[0152] Exemplarily, the effectiveness of the method is described below through experimental data.

[0153] (1) Data collection: In this experiment, sonar data of four types of targets: ships, airplanes, human bodies and cylinders were collected. The data of ships, airplanes, human bodies and part of the cylinder targets were obtained from the IEEE Dataport public side-scan sonar image dataset, and additional cylinder data were collected by the laboratory. During the data collection process, an unmanned surface vehicle equipped with a Tritech Seaking ROV side-scan sonar was used. The sonar device has a range of 40 m, a frequency of 325 kHz, a vertical beam angle of 30° and a horizontal beam angle of 1°, and the experimental site is located in Qishuang Lake (100 m × 200 m, average water depth 1.5 m). Four cylinders were placed on the lake bottom, with lengths between 780 mm and 1200 mm and diameters between 300 mm and 650 mm.

[0154] Reference Figure 3 shows that during the experiment, the unmanned surface vehicle travels at a speed of 4 knots along the pre-planned parallel mowing path. Starting from the designated starting point, the unmanned surface vehicle collects data along the planned route and returns along the same path. By collecting data in different directions and trajectories multiple times, a total of 221 side-scan sonar images of cylinder targets were collected.

[0155] (2) Data description: The dataset used in this experiment contains 916 side-scan sonar images, of which 695 are from IEEE Dataport and 221 are collected by the laboratory. The dataset contains 128 human body target images, 393 cylinder target images, 123 airplane target images and 272 ship target images.

[0156] Reference Table 1 shows the distribution of each type of target in different datasets. In order to train and evaluate the model, the dataset is randomly divided into training set, validation set and test set in the ratio of 7:1:2, containing 641, 92 and 183 images respectively. All images are labeled in VOC format, including the location and category of each target.

[0157] Table 1

[0158]

[0159] (3) Experimental environment configuration: The method proposed in this patent is trained and tested on the Ubuntu operating system, using the Python programming language, and is implemented based on the PyTorch framework. The hardware configuration includes an Intel® Xeon® Silver 4110 CPU (2.10 GHz), 64 GB of memory, and an NVIDIA GeForce RTX 3080 GPU that supports CUDA 12.4 acceleration.

[0160] (4) Model parameter settings: 1) ROI extraction stage:

[0161] In the ROI extraction stage, the YOLOv7 network model is used, and the specific parameter settings are as follows: the number of anchors in the network is set to 9, and the width and height are set to [12, 16], [19, 36], [40, 28], [36, 75], [76, 55], [72, 146], [142, 110], [192, 243], and [459, 401] rectangular boxes. The input image size of the model is 640x640 pixels, and the number of classes is set to 1. The initial weights of the YOLOv7 model come from the pre-trained weights on the COCO dataset, which are further optimized through transfer learning to adapt to the current task.

[0162] 2) ROI classification stage: This part involves parameter configuration in four aspects: feature extraction, feature fusion, feature dimension reduction, and classification recognition.

[0163] a. Feature extraction stage: including ResNet-50 network under transfer learning, color feature extraction, shape feature extraction, and texture feature extraction. For the ResNet-50 network, the number of classification categories is set to 4, i.e., modifying the output dimension of the fully connected layer to 4. The loss function uses the cross-entropy loss function, and the initial weights come from the pre-trained weights trained on the COCO dataset. GLCM is used for texture feature extraction, and the calculation direction is set to [0°, 45°, 90°, 135°]. HOG features are used for shape feature extraction, with the parameter setting that the gradient direction is divided into 12 intervals, and L1 norm regularization is used for block normalization. The image is divided into multiple cells, each with a size of 8x8 pixels; multiple cells are combined into blocks, each containing 4x4 cells.

[0164] b. Feature dimension reduction stage: the deep learning features extracted by ResNet-50 are reduced to 600 dimensions, and the explicitly designed features are reduced to 1000 dimensions, to reduce the computational burden and optimize the model performance.

[0165] c. Classification identification part: for SVM, the kernel function is selected as linear kernel, and the number of classification label categories is 4. The parameter setting of RF classifier is that the number of trees in the forest is 100, and the number of classification label categories is also 4.

[0166] (5) Training parameter setting: 1) ROI extraction stage: in the ROI extraction stage, YOLOv7 is used as the target detection model. The target category is set to 1, the pre-training weight uses the weight trained on the COCO dataset, the download path is yolov7.pt, and the network structure configuration file is yolov7.yaml. The dataset path is the sonar.yaml file of the created sonar dataset. In the training process, the hyperparameters are set as follows: the confidence threshold is set to 0.25, the NMS IOU threshold is set to 0.45, the optimizer is selected as SGD, the total number of training rounds is set to 300, the batch size is set to 32, the learning rate is adopted. The strategy is cyclic, the initial learning rate is 0.01, the minimum learning rate is 0.0001, the learning rate momentum is 0.937, the learning rate decay method uses cosine annealing, the weight decay coefficient is set to 0.0005, the preheating learning rate is set to 3.0, the IoU loss weight is set to 0.05, and the anchor ratio is set to 4.0. In the training process, the freeze training strategy is adopted. The training batch number in the freeze stage is 5, and the training batch number in the unfreeze stage is 295.

[0167] 2) ROI classification stage: in the ROI classification stage, ResNet-50 is used for CNN feature extraction, and SVM and RF are used for classification. The CNN feature extraction model training parameter setting of ResNet-50 is as follows: the target category is set to 4, the batch size is set to 4, the pre-training weight uses the weight trained on the ImageNet dataset, the optimizer is selected as SGD, the initial learning rate is set to 0.01, the momentum parameter is 0.9, the weight decay is set to 0.0005, the cosine annealing strategy is adopted, the maximum learning rate is 0.01, the final learning rate is 0.0005, the training round is set to 200, and the input image size is set to [200, 200]. Similarly, the freeze training strategy is adopted, the training batch number in the freeze stage is 50, and the training batch number in the unfreeze stage is 150.

[0168] For the classifier, when training the SVM classifier, the regularization parameter is set to 1, and the kernel function coefficient is set to "scale". When training the RF classifier, the random number seed is set to 40 to ensure the reproducibility of the results.

[0169] (6) Model training: reference Figure 4 The training curves of the ROI extraction model (left) and the CNN feature extraction model (right) are shown. Figure 4The left graph of FIG. 1 shows the convergence curve of the training loss value of the ROI extraction model with the number of iterations. As can be seen from the graph, the introduction of transfer learning and the freezing training strategy significantly accelerates the convergence speed of the model. After about 10 iterations, the training loss value quickly approaches 0, which indicates that when the YOLOv7 model is simplified to be used only for ROI extraction, the learning workload of the model on data features is greatly reduced, so it can achieve better extraction effect by learning the features of a shallower level.

[0170] Figure 4 The right graph of FIG. 1 shows the convergence curve of the training loss value of the CNN feature extraction model in the classification and recognition stage with the number of iterations. In the first 50 epochs, the freezing training strategy is adopted, and part of the weights are fixed. In this stage, the training loss value quickly decreases and converges quickly, but the value is large, indicating that the freezing training limits the learning of new features by the model. After 50 epochs, the model parameters are unfreezed, and all parameters start to participate in the training. At this time, the training loss value decreases slowly, and the training process becomes slow, but the loss value gradually stabilizes at about 0.2, indicating that after unfreezing, the model can learn more features, although the convergence speed slows down, but the final loss value is significantly reduced.

[0171] By comparing the two training curves, it can be seen that separate training of the ROI extraction stage and the classification and recognition stage can set different training strategies according to different learning intensities, so that the network model can achieve better training effect in the two stages respectively.

[0172] (7) Model testing: reference Figure 5 shows the evaluation indicators and recognition results of the method proposed in the present application on the test set. The model achieves the following results on the test set: the miss rate (MR) is 0.00%, the mIOU is 80.61%, the accuracy (ACC) is 97.04%, the precision (P) is 97.16%, the recall (R) is 96.21%, and the F1 score is 96.63%. And Figure 6 shows part of the automatic target recognition results of the method proposed in the present application on the test set. The first row shows the recognition results of the human target, the second row shows the recognition results of the cylindrical target, the third row shows the recognition results of the airplane target, and the fourth row shows the recognition results of the ship target. From the visualization results, it can be seen that the method performs well in detecting large and small targets, and also achieves good performance in the positioning accuracy of medium and small targets.

[0173] (8) Comparison with current advanced ATR algorithms: The automatic target recognition of side scan sonar images is essentially a target detection task. Commonly used target detection algorithms can be divided into one-stage detection algorithms and two-stage detection algorithms. Generally speaking, one-stage detection algorithms have a significant advantage in real-time performance, but are slightly insufficient in detection accuracy; while two-stage detection algorithms perform better in detection accuracy, but have larger computational overhead and longer processing time. To comprehensively compare the performance of different detection algorithms, the present application selects representative one-stage detection algorithms in the YOLO series, including YOLOv4, YOLOv5m, YOLOX and YOLOv7; and introduces classic two-stage detection algorithms in the RCNN series, such as Faster R-CNN, Cascade R-CNN, Grid R-CNN, Dynamic R-CNN, Double Head R-CNN and Sparse R-CNN. To verify the effectiveness of the method proposed by the present application, the detection performance of each algorithm is systematically compared and analyzed using ACC, P, R and F1 score indicators.

[0174] It can be seen from the results in Table 2 that the R-CNN series algorithms are overall superior to the YOLO series algorithms in recognition accuracy. Among the YOLO series, YOLOv7 performs best, with an ACC of 95.59% in classification; while among the R-CNN series, Sparse R-CNN performs best, with an ACC of 94.00%. Compared with these two, the method proposed by the present application (Ours) shows significant advantages in all indicators: in terms of ACC, it is 1.45% higher than YOLOv7 and 2.92% higher than Sparse R-CNN; in terms of P, it is 3.69% higher than YOLOv7 and 4.49% higher than Sparse R-CNN; in terms of R, it is 0.54% higher than YOLOv7 and 2.43% higher than Sparse R-CNN; in terms of F1 score, it is 2.39% higher than YOLOv7 and 3.81% higher than Sparse R-CNN.

[0175] Table 2

[0176]

[0177] Reference Figure 7The detection true values under four target types (the leftmost side) and the automatic target recognition results of the method proposed by the application and other detection algorithms (YOLOX, YOLOv7, Grid R-CNN, Dynamic R-CNN, Double Head R-CNN and SparseR-CNN) are shown. It can be observed from the figure that the model using the two-stage detection framework is obviously superior to the one-stage framework in target bounding box positioning accuracy. The method proposed by the application further significantly improves the accuracy of target recognition on the basis of ensuring the target bounding box positioning accuracy, and shows superior comprehensive performance.

[0178] (9) Verify the effectiveness of the ROI extraction algorithm: the application adopts a framework similar to the two-stage target detection algorithm, and divides the task into two stages of ROI extraction and ROI classification, wherein the accurate extraction of the region of interest is crucial for the subsequent classification accuracy, and must avoid the missed detection phenomenon to the greatest extent. In order to verify the effectiveness of YOLOv7 based on transfer learning in the extraction of the region of interest, the application compares and analyzes it with several common ROI extraction algorithms, including Edge Boxes based on image features, Selective Search, and RPN and SSD based on deep learning. Edge Boxes is suitable for images with clear edges and has high computational efficiency; Selective Search generates candidate regions through image segmentation and clustering methods; RPN achieves a good balance between efficiency and accuracy; SSD generates candidate regions through multi-scale feature maps, and is particularly suitable for real-time applications. Through the statistics and analysis of the IoU, average IoU and missed detection rate of each type of target in the test set, the advantages and performance of the proposed method in the extraction of the region of interest are further verified.

[0179] It can be observed from Table 3 that the ROI extraction method proposed by the application achieves 80.61% in the mIoU index, among which the IoU of the human target is 79.94%, the cylinder is 87.50%, the airplane is 75.43%, and the ship is 82.71%. It is worth noting that the MR of this method is 0.00%, that is, no missed detection phenomenon occurs in the test process. Compared with the best performance SSD model, the method improves the mIoU index by 5.00% and reduces the MR by 1.00%. These results show that the ROI extraction algorithm proposed by the application has significant advantages in accuracy and stability.

[0180] Table 3

[0181]

[0182] Reference Figure 8The visual results of the ROI extraction algorithm proposed in the present application and Edge Boxes, Selective Search, RPN and SSD algorithm in the target region extraction task are shown. For large target extraction (left graph of Figure 8 ), the traditional method such as Edge Boxes performs poorly in boundary accuracy and target integrity, while the deep learning method (RPN, SSD and the method proposed in the present application) can better cover the target region, and the performance is close. In the aspect of medium and small target extraction (right graph of Figure 8 ), Edge Boxes almost fail, Selective Search performs well on medium targets, but is poor in small target extraction; RPN has a large deviation in target positioning, and SSD is poor in extraction accuracy of medium targets and is prone to miss detection or positioning deviation. Comprehensive comparison shows that Edge Boxes, Selective Search, RPN and SSD have different degrees of limitations in medium and small target detection and extraction tasks. The ROI extraction algorithm proposed in the present application introduces a target detection model based on YOLOv7, fully utilizes its advantages in small target detection, effectively improves the region extraction accuracy of medium and small targets, realizes zero miss detection, and ensures the accuracy and stability of target region extraction.

[0183] (10) Verify the effectiveness of the region of interest classification algorithm: compared with the current most advanced classification algorithm: in the two-stage target detection algorithm, the subsequent classification and recognition stage is the key factor affecting the final ATR result. The second stage actually classifies the region of interest, and the process includes feature extraction and classification based on extracted features. In the existing classification algorithm, the deep learning-based method performs better in classification accuracy than the traditional method. Therefore, we selected several popular and excellent deep learning classification networks for comparison, including ResNet-50, GoogLeNet, MobileNet, ShuffleNet, AlexNet and VGG16. In order to ensure fairness, transfer learning is introduced in these networks, the pre-trained weights of the COCO dataset are transferred to the sonar dataset, and the same training method as the method proposed in the present application is adopted.

[0184] Reference Table 4 shows the results of different classification methods on the test set. In addition to the algorithm proposed by us, the best performance is the ResNet-50 network, and the ACC reaches 89.63%. From the results, it can be seen that the method proposed in the application is 7.41% higher than ResNet-50 in the ACC index, 11.02% higher in the P index, 4.59% higher in the R index, and 7.84% higher in the F1 index. These results show that the region of interest classification algorithm proposed in the application has a significant improvement in classification accuracy and accuracy.

[0185] Table 4

[0186]

[0187] (11) Verify the effectiveness of feature fusion: In order to verify the effectiveness of the proposed fusion of CNN features and artificial features, different artificial features, single CNN features and feature fusion are tested respectively, and the index results of each are recorded.

[0188] Reference Table 5 shows the index results under different feature extraction methods. The first four rows correspond to the results of single artificial features and CNN features respectively. It can be seen that the performance of CNN features in target classification is obviously better than that of artificial features alone. Rows 5 to 8 show the results after fusing different artificial features, among which the classification accuracy of fusing two artificial features is obviously higher than that of single feature, and the classification accuracy of fusing the designed three artificial features is obviously better than that of fusing two features. Rows 9 to 15 show the results of fusing CNN features and artificial features. For the fusion of CNN features and single artificial features, the classification effect is improved after adding the HOG feature representing shape, while the addition of the GLCM feature representing texture leads to a decline in classification effect, indicating that in this case, feature fusion may introduce redundancy. For the fusion of CNN features and two artificial features, the classification effect is obviously improved after adding Pixel+HOG and GLCM+HOG features, but the Pixel+GLCM fusion result shows performance decline, which also indicates that redundant features may be introduced in this fusion process. The last row shows the result of all feature fusion, i.e. hybrid feature fusion. It can be seen that the fusion of artificial features and CNN features improves the ACC index by 2.23%, the P index by 3.8%, the R index by 3.03%, and the F1 index by 3.41% compared with using only CNN features.

[0189] Table 5

[0190]

[0191] In summary, the mixed feature fusion improves the classification accuracy and precision. However, after completely fusing all features, redundant features appear, which affects the effect of feature fusion. Therefore, feature dimension reduction is increasingly important after feature extraction to avoid the negative impact of feature redundancy on performance.

[0192] (12) Verify the effectiveness of feature dimension reduction: According to the analysis of the aforementioned feature fusion comparison experiment, simply fusing all features may introduce redundant features, thereby reducing the accuracy of classification and recognition. Therefore, this step discusses the effectiveness of feature dimension reduction and compares the performance in classification and recognition before and after feature dimension reduction.

[0193] Reference Table 6 shows the comparison of results before and after feature dimension reduction. According to the previous content, ANOVA is used to reduce artificial features, and PCA is used to reduce CNN features. As can be seen from the figure, for artificial features, the ACC after feature dimension reduction is improved by 5.19%, P is improved by 7.67%, R is improved by 10.53%, and F1 score is improved by 7.43%. For CNN features, the ACC after dimension reduction is improved by 1.48%, P is improved by 2.11%, R is improved by 2.32%, and F1 score is improved by 2.16%. For mixed features, the ACC after dimension reduction is improved by 1.48%, P is improved by 2.99%, R is improved by 1.43%, and F1 score is improved by 2.16%. The above results show that feature dimension reduction can effectively remove redundant features, thereby improving the classification and recognition effect of sonar targets.

[0194] Table 6

[0195]

[0196] (13) Verify the effectiveness of ensemble learning: The classifier of convolutional neural network usually completes classification through full connection layer, but the full connection layer has large parameter quantity, high calculation complexity, long training time and poor performance on small data set. In contrast, machine learning classifiers such as SVM and RF have small parameter quantity, high calculation efficiency, fast training and better stability on small data set. To verify the effectiveness of ensemble learning classifier, two types of comparison experiments are designed: one is to compare the performance of single machine learning classifier (such as SVM, RF) and ensemble learning classifier; the second is to compare full connection layer and ensemble learning classifier, by keeping full connection layer in ResNet50 and replacing full connection layer with ensemble learning classifier, to evaluate the influence of ensemble learning on classification performance.

[0197] As can be seen in Table 7, the ACC of the classification result of the SVM reaches 94.07% in the linear classifier; the ACC of the classification result of the RF is also 94.07% in the nonlinear classifier. It is also shown in the figure that the performance of the combined classifier based on ensemble learning on the test set is better than that of the single classifier, and the ACC is 2.97% higher than that of the best linear classifier SVM and 2.97% higher than that of the best nonlinear classifier RF; the P index is 5.85% higher than that of the SVM and 5.61% higher than that of the RF; the R index is 2.86% higher than that of the SVM and 2.53% higher than that of the RF; the F1 index is 4.31% higher than that of the SVM and 4.03% higher than that of the RF.

[0198] Table 7

[0199]

[0200] The comparison results with the full connection layer also show that the ACC of the ResNet-50 based on the ensemble learning classifier in the test set is 3.7% higher than that of the ResNet-50 using the full connection layer, the P index is 4.23% higher, the R index is 0.13% higher, and the F1 index is 2.27% higher. In summary, ensemble learning not only performs better in the single classifier, but also significantly improves the target classification effect in the classification scheme replacing the full connection layer, proving the effectiveness of ensemble learning in improving the classification accuracy and stability.

[0201] The method provided by the application extracts the target region of interest through the optimized YOLOv7 model, constructs a hybrid feature set by combining deep learning features and artificial features, performs feature dimension reduction and fusion through principal component analysis (PCA) and analysis of variance (ANOVA), finally performs target classification through an ensemble classifier, and further improves the classification accuracy through decision-level fusion.

[0202] By combining the implicit deep features of the CNN with the explicit global description of the artificial features, a more comprehensive sonar target feature expression is constructed, which significantly enhances the distinguishing ability and expression integrity of the features, and provides a richer and more comprehensive feature basis for accurate identification of sonar targets.

[0203] It should be noted that although several modules or units of the device for action execution are mentioned in the foregoing detailed description, such a division is not mandatory. Indeed, according to embodiments of the application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into several modules or units embodied.

[0204] Figure 9 A schematic diagram of an electronic device suitable for use in implementing embodiments of the application is shown.

[0205] It should be noted that Figure 9 The electronic device 1000 shown is merely an example and should not be taken as limiting the functionality or use of embodiments of the application.

[0206] As Figure 9 shown, the electronic device 1000 includes a central processing unit (CPU) 1001 which can perform various appropriate actions and processes in accordance with a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004. The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; the storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as necessary. A removable media 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1010 as necessary, so that a computer program read therefrom is installed into the storage section 1008 as necessary.

[0207] In particular, in accordance with embodiments of the present application, the processes described below with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a storage medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 1009, and / or installed from the removable media 1011. When the computer program is executed by the central processing unit (CPU) 1001, various functions defined in the system of the present application are performed.

[0208] Specifically, the above-mentioned electronic device can be a computer, a tablet computer or a server device.

[0209] It should be noted that the storage medium described in the embodiments of the present invention may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. The program code contained on the storage medium may be transmitted using any suitable medium.

[0210] The units involved in the embodiments of the present invention may be implemented in software or hardware, and the units described may also be provided in a processor. In some cases, the names of these units do not limit the units themselves.

[0211] It should be noted that, as another aspect, the present application also provides a storage medium, which can be included in an electronic device; or it can exist independently without being installed in the electronic device. The above storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments. For example, the electronic device can implement the following Figure 1 The individual steps of the method are shown.

[0212] In one embodiment, the present application provides a computer program product, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0213] Furthermore, the above-described figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above-described figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0214] It should be understood that the application is not limited to the precise construction which has been described above and which shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should be determined by the claims appended hereto.

Claims

1. A side scan sonar target recognition method based on hybrid feature fusion and ensemble learning, characterized in that, The method comprises: Pre-training a convolution feature extraction model, comprising: performing grayscale processing on images in a CIFAR-10 dataset, and performing preliminary training on a ResNet-50 network using the images after grayscale processing to obtain initial weight data of the convolution feature extraction model; performing fine-tuning training on the ResNet-50 model using a side scan sonar image dataset, comprising: freezing the first three residual stages of the ResNet-50 model; optimizing and training the model parameters of the last three residual stages of the ResNet-50 model and the fully connected layer; after the training is completed, removing the fully connected layer, retaining the feature extraction part, and encapsulating it as a convolution feature extraction model; wherein the convolution feature extraction model based on the ResNet-50 network takes the ResNet-50 network as a backbone network; Obtaining a side scan sonar image to be processed, and performing region of interest extraction on the side scan sonar image to obtain a candidate region image; Performing feature extraction on the candidate region image using the convolution feature extraction model based on the ResNet-50 network to obtain image deep learning features; Extracting pixel features, texture features and target shape features from the candidate region image, and performing feature fusion processing to obtain image attribute features; Respectively performing dimension reduction processing on the image deep learning features and the image attribute features, and performing feature fusion processing on the dimension-reduced feature data to obtain a hybrid feature vector, comprising: performing dimension reduction processing on the image deep learning features using a principal component analysis method PCA; performing dimension reduction processing on the target shape features using a variance analysis method ANOVA; maintaining the original dimensions of the pixel features and the texture features, and performing feature splicing processing based on the vertical direction on the dimension-reduced image deep learning features and target shape features to obtain a hybrid feature vector after feature fusion; wherein the image attribute features after dimension reduction retain the original dimensions of the pixel features and the texture features; Inputting the hybrid feature vector into an integrated classifier to enable a first sub-classifier and a second sub-classifier of the integrated classifier to respectively perform classification processing on the hybrid feature vector and obtain first prediction data and second prediction data; wherein the first sub-classifier is constructed based on a support vector machine, and the second sub-classifier is constructed based on a random forest; Performing weighted fusion processing on the first prediction data and the second prediction data to obtain a fusion classification result; wherein the weight coefficients of the weighted fusion processing are determined by a cross-validation method.

2. The method of claim 1, wherein, The region of interest extraction on the side scan sonar image to obtain a candidate region image comprises: Performing region of interest extraction on the side scan sonar image using a region of interest extraction network based on a YOLOv7 network to obtain a candidate region containing a target of interest, and determining position information of a clipping box; Based on a preset pixel size, expanding the clipping box outward to obtain a size-expanded clipping box; Based on the size-expanded clipping box, performing clipping to obtain a candidate region image.

3. The method of claim 2, wherein, The method further comprises: pre-training a region of interest extraction network based on a YOLOv7 network, comprising: The YOLOv7 model pre-trained based on the COCO dataset is taken as a base model; The first two layers of the backbone feature extraction network and the feature fusion network of the base model are frozen; The SGD optimizer is adopted to optimize and train the model parameters of the last two layers of the backbone feature extraction network and the feature fusion network and the detection head part, so as to obtain a region of interest extraction network; wherein the initial learning rate of the SGD optimizer is configured as 0.01, and is dynamically adjusted by combining the cosine annealing strategy.

4. The method of claim 1, wherein, Pixel features, texture features and target shape features are extracted from the candidate region image, including: Statistical moments corresponding to the candidate region image are extracted, and pixel features are calculated according to the statistical moments; wherein the statistical moments include a gray distribution histogram, a first-order moment, a second-order moment and a third-order moment; A gray level co-occurrence matrix GLCM is constructed for the candidate region image, and statistical quantities are calculated to obtain the texture features; wherein the statistical quantities include energy, entropy, contrast and correlation; Gradient intensity and direction of the candidate region image are calculated, the candidate region image is divided into sub-regions, and gradient direction histograms corresponding to each sub-region are determined based on the gradient intensity and direction of the candidate region image; the gradient direction histograms are normalized to obtain the target shape features.

5. The method according to claim 1 or 4, characterized in that, The extracted pixel features, texture features and target shape features are fused to obtain image attribute features, including: The pixel features, texture features and target shape features are vertically spliced and normalized to obtain the image attribute features.

6. The method of claim 1, wherein, Principal component analysis PCA is used to reduce the dimension of the image deep learning features, including: A covariance matrix is calculated using the CNN feature vector extracted from the public side-scan sonar dataset; The covariance matrix is subjected to eigenvalue decomposition to obtain eigenvalues and corresponding eigenvectors; The selected principal components are combined into a projection matrix; The projection matrix is used to map the image deep learning features to a low-dimensional space to realize the dimension reduction of the image deep learning features. The mixed feature vector is input into the integrated classifier, so that the first sub-classifier and the second sub-classifier of the integrated classifier respectively perform classification processing on the mixed feature vector, and first prediction data and second prediction data are obtained, including:

7. The method of claim 1, wherein, The mixed feature vector is input into the first sub-classifier based on the support vector machine, the linear decision boundary is used for prediction, and the class label and the corresponding prediction probability are output; The mixed feature vector is input into the second sub-classifier based on the random forest, the prediction results of multiple decision trees are combined by Bagging and majority voting strategy to determine the class label and the corresponding prediction probability. The computer program is executed by the processor to realize the side-scan sonar target recognition method based on mixed feature fusion and ensemble learning in any one of claims 1 to 7.

8. A computer program product comprising a computer program, characterized in that, ​

Images