Submarine topography classification method and device, electronic equipment and storage medium

Abstract

The invention provides a submarine topography classification method and device, electronic equipment and a storage medium, and relates to the technical field of image processing.The submarine topography classification method comprises the steps that backscattering data and water depth data corresponding to a target submarine area are input into a first modeling network and a second modeling network respectively, extracting a context feature of the backscattering data through a first modeling network to obtain a first modal feature, and extracting a context feature of the water depth data through a second modeling network to obtain a second modal feature; fusing the first modal feature and the second modal feature through a modal shuffling network to obtain a fused feature; performing enhancement processing on the fusion features through an adjacent matrix in the graph transformer network to obtain enhanced features; performing up-sampling processing on the enhanced features through an up-sampling network to obtain target features; and determining a terrain classification result of the target seabed area according to the target features. According to the method, the classification precision of the submarine topography and landform is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, electronic equipment and storage medium for classifying seabed terrain. Background Art

[0002] Seabed topography and geomorphology mapping plays a vital role in scientific research, resource exploration and development, marine engineering and construction, navigation and safety assurance, and environmental protection and management. It is an indispensable basic tool for modern marine activities. Traditional seabed topography and geomorphology survey methods are difficult to operate in deep sea areas, have limited coverage, and are time-consuming, making it difficult to meet the needs of modern marine activities. The use of acoustic detection methods such as multi-beam echo sounding can quickly cover a wide range of sea areas and generate high-resolution three-dimensional seabed topographic maps, greatly improving work efficiency and measurement accuracy.

[0003] However, most of the related technologies are based on traditional convolutional neural networks to extract features from multi-beam data, which leads to incomplete feature extraction of seabed topography and landforms. In addition, due to the similarity of a large number of landform categories, the feature differences are small, which makes the final classification accuracy of seabed topography and landforms low. Summary of the invention

[0004] The problem solved by the present invention is how to improve the classification accuracy of seabed topography.

[0005] In order to solve the above problems, the present invention provides a method, device, electronic equipment and storage medium for classifying seabed terrain.

[0006] In a first aspect, the present invention provides a method for classifying seabed terrain, comprising: Inputting backscattering data and water depth data corresponding to the target seabed area into a first modeling network and a second modeling network respectively, and extracting context features of the backscattering data through the first modeling network to obtain a first modal feature, and extracting context features of the water depth data through the second modeling network to obtain a second modal feature; fusing the first modal feature and the second modal feature through a modal shuffle network to obtain a fused feature; Enhance the fused features by using an adjacency matrix in a graph transformer network to obtain enhanced features; Upsampling the enhanced features through an upsampling network to obtain target features; A terrain classification result of the target seabed area is determined according to the target features.

[0007] Optionally, fusing the first modal feature and the second modal feature through a modal shuffle network to obtain a fused feature includes: Performing convolution processing on the first modal feature to obtain a first query feature, a first key feature, and a first value feature corresponding to the first modal feature; Performing convolution processing on the second modal feature to obtain a second query feature, a second key feature, and a second value feature corresponding to the second modal feature; Determine an interaction feature from the first modal feature to the second modal feature according to the first query feature, the second key feature, and the second value feature to obtain a first interaction feature; Determine an interaction feature from the second modal feature to the first modal feature according to the second query feature, the first key feature, and the first value feature to obtain a second interaction feature; The fusion feature is determined according to the first interaction feature and the second interaction feature.

[0008] Optionally, determining an interaction feature from the first modal feature to the second modal feature according to the first query feature, the second key feature, and the second value feature to obtain a first interaction feature includes: The first interaction feature is determined by the following formula:

[0009] in, represents the first interaction feature, represents the first query feature, represents the dimension of the second key feature, represents the second value feature, Indicates the second key feature The transpose of , A represents the first modal feature, B represents the second modal feature, Represents a normalization operation.

[0010] Optionally, determining the fusion feature according to the first interaction feature and the second interaction feature includes: Splicing the first interaction feature and the second interaction feature to obtain a first splicing feature; Performing dimensionality reduction processing on the first concatenated feature to obtain a first dimensionality reduction feature; The first dimensionality reduction feature is processed using a batch normalization function and an activation function to obtain the fusion feature.

[0011] Optionally, extracting the context feature of the backscattered data through the first modeling network to obtain the first modal feature includes: Downsampling and normalizing the backscattered data to obtain a first input feature; Inputting the first input feature into the state feature extraction module in the first modeling network, wherein the state feature extraction module includes N extraction submodules, and the output feature output by the previous extraction submodule among the N extraction submodules is determined as the input feature corresponding to the next extraction submodule, where N is an integer greater than 1 and N is a preset value; The output features of the Nth extraction submodule among the N extraction submodules are used to determine the first modal features.

[0012] Optionally, the upsampling network includes N upsampling blocks, and the upsampling process is performed on the enhanced feature through the upsampling network to obtain the target feature, including: Obtain the skip connection feature and input sampling feature corresponding to the i-th upsampling block, and obtain the i-th skip connection feature and the i-th input sampling feature, wherein the i-th skip connection feature is the output feature output by the j-th extraction submodule, the first input sampling feature is the enhanced feature, and the i-th input sampling feature is the upsampling feature output by the i-1-th sampling block, i+j=N+1, 1≤i≤N, 1≤j≤N; Inputting the i-th input sampling feature and the i-th skip connection feature into the i-th sampling block, so as to output the i-th up-sampled feature through the i-th sampling block, and determining the i-th up-sampled feature as the i+1-th input sampling feature; The target feature is determined according to the up-sampled feature output by the Nth up-sampling block.

[0013] Optionally, outputting the i-th up-sampled feature through the i-th sampling block includes: Performing convolution processing on the i-th skip connection feature through a first convolution block to obtain a first convolution feature; Performing a series operation on the first convolution feature and the i-th input sampling feature to obtain a first series feature; Performing convolution processing on the first series feature through a second convolution block to obtain a second convolution feature; Perform a transposed convolution operation on the second convolution feature to obtain the i-th up-sampled feature.

[0014] Optionally, before performing enhancement processing on the fused features through the adjacency matrix in the graph transformer network to obtain enhanced features, the method further includes: Acquire a seabed topography knowledge graph, extract text information from the seabed topography knowledge graph, and obtain a knowledge matrix; Mapping the text information corresponding to each terrain in the knowledge matrix to a vector space to obtain a plurality of knowledge vectors; Determine the cosine similarity of any two knowledge vectors among the plurality of knowledge vectors to obtain a plurality of similarity values; The adjacency matrix is ​​constructed according to the multiple similarity values.

[0015] In a second aspect, the present invention provides a submarine terrain classification device, comprising: A feature extraction module, the feature extraction module is used to input the backscattering data and the water depth data corresponding to the target seabed area into the first modeling network and the second modeling network respectively, and extract the context features of the backscattering data through the first modeling network to obtain the first modal features, and extract the context features of the water depth data through the second modeling network to obtain the second modal features; A modal shuffling module, wherein the modal shuffling module is used to fuse the first modal feature and the second modal feature through a modal shuffling network to obtain a fused feature; An enhancement processing module, the enhancement processing module is used to enhance the fusion feature through the adjacency matrix in the graph transformer network to obtain an enhanced feature; An upsampling processing module is used to perform upsampling processing on the enhanced features through an upsampling network to obtain target features.

[0016] In a third aspect, the present invention provides an electronic device, including a memory and a processor; The memory is used to store computer programs; The processor is used to implement the seabed topography classification method as described in the first aspect when executing the computer program.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the seabed topography classification method as described in the first aspect is implemented.

[0018] The beneficial effects of the method for constructing a seabed topography classification model of the present invention are: context feature extraction is performed on backscatter data and water depth data respectively, which can effectively capture the features of different modal data of the target seabed area, form complementarity in the classification process, and improve the comprehensiveness and accuracy of feature expression; and the fusion of the first modal feature and the second modal feature is realized through the modal shuffle network, which can integrate information from different data sources, and can make full use of the advantages of different modal data, so that the fusion feature contains more comprehensive and rich information. The adjacency matrix in the graph transformer network is used to enhance the fusion feature, which can fully explore the relationship between each node in the fusion feature, consider the neighboring nodes and the global graph structure information, and obtain more abundant and effective added features. The enhanced features are processed by the upsampling network, and the feature map can be restored to a higher resolution, so that small landform changes can be captured. For landform categories with small feature differences but small differences in spatial distribution, the upsampling network ensures that the subtle differences of these features are highlighted through detail enhancement, thereby further improving the classification ability and the accuracy of classification. In this embodiment, when extracting features of seabed topography, comprehensive and rich information can be extracted, more complex seabed topography features can be processed, and the classification accuracy of seabed topography is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of a method for classifying seabed terrain according to an embodiment of the present invention; Figure 2 A schematic diagram of a visual state space module in a method for seabed terrain classification according to an embodiment of the present invention; Figure 3 A schematic diagram of a 2D selective scanning process in a method for seabed terrain classification according to an embodiment of the present invention; Figure 4 A schematic diagram of a seabed terrain classification process according to an embodiment of the present invention; Figure 5 A schematic diagram of skip connection features in an upsampling network of a method for seafloor terrain classification according to an embodiment of the present invention; Figure 6 It is a schematic diagram of a flow chart of a sampling block outputting sampling features in a method for seabed terrain classification according to an embodiment of the present invention; Figure 7 A partial flow chart of a method for classifying seabed terrain according to an embodiment of the present invention; Figure 8 A schematic diagram of a submarine terrain classification device according to an embodiment of the present invention; Fig. 9 The figure is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0021] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0022] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0023] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0024] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0025] This embodiment provides a method, device, electronic device and storage medium for classifying seabed terrain.

[0026] like Figure 1 As shown, a method for classifying seabed terrain provided by an embodiment of the present invention includes: Step S100, backscatter data and water depth data corresponding to the target seabed area are respectively input into a first modeling network and a second modeling network, and context features of the backscatter data are extracted through the first modeling network to obtain a first modal feature, and context features of the water depth data are extracted through the second modeling network to obtain a second modal feature; Specifically, first obtain the backscatter data and water depth data of the target seabed area. The backscatter data is the acoustic signal reflection intensity data obtained by the multi-beam echo sounder, which reflects the physical characteristics of the seabed topography, such as roughness, hardness and sediment type. The water depth data is obtained by measuring the vertical distance from the seabed to the water surface, usually generated by a multi-beam echo sounder, which can reflect the topography of the seabed and is the key supplementary information for terrain classification.

[0027] Backscatter data and water depth data are different forms of data corresponding to the target seabed area, and are data of two different modes. Context features in the two modes of data are extracted to obtain features of different modes, namely, first modal features and second modal features.

[0028] When the first modeling network extracts the upper and lower features of the backscatter data, the context features may include at least one of the following: neighborhood information, time series features, and spatial distribution; when the second modeling network extracts the upper and lower features of the water depth data, the context features may include at least one of the following: depth changes, terrain features, etc.

[0029] The first modeling network and the second modeling network can be convolutional neural networks (CNNs), which extract local and global contextual characteristics of backscattered data, such as texture information, optical reflection characteristics, etc., and extract local depth change patterns (such as slopes, mutation points) and global spatial distribution trends of water depth data.

[0030] Specifically, the first modeling network and the second modeling network can also be a long sequence modeling network based on Mamba. The long sequence modeling network uses the visual state space (VSS) block as a basic unit, so that the model selectively propagates or forgets information according to the current input, thereby improving the processing speed and performance of the model in long sequence tasks.

[0031] Step S200, fusing the first modal feature and the second modal feature through a modal shuffle network to obtain a fused feature; Specifically, the Modality-Shuffle Network is a method for enhancing multimodal feature learning. The features extracted from the two modal data are fused in a two-way interactive manner to capture the correlation and dependency between different modal data. For example, backscatter data can usually reflect the physical properties of the seabed surface (such as roughness, material, etc.) well, which is beneficial for identifying different types of bottom materials (such as sand, rock or silt); water depth data is better at capturing the height changes and overall structure of the seabed terrain, which is beneficial for identifying terrain features such as continental shelves, trenches, and seamounts; therefore, the feature fusion of backscatter data and water depth data is realized through the modal shuffle network, that is, the first modal feature and the second modal feature are fused, which can integrate information from different data sources, generate comprehensive fusion features, and capture the correlation between each modality, so that the performance of the fusion feature is increased.

[0032] Step S300, enhancing the fused features through the adjacency matrix in the graph transformer network to obtain enhanced features; Specifically, in the Graph Transformer Network (GTN), the adjacency matrix can express the connection relationship between nodes. As the basis of information transmission, the Graph Transformer uses the adjacency matrix to enhance the features between nodes in the fusion features, thereby taking into account the information of neighbor nodes in the feature expression in the enhanced features.

[0033] Through the graph transformer network, the input fusion features are processed using the constructed adjacency matrix. For each node, not only the characteristics of the node itself are considered, but also the information of other nodes adjacent to or associated with it is integrated.

[0034] Step S400, performing upsampling processing on the enhanced features through an upsampling network to obtain target features; Specifically, the upsampling network refers to the process of restoring low-resolution feature maps or data to high resolution in fields such as image processing, computer vision, and deep learning. The upsampling network is used to restore the high-resolution feature map to obtain the target feature.

[0035] Figure 2It is a schematic diagram of the model structure for extracting target features in this embodiment, including a first modeling network, a second modeling network, a module shuffle network, a graph transformer network and an upsampling network. The backscattering data is extracted through the first modeling network to obtain the first modal feature, and the water depth data is extracted through the second modeling network to obtain the second modal feature. The first modal feature and the second modal feature are fused through the modal shuffle network to obtain the fusion feature of the backscattering data and the water depth data. The fused feature is enhanced through the adjacency matrix in the graph transformer network to obtain the enhanced feature, and then the enhanced feature is upsampled through the upsampling network to obtain the target feature, and the terrain classification result of the seabed area is determined according to the target feature.

[0036] Step S500, determining a terrain classification result of the target seabed area according to the target features.

[0037] Specifically, the processed high-resolution feature map (i.e., target feature) is used to identify and mark the specific terrain types at different locations on the seabed, such as sandy bottom, rocky bottom, muddy bottom, etc.

[0038] In this embodiment, context feature extraction is performed on backscatter data and water depth data respectively, which can effectively capture the features of different modal data of the target seabed area, form complementarity in the classification process, and improve the comprehensiveness and accuracy of feature expression; and the fusion of the first modal feature and the second modal feature is realized through the modal shuffle network, which can integrate information from different data sources, and can make full use of the advantages of different modal data, so that the fusion feature contains more comprehensive and rich information. The adjacency matrix in the graph transformer network is used to enhance the fusion feature, which can fully explore the relationship between each node in the fusion feature, consider the neighboring nodes and the global graph structure information, and obtain more abundant and effective added features. The enhanced features are processed by the upsampling network, and the feature map can be restored to a higher resolution, so that subtle landform changes can be captured. For landform categories with small feature differences but slight differences in spatial distribution, the upsampling network ensures that the subtle differences of these features are highlighted through detail enhancement, thereby further improving the classification ability and improving the classification accuracy. In this embodiment, when extracting features of seabed topography, comprehensive and rich information can be extracted, more complex seabed topography features can be processed, and the classification accuracy of seabed topography is significantly improved.

[0039] Optionally, fusing the first modal feature and the second modal feature through a modal shuffle network to obtain a fused feature includes: Performing convolution processing on the first modal feature to obtain a first query feature, a first key feature, and a first value feature corresponding to the first modal feature; Performing convolution processing on the second modal feature to obtain a second query feature, a second key feature, and a second value feature corresponding to the second modal feature; Determine an interaction feature from the first modal feature to the second modal feature according to the first query feature, the second key feature, and the second value feature to obtain a first interaction feature; Determine an interaction feature from the second modal feature to the first modal feature according to the second query feature, the first key feature, and the first value feature to obtain a second interaction feature; The fusion feature is determined according to the first interaction feature and the second interaction feature.

[0040] Specifically, the modal shuffling module consists of two independent information interaction sub-modules and a feature dimension reduction module, and the two information interaction sub-modules process the first modal feature and the second modal feature respectively.

[0041] The first modal feature is convolved to generate three different feature representations through the convolution layer: the first query feature, the first key feature, and the first value feature; the second modal feature is convolved similarly to generate the second query feature, the second key feature, and the second value feature; the dot product between the first query feature and the second key feature is used to calculate the attention weight, and then the second value feature is weighted and summed according to these weights to obtain the interaction feature from the first modal feature to the second modal feature, that is, the first interaction feature; the dot product between the second query feature and the first key feature is used to calculate the attention weight, and the first value feature is weighted and summed accordingly to generate the interaction feature from the second modal feature to the first modal feature, that is, the second interaction feature; the first interaction feature and the second interaction feature respectively represent the results of the two-way information interaction between the two modal features. Combining the two can fully capture the correlation and dependency between the first modal feature and the second modal feature, while retaining the complementary information of the two modal features, thereby generating a more expressive fusion feature.

[0042] Given an input feature map , the following features are generated through convolution operation:

[0043]

[0044]

[0045] in, , and denote the convolution operations for generating queries, keys, and values, respectively.

[0046] In this optional embodiment, by generating query, key and value features for the first modal feature and the second modal feature respectively, the data of each modality can be parsed more carefully, and important contextual information can be extracted, providing a basis for subsequent interactive operations, making the interactive process more efficient. By calculating the correlation between the query feature of the first modal feature and the key feature of the second modal feature, the correlation between the backscattered data feature and the water depth data feature can be effectively captured, and the interactive information is further generated through the value feature of the second modal feature, which enhances the expression ability of the first modal feature and includes information related to the second modality. By calculating the correlation between the query feature of the second modal feature and the key feature of the first modal feature, the complementary information between the water depth data and the backscattered data can be fully extracted, and the interactive information is generated through the value feature of the first modal feature, which enhances the expression ability of the second modal feature and includes information related to the first modality. By combining the first interactive feature and the second interactive feature to generate a fused feature, we can fully explore the correlation and complementarity between the two modal data, avoid the expression limitations of a single modal feature, and ensure that the generated fused feature has stronger classification and semantic expression capabilities, providing a more reliable input for subsequent terrain classification.

[0047] Optionally, determining an interaction feature from the first modal feature to the second modal feature according to the first query feature, the second key feature, and the second value feature to obtain a first interaction feature includes: The first interaction feature is determined by the following formula:

[0048] in, represents the first interaction feature, represents the first query feature, represents the dimension of the second key feature, represents the second value feature, Indicates the second key feature The transpose of , A represents the first modal feature, B represents the second modal feature, Represents a normalization operation.

[0049] The second interaction feature is determined by the following formula:

[0050] in, represents the second interaction feature, represents the second query feature, represents the dimension of the first key feature, represents the first value feature, Indicates the first key feature The transpose of , A represents the first modal feature, B represents the second modal feature, Represents a normalization operation.

[0051] In this optional embodiment, by calculating the dot product of the first query feature and the second key feature, the correlation between the first modal feature and the second modal feature can be effectively captured, the semantic relationship between the two modal features can be extracted, and an accurate correlation representation can be provided for subsequent attention calculations. Through the Softmax normalization operation, the dot product result is converted into an attention weight matrix to ensure the numerical stability of the weight distribution, while highlighting the features of high correlation, suppressing the features of low correlation, and improving the effectiveness of the interactive features. The second value feature is weightedly calculated using the attention weight, and the generated interactive features not only retain the important semantic information of the second modal feature, but also are closely integrated with the contextual information of the first modal feature, thereby enhancing the expressive power of the first modal feature. The generated first interactive feature contains the relevant information between the first modal feature and the second modal feature, which can effectively make up for the limitations of the single modal feature and lay a solid foundation for the subsequent generation of fusion features.

[0052] Optionally, determining the fusion feature according to the first interaction feature and the second interaction feature includes: Splicing the first interaction feature and the second interaction feature to obtain a first splicing feature; Performing dimensionality reduction processing on the first concatenated feature to obtain a first dimensionality reduction feature; The first dimensionality reduction feature is processed using a batch normalization function and an activation function to obtain the fusion feature.

[0053] Specifically, after completing the two-way information interaction, the features obtained by the interaction are merged through channel splicing to form a higher-dimensional feature representation. Assume that the first modal features and the second modal features after the interaction are and , concatenate them in the channel dimension to obtain the first concatenated feature.

[0054] The first splicing feature is determined by the following formula:

[0055] In order to control the dimension of the output features and reduce the computational complexity, a 1×1 convolution layer is further introduced to perform dimensionality reduction processing to obtain the first dimensionality reduction feature. The first dimensionality reduction feature is processed in combination with batch normalization and activation functions (such as ReLU) to enhance the model's expressiveness and training stability.

[0056] The fusion feature is determined by the following formula:

[0057] In this optional embodiment, the modal shuffling module can not only effectively complete the complex multimodal feature interaction and improve the performance of the model in processing complex tasks, but also compress the fused feature representation into a more reasonable dimensional range through dimensionality reduction operation, thereby reducing the computational overhead of the model. In addition, the module has good adaptability and scalability and can be easily integrated into various network architectures. By splicing the first interactive feature and the second interactive feature, the model can synthesize the data of the two modes to obtain a more comprehensive feature representation, which helps to capture the complex relationship that is difficult to express with a single data source. The dimensionality reduction process of the spliced ​​features not only reduces the complexity of subsequent calculations, but also avoids the overfitting problem caused by the high feature dimension. The dimensionality reduction operation makes the model more efficient while maintaining sufficient expression ability. The introduction of batch normalization can help solve the problem of internal covariate offset, speed up the convergence speed, and improve the robustness of the model. The activation function increases the nonlinear expression ability of the model and helps to learn more complex topographic features. Through the combination of batch normalization and activation function, the quality of feature representation can be effectively enhanced, making the final fusion feature richer and more distinguishable.

[0058] Optionally, extracting the context feature of the backscattered data through the first modeling network to obtain the first modal feature includes: Downsampling and normalizing the backscattered data to obtain a first input feature; Inputting the first input feature into the state feature extraction module in the first modeling network, wherein the state feature extraction module includes N extraction submodules, and the output feature output by the previous extraction submodule among the N extraction submodules is determined as the input feature corresponding to the next extraction submodule, where N is an integer greater than 1 and N is a preset value; The output features of the Nth extraction submodule among the N extraction submodules are used to determine the first modal features.

[0059] Specifically, the first modeling network includes a downsampling processing module and a state extraction module. The downsampling module downsamples the backscattering data to obtain the downsampling processing features. Downsampling processing can reduce data resolution and computational complexity, while retaining the key features of the backscattering data, which is convenient for subsequent feature extraction operations. The downsampling processing features are then normalized to obtain the first input feature. Normalization processing can unify the data distribution range, improve data stability and network training convergence speed, and avoid model instability problems caused by data outliers. The state feature extraction module in the first modeling network consists of N extraction submodules, each of which is responsible for processing the input features layer by layer. Specifically, the first extraction submodule receives the first input feature as input and extracts preliminary low-level features; then, the output of each submodule is used as the input of the next submodule to extract higher-level features one by one. The state feature extraction module extracts the contextual features of the backscattering data from low to high through a layer-by-layer progressive processing method, gradually captures the global information and local information in the features, and realizes a more comprehensive feature expression. The output feature of the Nth extraction submodule among the N extraction submodules is determined as the final first modal feature. This feature not only contains the physical properties of the backscatter data, but also combines multi-layer context features, which can more accurately reflect the terrain information and semantic characteristics in the backscatter data.

[0060] Specifically, when the value of N is 4, the state feature extraction module in the first modeling network consists of 4 extraction submodules. The backscattered data is downsampled and normalized to obtain the first input feature as the input feature of the first extraction submodule. The first extraction submodule uses a patch embedding layer (the patch size can be 2*2) and 2 VSS blocks. The patch embedding layer is used to reduce the resolution of the first feature and increase the number of channels (i.e., feature dimension) of the first feature. The 2 VSS blocks are used to further enhance the feature representation capability to obtain the first output feature. The purpose of the patch embedding layer is to divide the input image or feature into small blocks, flatten these small blocks, and then map them to a high-dimensional feature space. By dividing the image or feature into small blocks of fixed size, the low-dimensional features of each patch are embedded into the high-dimensional feature representation through convolution operations or fully connected layers. The first output feature of the first extraction submodule is used as the input feature of the second extraction submodule. The second extraction submodule reduces the resolution of the second feature by using a patch merging layer and two VSS blocks. Through the process of step-by-step downsampling, features at different scales are extracted to capture a larger receptive field. Two VSS blocks are used to further enhance the feature representation capability to obtain the second output feature. The purpose of the patch merging layer is to downsample the feature map, merge adjacent patches into larger blocks, and perform feature aggregation on them. Adjacent patches are fused through convolution operations or other aggregation methods to reduce the spatial resolution while increasing the number of feature channels (or keeping it unchanged). The second output feature of the second extraction submodule is used as the input feature of the third extraction submodule. The third extraction submodule obtains the third output feature by using a patch merging layer and 9 VSS blocks. The third output feature of the third extraction submodule is used as the input feature of the fourth extraction submodule. The fourth extraction submodule obtains the fourth output feature, i.e., the first modal feature, by using a patch merging layer and two VSS blocks.

[0061] Figure 4 The internal structure processing flow diagram of the visual state space (VSS) block is as follows: Figure 4As shown, the VSS block expands the feature map through the 2D selective scanning module (SS2D) to form a global receptive field. The SS2D module expands the image block into a sequence by scanning the image along four different directions. The information on each scanning path is processed separately, and then the information is reintegrated to construct a global receptive field. The global receptive field refers to the size of the area mapped by the pixel points on the output feature map of a certain layer in the convolutional neural network (CNN) on the input image. In one embodiment, the backscattered data is used as the input data of the VSS block. First, it is normalized by the first normalization layer to standardize the distribution of the data, thereby accelerating the training process and improving the stability of the model. Next, the normalized data part is passed to the first linear layer, which is mapped to a new feature space by linear transformation to increase the expression ability of the model, and the remaining part of the normalized data passes through the second linear layer. Subsequently, the data passes through the depth separable convolution layer, which extracts features and fuses information from different channels through a combination of depth convolution and point convolution. The data after the depth separable convolution is further passed through the SS2D module for further refinement of feature extraction. Then, the data is normalized through the second normalization layer to ensure that the distribution of the feature map remains stable. Next, the features after the second normalization and the features processed by the second linear layer are convolved, and the features after the convolution operation are further linearly transformed through the third linear layer. Finally, through the residual connection, the original input features are added to the processed features to retain the original information and learn the residual information. The final features are used as the final output of the VSS block.

[0062] Figure 5 is a schematic diagram of 2D selective scanning, such as Figure 5 As shown in the figure, the scan expansion operation expands the input image into sequences along four different directions (from top left to bottom right, from bottom right to top left, from top right to bottom left, and from bottom left to top right). These sequences are then processed by the S6 module to extract features, ensuring that information from all directions is thoroughly scanned to capture a variety of features. The scan merging operation sums and merges the sequences from different directions in four directions, restoring the output image to the same size as the input image. The numbers in the figure are used to represent pixel positions.

[0063] Specifically, SS2D is a recursive computation of the selective scanning mechanism and the state space model (SSM). Given the input features , the output characteristics of SS2D It can be written as:

[0064]

[0065]

[0066] in, Indicates four different scanning directions, Indicates a scan extension operation. Represents the scan merging operation. S6 represents the selective scan space state order model S is the core SSM operator of the VSS block, which enables each element in the one-dimensional array to interact with any previously scanned sample by compressing the hidden state; Specifically, the backscatter data of the target seabed area is input into the Mamba-based long sequence modeling network, and the first modal features are extracted through the Mamba-based long sequence modeling network. The first modal features include texture features, edge and boundary features, shape features, etc. of the target seabed area. Texture features include the roughness and particle size of the surface material of the target seabed area; edge and boundary features include the boundaries between different terrain units, such as the boundary between seamounts and surrounding flat areas; shape features include the contour features of rock outcrops, coral reefs, etc. in the target seabed area; Specifically, the water depth data characteristics of the target seabed area are input into the Mamba-based long sequence modeling network, and the second modal features are obtained through the Mamba-based long sequence modeling network. The second modal features include the terrain undulation, gradient, shape, etc. of the target seabed area. The terrain undulation of the target seabed area indicates the degree of change in the height of the seabed surface; the gradient indicates the degree of terrain inclination; and the shape indicates the landform type of the seabed, such as seamounts, trenches, continental shelves, etc.

[0067] In this optional embodiment, by downsampling, the data resolution is reduced, the computational complexity is reduced, the model training speed is accelerated, and the key feature information of the backscattered data is maintained, providing efficient input data for subsequent feature extraction. By unifying the data distribution range, the training stability and convergence speed of the model are enhanced, and the adverse effects of numerical fluctuations on model performance are reduced. Each extraction submodule performs progressive processing on the input features, gradually extracts the features of the backscattered data, and ensures the hierarchy and integrity of the features. Through the iterative processing of multiple submodules, the global information and local details in the backscattered data can be captured, and a more comprehensive feature extraction can be achieved. By using the output features of the Nth extraction submodule as the final first modal features, it can be ensured that the extracted features have higher semantic expression capabilities and spatial resolution capabilities. The first modal features not only reflect the physical properties of the backscattered data, but also combine multi-layer context information, provide high-quality feature input for subsequent modal fusion and classification operations, and significantly improve the accuracy and robustness of classification.

[0068] Optionally, the upsampling network includes N upsampling blocks, and the upsampling process is performed on the enhanced feature through the upsampling network to obtain the target feature, including: Obtain the skip connection feature and input sampling feature corresponding to the i-th upsampling block, and obtain the i-th skip connection feature and the i-th input sampling feature, wherein the i-th skip connection feature is the output feature output by the j-th extraction submodule, the first input sampling feature is the enhanced feature, and the i-th input sampling feature is the upsampling feature output by the i-1-th sampling block, i+j=N+1, 1≤i≤N, 1≤j≤N; It should be noted that, when i=1, the upsampled feature output by the 0th sampling block, that is, the 1st input sample, is the enhanced feature.

[0069] Inputting the i-th input sampling feature and the i-th skip connection feature into the i-th sampling block, so as to output the i-th up-sampled feature through the i-th sampling block, and determining the i-th up-sampled feature as the i+1-th input sampling feature; The target feature is determined according to the up-sampled feature output by the Nth up-sampling block.

[0070] Specifically, Figure 5 As shown in the figure, the upsampling network consists of N upsampling blocks. Each upsampling block is responsible for gradually upsampling the input sampling features and combining the skip connection features to generate a higher resolution feature map. The skip connection features come from the extraction submodule, and the low-level feature information is combined with the high-level feature information through cross-layer connections to ensure the effective fusion of multi-level features. Each upsampling block receives two input features: input sampling features and skip connection features. Among them, the first input sampling feature is an enhanced feature, and the subsequent input sampling features are determined by the output features of the previous upsampling block. The skip connection features come from the jth extraction submodule and satisfy the relationship: i+j=N+1. Each upsampling block outputs an upsampling feature, which is determined as the input sampling feature of the next upsampling block, and finally the output feature of the Nth upsampling block is determined as the target feature. The skip connection feature introduces low-level feature information through cross-layer connections, retains detailed features and local information, and effectively makes up for the details that may be lost in the high-level features during the upsampling process. Through skip connections, low-level and high-level features can be fused in each upsampling block to generate more complete and detailed target features. The sampling feature output by the Nth upsampling block is determined as the final target feature. This feature restores high-resolution feature information through step-by-step upsampling, and combines the semantic information and detail information of multi-level features, providing high-quality input features for seabed terrain classification.

[0071] For example, the upsampling network contains 4 upsampling blocks (N=4). For the first upsampling block (i=1), its jump connection features come from the output features of the last extraction submodule (i.e., the fourth extraction submodule, j=4), and the input features of the second sampling block are the first input sampling features combined with the output features of the fourth extraction submodule.

[0072] In this optional embodiment, the jump connection feature is derived from the output feature of the extraction submodule, which retains the detail information and local characteristics of the low-level features, while the input sampling feature is derived from the output feature of the previous upsampling block, which contains the global information of the high-level features. By combining the jump connection feature with the input sampling feature, the fusion of multi-level features can be achieved in the upsampling process, and the integrity and expression ability of the target features can be improved. The upsampling block gradually restores the enhanced features to a higher resolution, and at the same time, through the fusion with the jump connection feature, it can effectively make up for the detail information that may be lost in the upsampling process of the high-level features, ensuring that the generated upsampling features contain both global semantic information and local detail information. Through step-by-step upsampling and multi-level feature fusion, the features output by the Nth upsampling block have high resolution and high expression ability, which can more accurately reflect the terrain characteristics of the target seabed area, and provide high-quality input features for subsequent terrain classification tasks.

[0073] Optionally, outputting the i-th up-sampled feature through the i-th sampling block includes: Performing convolution processing on the i-th skip connection feature through a first convolution block to obtain a first convolution feature; Performing a series operation on the first convolution feature and the i-th input sampling feature to obtain a first series feature; Performing convolution processing on the first series feature through a second convolution block to obtain a second convolution feature; Perform a transposed convolution operation on the second convolution feature to obtain the i-th up-sampled feature.

[0074] Specifically, the sampling block includes two convolution blocks, the first convolution block and the second convolution block. Each convolution block consists of two convolution layers with LeakyRELU activation. The i-th skip connection feature is input into the first convolution block, and the local information and semantic features in the skip connection feature are extracted by convolution operation to obtain the first convolution feature. The function of the first convolution block is to enhance the features of the skip connection feature and extract a more refined feature representation. The first convolution feature is connected in series with the i-th input sampling feature to obtain the first series feature. The series operation fuses the skip connection feature with the input sampling feature, so that the series feature contains both the local information of the skip connection feature and the global information of the input sampling feature. The first series feature is input into the second convolution block, and the deep information in the fused feature is further extracted by convolution operation to generate the second convolution feature. The function of the second convolution block is to reorganize and enhance the series feature to extract more expressive features. The second convolution feature is transposed convolution operation, the resolution of the feature map is upsampled, and the i-th upsampled feature is generated. The role of transposed convolution is to enlarge the spatial size of the feature map, restore the original resolution, and maintain the spatial structure information of the feature.

[0075] Specifically, a commonly used U-shaped architecture with dense skip connections is used to build a Mamba-based long sequence modeling network. The architecture uses skip connections to recover low-level details and adopts an encoder-decoder structure to extract high-level information. To enhance the native upsampling block in U-Net, additional convolutional blocks with residual connections (i.e., features in submodules) are used to process skip connection features.

[0076] like Figure 6 As shown, given Skip connection features of extracted submodules and the features of the previous upsampling block , No. The output features of the upsampling blocks It can be expressed as:

[0077]

[0078] in, is a feature concatenation operation, Transposed convolution. It is in The extraction submodule has a first convolutional block with residual connections, It is in The extraction submodule has a second convolutional block with residual connections, each It consists of two convolutional layers with LeakyRELU activation.

[0079] In this optional embodiment, the convolution operation can extract local patterns and semantic information in the jump connection features, enhance the features, and make them contain more contextual information. The series operation realizes the fusion of the jump connection features and the input sampling features, fully combines the low-level detail information and the high-level semantic information, so that the features after series connection have multi-level expression capabilities, can effectively make up for the detail information lost in the upsampling process of the input sampling features, and at the same time strengthen the contextual relevance of the jump connection features. The second convolution block further processes the features after series connection, extracts the deep information therein, eliminates redundant features, highlights key features, enhances the expression ability of the fused features, and provides feature input for subsequent upsampling operations. The transposed convolution operation upsamples the feature map resolution to a higher spatial size while retaining the spatial structure information of the features. It can gradually restore the original resolution of the feature map, ensuring that the i-th upsampled feature generated contains both high-quality semantic information and sufficient spatial details, providing support for subsequent target feature generation.

[0080] Alternatively, if Figure 7 As shown, before the fusion feature is enhanced by the adjacency matrix in the graph transformer network to obtain the enhanced feature, the method further includes: Step S210, obtaining a seabed topography knowledge graph, extracting text information in the seabed topography knowledge graph, and obtaining a knowledge matrix; Specifically, the knowledge graph is a structured data form used to represent entities (such as different types of submarine topography) and their relationships in the field of submarine topography. By obtaining the submarine topography knowledge graph, the text information about the topography (such as terrain description, attributes or features) can be extracted to generate a knowledge matrix. Each submarine topography entity is composed of several words that describe its characteristics. These words constitute the basic unit of the knowledge matrix. Each row or column of the knowledge matrix represents a terrain entity or its corresponding feature word, and the element values ​​in the matrix reflect the relationship between these words and entities. For example, for a given submarine ridge entity , and its associated word set can be expressed as .

[0081] Step S220, mapping the text information corresponding to each terrain in the knowledge matrix to a vector space to obtain a plurality of knowledge vectors; Specifically, for each terrain text information in the knowledge matrix, natural language processing technology (such as word embedding model) is used to map it to the vector space to generate a knowledge vector. Each knowledge vector is a numerical representation of the terrain text information in the vector space, which can retain the semantic features of the terrain text.

[0082] Through word embedding technology (such as Word2Vec pre-training model), each text is mapped into a continuous vector space, so that similar words are close to each other in the vector space. For each terrain entity and its characteristic words, the corresponding vector representation is generated through the embedding function:

[0083] is a d-dimensional real-valued vector capturing the word The semantic information of the terrain entity The feature representation of .

[0084] Since the vectors of different terrain entities or feature words may be in different dimensional spaces, it is not reasonable to directly calculate the similarity between them. Therefore, it is necessary to map these vectors into a unified dimensional space through linear projection. The process of linear projection can be expressed as:

[0085] in, is a learnable linear projection matrix, is the bias term, is the dimension after projection. Ensure that all feature vectors are in the same dimensional space to facilitate subsequent similarity calculations.

[0086] Step S230, determining the cosine similarity between any two knowledge vectors among the plurality of knowledge vectors to obtain a plurality of similarity values; Specifically, for the generated multiple knowledge vectors, the cosine similarity between any two knowledge vectors is calculated. Cosine similarity is an indicator to measure the directional similarity of two vectors in the vector space. Its value is between [-1, 1]. The larger the value, the closer the semantics of the two vectors are. By calculating the cosine similarity, the degree of association between terrain text information can be quantified.

[0087] The cosine similarity is calculated using the following formula:

[0088] in, Representation vector The norm of . By calculating the cosine similarity between the knowledge vectors corresponding to different landform entities, a similarity matrix of different landform entities is constructed, which reflects the potential semantic relationship and similarity between different landform entities. The similarity matrix is ​​used as the adjacency matrix in the graph transformer network to guide the propagation and aggregation of node features in the network.

[0089] Step S240: construct the adjacency matrix according to the multiple similarity values.

[0090] Specifically, an adjacency matrix is ​​constructed based on the cosine similarity values ​​of multiple knowledge vectors. The adjacency matrix is ​​used to describe the semantic relationship between terrain entities, where each element of the matrix represents the similarity value between two terrain entities. The larger the similarity value, the stronger the semantic association between the two terrain entities. In the graph transformer network, the adjacency matrix is ​​used as a key part of the graph structure to guide the enhancement processing of terrain features.

[0091] Specifically, in order to consider multiple types of meta-paths simultaneously, the graph transformer network generates multiple graph structures by setting the output channels of the 1×1 filter to C. and the weight vector of the kth GTLayer Become the weight matrix , the output of the kth GT layer is no longer a simple matrix, but a three-dimensional tensor, in which each layer corresponds to a specific graph structure, and the weight vector of each GT layer becomes a weight matrix, which performs parameterized learning on the graph structures in different channels.

[0092] It is expressed in the form of the following tensor equation:

[0093] in, represents the node representation matrix of the kth layer, represents the degree matrix of the kth layer, which is usually a diagonal matrix whose diagonal elements are the degree of each node (that is, the number of edges connected to the node). represents the node representation matrix of the k−1th layer, Depends on the node representation matrix A and parameters , used to generate new node representations.

[0094] After stacking K GT layers, a multi-layer GNN is applied to the output tensor Each channel of and updates the node representation Z.

[0095] It is expressed by the following formula:

[0096] in, is the concatenation operator, C represents the number of channels, Denotes that the node represents the lth GNN layer, Is from The self-loop adjacency matrix of the c-th channel, yes The degree matrix of is a trainable weight matrix shared across channels, is the feature matrix , It is a channel aggregation function, which is used to summarize the information of all channels.

[0097] In this optional embodiment, by acquiring and utilizing the submarine topography knowledge graph, the model can obtain rich background information about the submarine topography, which helps to understand the complex relationship between different terrain entities and improve the accuracy and robustness of classification. Extracting text information to generate a knowledge matrix helps to provide rich semantic information input for each terrain entity. Mapping the text information in the knowledge graph to the vector space enables the model to process these semantic information in numerical form. Using the word embedding model vectorization method, the implicit semantic relationship in the terrain text information can be captured, which is convenient for subsequent similarity calculation and adjacency matrix construction. The vectorization operation makes the text information operable in the mathematical space and is easy to merge with other features. Cosine similarity can quantify the degree of semantic association between the text information of two terrain entities, ensuring the accuracy and effectiveness of similarity calculation. By calculating cosine similarity, the potential relationship between terrain entities hidden in the text information can be mined, providing accurate association data for the construction of the adjacency matrix. As the input of the graph transformer network, the adjacency matrix can guide the enhanced processing of the fusion feature, so that the enhanced feature contains more knowledge semantic information, and improves the expression ability and classification performance of the model. The process of constructing the adjacency matrix combines the semantic information of the knowledge graph with the feature enhancement task, realizes knowledge-driven feature optimization, and further improves the accuracy of seabed topography classification.

[0098] like Figure 8 As shown, an embodiment of the present invention provides a submarine terrain classification device 800, comprising: A feature extraction module 810, the feature extraction module is used to input the backscattering data and the water depth data corresponding to the target seabed area into the first modeling network and the second modeling network respectively, and extract the context features of the backscattering data through the first modeling network to obtain the first modal features, and extract the context features of the water depth data through the second modeling network to obtain the second modal features; A modal shuffling module 820, configured to fuse the first modal feature and the second modal feature through a modal shuffling network to obtain a fused feature; An enhancement processing module 830, the enhancement processing module is used to enhance the fused features through the adjacency matrix in the graph transformer network to obtain enhanced features; The upsampling processing module 840 is used to perform upsampling processing on the enhanced features through an upsampling network to obtain target features.

[0099] like Fig. 9 As shown, an embodiment of the present invention provides an electronic device 900, including a memory 910 and a processor 920; The memory 910 is used to store computer programs; The processor 920 is used to implement the seabed terrain classification method as described above when executing the computer program.

[0100] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the seabed terrain classification method as described above is implemented.

[0101] An electronic device 900 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 900 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein.

[0102] The electronic device 900 includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0103] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. In the present application, the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present invention. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0104] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for classifying seafloor topography, characterized in that: include: Inputting backscattering data and water depth data corresponding to the target seabed area into a first modeling network and a second modeling network respectively, and extracting context features of the backscattering data through the first modeling network to obtain a first modal feature, and extracting context features of the water depth data through the second modeling network to obtain a second modal feature; fusing the first modal feature and the second modal feature through a modal shuffle network to obtain a fused feature; Enhance the fused features by using an adjacency matrix in a graph transformer network to obtain enhanced features; Upsampling the enhanced features through an upsampling network to obtain target features; A terrain classification result of the target seabed area is determined according to the target features.

2. The method for classifying seabed topography according to claim 1, characterized in that: The step of fusing the first modal feature and the second modal feature through a modal shuffle network to obtain a fused feature includes: Performing convolution processing on the first modal feature to obtain a first query feature, a first key feature, and a first value feature corresponding to the first modal feature; Performing convolution processing on the second modal feature to obtain a second query feature, a second key feature, and a second value feature corresponding to the second modal feature; Determine an interaction feature from the first modal feature to the second modal feature according to the first query feature, the second key feature, and the second value feature to obtain a first interaction feature; Determine an interaction feature from the second modal feature to the first modal feature according to the second query feature, the first key feature, and the first value feature to obtain a second interaction feature; The fusion feature is determined according to the first interaction feature and the second interaction feature.

3. The method for classifying seabed topography according to claim 2, characterized in that: The determining, according to the first query feature, the second key feature, and the second value feature, an interaction feature from the first modal feature to the second modal feature to obtain a first interaction feature includes: The first interaction feature is determined by the following formula: in, represents the first interaction feature, represents the first query feature, represents the dimension of the second key feature, represents the second value feature, Indicates the second key feature The transpose of , A represents the first modal feature, B represents the second modal feature, Represents a normalization operation.

4. The method for classifying seabed topography according to claim 1, characterized in that: The extracting the context feature of the backscattered data through the first modeling network to obtain the first modal feature includes: Downsampling and normalizing the backscattered data to obtain a first input feature; Inputting the first input feature into the state feature extraction module in the first modeling network, wherein the state feature extraction module includes N extraction submodules, and the output feature output by the previous extraction submodule among the N extraction submodules is determined as the input feature corresponding to the next extraction submodule, where N is an integer greater than 1 and N is a preset value; The output features of the Nth extraction submodule among the N extraction submodules are used to determine the first modal features.

5. The method for classifying seabed topography according to claim 4, characterized in that: The upsampling network includes N upsampling blocks, and the enhanced features are upsampled by the upsampling network to obtain target features including: Obtain the skip connection feature and input sampling feature corresponding to the i-th upsampling block, and obtain the i-th skip connection feature and the i-th input sampling feature, wherein the i-th skip connection feature is the output feature output by the j-th extraction submodule, the first input sampling feature is the enhanced feature, and the i-th input sampling feature is the upsampling feature output by the i-1-th sampling block, i+j=N+1, 1≤i≤N, 1≤j≤N; Inputting the i-th input sampling feature and the i-th skip connection feature into the i-th sampling block, so as to output the i-th up-sampled feature through the i-th sampling block, and determining the i-th up-sampled feature as the i-th input sampling feature; The target feature is determined according to the up-sampled feature output by the Nth up-sampling block.

6. The method for classifying seabed topography according to claim 3, characterized in that: Outputting the i-th up-sampled feature through the i-th sampling block includes: Performing convolution processing on the i-th skip connection feature through a first convolution block to obtain a first convolution feature; Performing a series operation on the first convolution feature and the i-th input sampling feature to obtain a first series feature; Performing convolution processing on the first series feature through a second convolution block to obtain a second convolution feature; Perform a transposed convolution operation on the second convolution feature to obtain the i-th up-sampled feature.

7. The method for classifying seabed topography according to claim 1, characterized in that: Before enhancing the fused features through the adjacency matrix in the graph transformer network to obtain the enhanced features, the method further includes: Acquire a seabed topography knowledge graph, extract text information from the seabed topography knowledge graph, and obtain a knowledge matrix; Mapping the text information corresponding to each terrain in the knowledge matrix to a vector space to obtain a plurality of knowledge vectors; Determine the cosine similarity of any two knowledge vectors among the plurality of knowledge vectors to obtain a plurality of similarity values; The adjacency matrix is ​​constructed according to the multiple similarity values.

8. A device for classifying seabed terrain, characterized in that: include: A feature extraction module, the feature extraction module is used to input the backscattering data and the water depth data corresponding to the target seabed area into the first modeling network and the second modeling network respectively, and extract the context features of the backscattering data through the first modeling network to obtain the first modal features, and extract the context features of the water depth data through the second modeling network to obtain the second modal features; A modal shuffling module, wherein the modal shuffling module is used to fuse the first modal feature and the second modal feature through a modal shuffling network to obtain a fused feature; An enhancement processing module, the enhancement processing module is used to enhance the fusion feature through the adjacency matrix in the graph transformer network to obtain an enhanced feature; An upsampling processing module is used to perform upsampling processing on the enhanced features through an upsampling network to obtain target features.

9. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the seabed topography classification method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the seabed topography classification method as described in any one of claims 1 to 7 is implemented.

Images