Attention mechanism fused deep sea multi-beam sound intensity data reconstruction method and device
By incorporating the UNet-MCSA model with an attention mechanism and combining shallow and deep-sea multibeam acoustic intensity data, the problem of repairing seabed backscattering data in deep-sea multibeam bathymetry systems has been solved, achieving high-precision and high-visual-effect image repair and filling the gap in deep learning for deep-sea seabed intensity image repair.
Patent Information
- Application Number
- CN202511662779.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-13
- Publication Date
- 2026-02-17
Smart Images

Figure CN121544481A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of multibeam ocean mapping, image processing and deep learning, and specifically to a method and apparatus for reconstructing deep-sea multibeam acoustic intensity data by incorporating an attention mechanism. Background Technology
[0002] The deep sea, as a crucial base for human sustainable development, contains abundant mineral and fishery resources. However, its complex and fragile habitat system and the difficulty of observation hinder exploration. Refined observation of the seabed has become a key step in developing seabed resources. In response, countries around the world are gradually implementing relevant deep-sea exploration programs and strategies, such as the "Digital Deep-Sea Typical Habitat Ocean Science Program (China)," the "Ocean Observation Program (USA)," and the "Blue Growth Strategy (EU)." These programs focus on research areas such as deep-sea ecological analysis and protection, resource assessment and development, military applications, and security, encouraging more experts and scholars to participate in related technology development and promoting the progress of refined and comprehensive deep-sea exploration.
[0003] Deep-sea exploration equipment mainly includes manned submersibles, AUVs, ROVs, multibeam echo sounders, side-scan sonars, synthetic aperture sonars, and deep-sea sampling equipment. Among them, multibeam echo sounders, with their advantages of high precision and full coverage, have become a core technology for deep-sea scientific research. However, in deep-sea conditions, the long acoustic path and complex, unknown environment make it extremely difficult to acquire full-coverage seabed backscattering data. This leads to problems such as central beam anomalies, missed measurements, and false measurements in the generated seabed backscattering mosaic intensity images. To address this issue, noise (central beam anomalies and false measurements) is usually removed first, and then interpolation methods are used to repair the damage. The advantage of this method is that it does not rely on prior models, makes fewer assumptions about the data itself, and maintains a consistent grayscale distribution with the original image. During the repair process, it was found that conventional interpolation methods have good repair effects for smaller damaged areas, but in larger damaged areas, obvious jagged edges, blurring, and incomplete repair occur. On the other hand, the scarcity of deep-sea data means that research on deep learning-based deep-sea seabed intensity image damage repair remains lacking. Therefore, there is an urgent need to build an intensity image restoration model with higher stability and better generalization. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to propose a method and apparatus for reconstructing deep-sea multibeam acoustic intensity data by incorporating an attention mechanism. This method balances the accuracy and visual effect of deep-sea intensity image restoration, demonstrates the effectiveness of the combined deep-shallow image restoration method, and provides a new paradigm for deep-sea multibeam backscattering intensity image restoration.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for reconstructing deep-sea multibeam acoustic intensity data by incorporating an attention mechanism includes the following steps: Step 1: Sample set construction. Shallow and deep-sea multibeam backscatter intensity images are cropped into square images of the same size, and complete regions are selected as the original images. Rectangular and irregular distortions are applied to the original images respectively. The width of the rectangular distortions is set to four different sizes in eight directions. For the irregular distortions, multiple segments are randomly selected using a superpixel segmentation method and nullified. The shallow-sea intensity images are divided into training and testing sets, and the deep-sea intensity images are used as the validation set. Step 2: UNet-MCSA model training. Based on the training set, a UNet-MCSA model based on the UNet structure is constructed. This model includes a residual module, a downsampling module, an upsampling module, and a multi-scale channel spatial attention mechanism module. The feature layers are skipped through the downsampling module and the upsampling module, and then input into the multi-scale channel spatial attention mechanism module. The model is trained using the mean squared error loss function to obtain the UNet-MCSA restoration model for restoring intensity images. Step 3: UNet-MCSA-NNI post-processing workflow. For deep-sea intensity images containing damage, firstly, the nearest neighbor interpolation method is used to generate an NNI repair image; then, the UNet-MCSA model is used to generate a preliminary repair image; finally, using the NNI repair image as a reference, histogram registration is performed on the preliminary repair image generated by UNet-MCSA to obtain the final UNet-MCSA-NNI repair image.
[0006] In step 1, the original image has a size of 128×128 pixels, and the width of the rectangular breakage includes three sizes: 1 pixel, 5 pixels, 10 pixels, and 15 pixels.
[0007] The irregular damage is addressed by dividing the image into 128 superpixel blocks using a superpixel segmentation method, and randomly selecting at least 5 blocks for nullification.
[0008] The shallow sea intensity images are divided into a training set and a test set in a ratio of 8:2.
[0009] The multi-scale channel spatial attention mechanism module includes: a multi-scale channel attention branch, which processes the feature layers after skip connections. Z Pooling was performed using 1×1, 2×2, and 4×4 global average pooling kernels, and the multi-scale channel pooling parameters were obtained by flattening and concatenating the results. CA ; through a fully connected layer with 16x channel compression and Sigmoid Activation function generates channel weights A C ,Will A C andZ Multiplication yields multi-scale channel feature layers Z' Multi-scale spatial attention branch, for Z' Convolution was performed using 3×3, 5×5, and 7×7 kernels respectively, and the parameters were then upsampled and concatenated along the channel dimension to obtain the multi-scale spatial convolution parameters. S A ; through 1×1 convolution and Sigmoid Activation function generates spatial weights A S ,Will A S and Z' Multiplication yields the final multi-scale spatial-channel feature layer. f M .
[0010] The histogram registration uses the cumulative distribution function alignment method to make the grayscale distribution of the UNet-MCSA preliminary repaired image consistent with the grayscale distribution of the NNI repaired image.
[0011] The UNet-MCSA model uses the mean squared error loss function, the optimizer is Adam, and the initial learning rate is 1×10⁻. 5 The batch size is 32, and the training rounds are 1000.
[0012] The evaluation metrics for the restored images include root mean square error, peak signal-to-noise ratio, structural similarity, and Laplacian variance.
[0013] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the deep-sea multibeam acoustic intensity data reconstruction method based on the fusion attention mechanism.
[0014] A deep-sea multibeam acoustic intensity data reconstruction device incorporating an attention mechanism includes: The sample set construction module is used to crop the shallow and deep sea multibeam backscatter intensity images into square images of the same size, and select complete regions from them as the original images; rectangular damage and irregular damage are applied to the original images respectively, wherein the width of the rectangular damage is set to four different sizes in eight directions, and the irregular damage is processed by randomly selecting multiple segments using the superpixel segmentation method and performing null value processing; the shallow sea intensity images are divided into training set and test set, and the deep sea intensity images are used as validation set; The UNet-MCSA model training unit is used to build a UNet-MCSA model based on the UNet structure. This model includes a residual module, a downsampling module, an upsampling module, and a multi-scale channel spatial attention mechanism module. The feature layers are skipped through the downsampling module and the upsampling module, and then fed into the multi-scale channel spatial attention mechanism module. The model is trained using the mean squared error loss function to obtain a UNet-MCSA restoration model for restoring intensity images. The UNet-MCSA-NNI post-processing unit is used to generate an NNI-repaired image from a damaged deep-sea intensity image using nearest neighbor interpolation; then, a preliminary repaired image is generated using the UNet-MCSA model; finally, histogram registration is performed on the preliminary repaired image generated by UNet-MCSA, using the NNI-repaired image as a reference, to obtain the final UNet-MCSA-NNI-repaired image.
[0015] The beneficial technical effects of this invention are as follows: This invention discloses a deep-sea multibeam acoustic intensity data reconstruction method incorporating an attention mechanism, achieving effective restoration of deep-sea intensity images based on the concept of "combining deep and shallow sea data for complementary advantages." Compared with existing methods, the combination of deep and shallow sea data effectively solves the problem of insufficient deep-sea intensity data, filling the gap in deep learning-based deep-sea multibeam intensity image restoration methods. The UNet-MCSA-NNI model achieves dual improvements in accuracy and visual performance, and demonstrates consistent restoration performance across different intensity datasets in both deep and shallow seas, providing an effective approach and strategy for the restoration of deep-sea acoustic images. Attached Figure Description
[0016] Figure 1A This is a diagram showing the construction of multibeam intensity images and sample sets for the sea areas surrounding a certain country.
[0017] Figure 1B Deep-sea multibeam intensity image of a certain area of the Pacific Ocean Figure 2 This is the flowchart of the deep-sea multibeam acoustic intensity data reconstruction method based on the fusion attention mechanism in this invention.
[0018] Figure 3 This diagram illustrates the specific implementation of the UNet-MCSA-NNI model in this invention.
[0019] Figure 4 This is a structural diagram of the MCSA model in this invention.
[0020] Figure 5 This describes the data flow method of the UNet structure in this invention.
[0021] Figure 6(a) shows the deep-sea multibeam backscattering damage intensity image in this invention.
[0022] Figure 6(b) is a restored image of the deep-sea multibeam backscattering damage intensity in this invention. Detailed Implementation
[0023] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0024] This section uses multibeam backscatter intensity images of the nearshore and surrounding waters of a certain country, sourced from a geological survey agency of that country. Figure 1A (North-South direction: positive numbers represent North latitude, negative numbers represent South latitude; East-West direction: positive numbers represent East longitude, negative numbers represent West longitude) This includes 491 intensity images of varying sizes. Data is from a survey area in the Pacific Ocean, with a water depth range of 2000–5000 m. This data was acquired in 2016 using an EM122 multibeam echo sounder and processed using Crais 8.0 software to obtain the multibeam backscatter intensity images. Figure 1B The processed intensity image showed severe central anomalies (part c1). Due to poor sea conditions during operation, the noise level near the transducer at the bottom of the ship was very high, and the echo intensity data was severely interfered with, resulting in mismeasurements and missed measurements (part c2), which appeared as rectangular and irregular damage in the intensity image. The specific process is as follows: Figure 2 As shown, the process includes: Step 1, sample set construction: Shallow and deep-sea multibeam backscattering intensity images of different sizes are cropped and filtered to form complete square intensity images of the same size. Rectangular and irregular damage are designed for these intensity images. The width of the rectangular damage is set differently in eight directions. For irregular damage, superpixel segmentation is used, and randomly selected segmented regions are nulled. Shallow-sea intensity images are selected as the training and test sets, and deep-sea intensity images are selected as the validation set. Step 2, UNet-MCSA model training: Based on the training set, a UNet-MCSA model based on the UNet structure is constructed. This model includes a residual module, a downsampling module, an upsampling module, and a multi-scale... The multi-scale channel spatial attention mechanism module performs skip connections between feature layers (stacked feature maps obtained by convolution of different modules are called feature layers) through the downsampling module and upsampling module, and inputs them into the multi-scale channel spatial attention mechanism module. It is trained using the mean square error loss function to obtain the UNet-MCSA restoration model for restoring intensity images. Step 3, UNet-MCSA-NNI model construction, uses the nearest neighbor interpolation method to interpolate and restore the damaged intensity image. Using the intensity restoration map as a reference, the UNet-MCSA intensity restoration map is based on histogram registration to construct the UNet-MCSA-NNI model. This model is used to restore the deep-sea multibeam backscatter intensity image.
[0025] Step 1, Sample set construction.
[0026] Step 1.1, intensity image cropping and screening: The multibeam backscatter intensity images obtained during the seabed survey are irregular, so they need to be cropped according to the required size. Square intensity images of the same size and with good measurement are selected. After screening and cropping, 5000 complete intensity images of size 128×128 are obtained.
[0027] Step 1.2, simulate damage. For intact intensity images, design rectangular and irregular damage. For rectangular damage, randomly select starting points and directions ([1, 1], [0, 1], [1, 0], [1, -1], [-1, 0], [0, -1], [-1, 1], [-1, -1]), and set different rectangle widths for the same intensity image for nullification. For irregular damage, use superpixel segmentation (initially set to 128 superpixel blocks), randomly select segmentation blocks for nullification, such as... Figure 1B A total of 25,000 sample pairs were obtained.
[0028] Step 1.3: Sample set partitioning. Select shallow sea intensity images and divide them into a training set at a ratio of 4:1. X The test set was used for model training. At the same time, both deep and shallow sea intensity images were grayscaled and median filtered to average the intensity distribution difference between the two and increase the generalization of the model. Meanwhile, the complete deep sea intensity image was selected as the validation set for accuracy verification.
[0029] In step 2, the UNet-MCSA model is constructed.
[0030] Step 2.1, Residual Module f R The algorithm constructs a model that learns the difference between a damage intensity image (input) and a repair intensity image (output) through residual modules and a two-layer convolutional structure. (1); (2); In the formula, F ( X ) represents the residual function that the residual module needs to learn, including the convolutional layers: W 3×3 For 3×3 convolution kernels, W 1×1 1×1 convolution kernel, normalized layer: BN Activation function: ReLU .
[0031] like Figure 3As shown in the residual module, the first layer is a convolutional layer with a kernel size of 3×3 and a stride of 1; the second and fourth layers are normalization layers; the third layer is a convolutional layer with a kernel size of 3×3 and a stride of 1; the shortcut module has a convolutional kernel size of 1×1 and a stride of 1.
[0032] Step 2.2, downsampling module f D Build by using the max pooling layer MP The pooling kernel size and step size of the pooling layer are set accordingly, and combined with the residual module, the spatial size of the intensity image features is reduced, thereby enhancing its receptive field structure. (3).
[0033] like Figure 3 As shown in the downsampling module, the first layer is the residual module; the second layer is the max pooling layer, with a pooling window of 2×2 and a step size of 2; the number of channels is set to 64, 128, 256, 512, and 1024 respectively.
[0034] Step 2.3, Upsampling Module f S Constructed by using bilinear interpolation BI By combining the residual module with the downsampled intensity feature layer, the original image size is reconstructed, thereby restoring its spatial resolution and ensuring that the image output size is aligned with the input size; (4) like Figure 3 As shown in the upsampling module, the first layer is an interpolation layer with a bilinear interpolation scaling factor set to 2; the second layer is a convolutional layer with a kernel size of 3×3 and a stride of 1; the third layer is a normalization layer; and the number of channels is set to 512, 256, 128, and 64 respectively.
[0035] Step 2.4, Construction of the multi-scale channel attention mechanism module, such as... Figure 3 As shown, the feature layers extracted by the upsampling and downsampling modules are concatenated according to their corresponding dimensions to obtain a new feature layer Z after skip connections. Global average pooling is performed on Z using different pooling kernels, namely 1×1, 2×2, and 4×4. Then, the Z layer is flattened and concatenated to obtain multi-scale channel pooling parameters. CA, The number of pooling parameters is 21.
[0036] Using the weight matrix w 1 and activation function ReLU Perform the first layer of 16x feature channel compression, combined with the weight matrix. w 2 and activation function Sigmoid Restore the original number of feature channels and obtain the weights of different feature channels. AC Finally, the feature layer is multiplied by its corresponding weights to obtain the multi-scale channel feature layer. Z’ ; (5); (6).
[0037] Step 2.5, Construction of the multi-scale spatial attention mechanism module, such as... Figure 4 As shown, different convolution kernels, namely 3×3, 5×5, and 7×7, with a stride of 1, are used to perform convolutions on the multi-scale channel feature layers with the average number of layers. The convolved feature layers are then upsampled and concatenated along their dimensions to obtain the multi-scale spatial convolution parameters. SA ; Using 1×1 convolution kernel and Sigmoid The activation function convolves the concatenated feature maps to obtain their spatial feature weights. The channel weight map is then multiplied with the spatial feature weights to obtain a multi-scale spatial-channel feature layer. f M ; (7); (8).
[0038] Step 2.6, UNet-MCSA intensity restoration map output, using as follows Figure 3 The skip connection shown uses two 1×1 convolutional kernels and Sigmoid The activation function restores the number of channels and size of the original image to obtain the predicted output image. Y ; (9).
[0039] Step 2.7, Loss Function: Select the mean squared error loss function, which is calculated by averaging the squares of all prediction errors to measure the difference between the UNet-MCSA predicted value and the true value. (10).
[0040] In the formula, N The sample size is ||·||2 where ||·||2 is the number of samples. L2 Norm.
[0041] Through the above steps, combined with, for example Figure 5 The model is built using the data flow method shown in the UNet structure, with the optimizer set to Adam and an initial learning rate of 10. -5The batch size was 32, and the model was trained for 1000 epochs. The model was saved every 50 epochs, and the best result was selected as the final model. This model was used to perform intensity image inpainting on 5000 test sets. The accuracy was evaluated using root mean square error, peak signal-to-noise ratio, structural similarity, and Laplacian variance. The results are shown in Tables 1 and 2. The inpainting accuracy of the UNet-MCSA model combined with the test set was 0.0152, 38.5288, 0.9270, and 41.1649, respectively; the mean inpainting accuracy combined with the validation set was 0.0095, 40.9011, 0.9688, and 9.5836, respectively.
[0042] Table 1 Table 2 In step 3, the construction of the UNet-MCSA-NNI model includes: Step 3.1, NNI restoration map acquisition: The pixel value at the damaged location in the intensity image is replaced with the pixel value closest to it in the original image using the nearest neighbor interpolation method to obtain the complete NNI deep-sea intensity restoration map, as shown in Tables 1 and 2. The restoration accuracy of NNI combined with the test set is 0.0173, 37.6383, 0.9045, and 58.3685, respectively, and the average restoration accuracy combined with the validation set is 0.0116, 38.8334, 0.9555, and 14.1203, respectively. Step 3.2: Output the UNet-MCSA-NNI restored image. Combine the UNet-MCSA intensity image restoration model from Step 2 to restore the deep-sea damaged intensity image. Based on this restored image and with the NNI restored image as a reference, the histogram registration method is used to obtain the UNet-MCSA-NNI restored image. This makes the restored intensity damaged image closer to the clarity of the original image with minor changes in accuracy. As shown in Tables 1 and 2, the restoration accuracy of the UNet-MCSA-NNI model combined with the test set is 0.0153, 38.5562, 0.9247, and 44.3605, respectively; the mean restoration accuracy of the model combined with the validation set is 0.0097, 40.7936, 0.9694, and 9.8946, respectively.
[0043] Finally, the deep-sea multibeam intensity image was comprehensively restored, combined with Figure 1BAfter median filtering and removal of central abnormal regions from the noise in the deep-sea backscatter intensity data, a damaged grayscale image as shown in Figure 6(a) was obtained. UNet-MCSA was applied to repair the intensity image of the deep-sea multibeam acquisition area, and UNet-MCSA-NNI was used to make the repaired intensity damaged image closer to the clarity of the original image. The result is shown in Figure 6(b), which shows that the method in this paper can take into account the repair accuracy, texture continuity and edge smoothness when processing the damaged area of the deep-sea multibeam backscatter intensity image. It is suitable for handling the problems of strip-shaped missing and cross-sectional noise, and significantly improves the image quality and readability while maintaining the original structural features of the image.
[0044] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, all of which fall within the protection scope of the present invention. The protection scope of the present invention is defined by the appended claims and any equivalent technical solutions.
Claims
1. A method for deep sea multi-beam sound intensity data reconstruction with attention mechanism, characterized in that, The method comprises the following steps: Step 1: sample set construction, shallow and deep sea multi-beam backscattering intensity images are cropped into square images of the same size, complete areas are selected from the square images as original images, rectangular damage and irregular damage are respectively applied to the original images, wherein the width of the rectangular damage is set to four different sizes in eight directions, the irregular damage is randomly selected by a superpixel segmentation method to perform null value processing on multiple segmentation blocks, and the shallow intensity images are divided into a training set and a test set, and the deep intensity images are used as a verification set; Step 2: UNet-MCSA model training, based on the training set, a UNet-MCSA model based on a UNet structure is constructed, the model comprises a residual module, a down-sampling module, an up-sampling module and a multi-scale channel spatial attention mechanism module, the down-sampling module and the up-sampling module are connected in a skip connection mode, and the connected feature maps are input into the multi-scale channel spatial attention mechanism module, the UNet-MCSA model is trained by using a mean square error loss function, and a UNet-MCSA repair model for repairing intensity images is obtained; Step 3: UNet-MCSA-NNI post-processing procedure, for a deep sea intensity image containing damage, a nearest neighbor interpolation method is used to generate an NNI repair image, a UNet-MCSA model is used to generate a preliminary repair image, and finally, the preliminary repair image generated by the UNet-MCSA model is histogram-matched with the NNI repair image as a reference to obtain a final UNet-MCSA-NNI repair image.
2. The method of claim 1, wherein, In step 1, the size of the original image is 128x128 pixels, and the width of the rectangular damage includes three sizes of 1 pixel, 5 pixels, 10 pixels and 15 pixels.
3. The method of claim 1, wherein, The irregular damage is divided into 128 superpixel blocks by a superpixel segmentation method, and at least 5 blocks are randomly selected for null value processing.
4. The method of claim 1, wherein, The ratio of the shallow intensity images divided into the training set and the test set is 8:
2.
5. The method of claim 1, wherein, The multi-scale channel spatial attention mechanism module includes: a multi-scale channel attention branch, which processes the feature layers after skip connections. Z Pooling was performed using 1×1, 2×2, and 4×4 global average pooling kernels, and the multi-scale channel pooling parameters were obtained by flattening and concatenating the results. CA ; through a fully connected layer with 16x channel compression and Sigmoid Activation function generates channel weights A C ,Will A C and Z Multiplication yields multi-scale channel feature layers Z' Multi-scale spatial attention branch, for Z' Convolution was performed using 3×3, 5×5, and 7×7 kernels respectively, and the parameters were then upsampled and concatenated along the channel dimension to obtain the multi-scale spatial convolution parameters. S A ; through 1×1 convolution and Sigmoid Activation function generates spatial weights A S ,Will A S and Z' Multiplication yields the final multi-scale spatial-channel feature layer. f M .
6. The method of claim 1, wherein, The histogram matching adopts a cumulative distribution function alignment method to make the gray distribution of the UNet-MCSA preliminary repair image consistent with the gray distribution of the NNI repair image.
7. The method of claim 1, wherein, The loss function of the UNet-MCSA model is a mean square error loss function, the optimizer is Adam, the initial value of the learning rate is 1x10⁻ 5 , the batch size is 32, and the training round is 1000.
8. The method of claim 1, wherein, The evaluation indexes of the repair image include a root mean square error, a peak signal-to-noise ratio, a structural similarity and a Laplacian variance. 9.A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the deep sea multi-beam sound intensity data reconstruction method with a fusion attention mechanism according to any one of claims 1 to 8.
10. A deep sea multi-beam sound intensity data reconstruction device with fusion attention mechanism, characterized in that, It comprises: A sample set construction module is configured to crop shallow and deep sea multi-beam backscattering intensity images into square images of the same size, select complete areas from the square images as original images, apply rectangular damage and irregular damage to the original images respectively, set the width of the rectangular damage to four different sizes in eight directions, randomly select multiple segmentation blocks by a superpixel segmentation method to perform null value processing on the irregular damage, and divide the shallow intensity images into a training set and a test set, and use the deep intensity images as a verification set. The UNet-MCSA model training unit is configured to construct a UNet-MCSA model based on a UNet structure, the model comprising a residual module, a down-sampling module, an up-sampling module, and a multi-scale channel spatial attention mechanism module, the down-sampling module and the up-sampling module being connected to each other in a skip connection manner, and the connection being input into the multi-scale channel spatial attention mechanism module, the model being trained by using a mean square error loss function, and a UNet-MCSA repair model for repairing an intensity image being obtained; The UNet-MCSA-NNI post-processing flow unit is configured to generate an NNI repair image by using a nearest neighbor interpolation method for a deep-sea intensity image containing damage; generate a preliminary repair image by using the UNet-MCSA model; and finally, perform histogram registration on the preliminary repair image generated by the UNet-MCSA model by taking the NNI repair image as a reference, so as to obtain a final UNet-MCSA-NNI repair image.