Forward-looking sonar image enhancement method based on feature space conversion and multi-frame fusion

Abstract

The invention relates to foresight sonar image enhancement, in particular to a foresight sonar image enhancement method based on feature space conversion and multi-frame fusion, and the method comprises the steps: mapping an original foresight sonar image from a low-quality pixel space to a feature space which is more stable to noise and is more consistent with a heterogeneous image through a feature space conversion module; the forward-looking sonar data self-supervising method comprises the following steps: constructing a self-supervising model to reduce a field gap between a sonar image and a heterogeneous image, so that the self-supervising model more effectively adapts to forward-looking sonar data, the modeling capability of the self-supervising model for unknown degradation factors is enhanced, and consistent input is provided for a multi-frame image fusion enhancement network; complementary information among continuous multi-frame images output by the feature space conversion module is mined and integrated through a multi-frame image fusion enhancement network, fine reconstruction of image details and improvement of image brightness are achieved, and noise in the images is eliminated; according to the technical scheme provided by the invention, the defects that the image brightness is difficult to improve while the noise is suppressed and the high-frequency details of the target cannot be well reserved can be overcome.

Description

Technical Field

[0001] The present invention relates to forward-looking sonar image enhancement, and in particular to a forward-looking sonar image enhancement method based on feature space conversion and multi-frame fusion. Background Art

[0002] Forward-Looking Sonar (FLS), with its outstanding underwater acoustic imaging capabilities, has gradually become an important sensing method for underwater target detection, environmental perception, and autonomous navigation tasks. However, in complex underwater environments, acoustic wave signals are easily affected by water boundary reflections, multipath interference, and the acoustic absorption characteristics of target materials during propagation. This results in the widespread presence of severe speckle noise, low signal-to-noise ratio, weak target contrast, and uneven brightness in forward-looking sonar images. These problems not only significantly reduce the visual quality of forward-looking sonar images, but also have a significant impact on the accuracy of subsequent tasks such as underwater target detection and fine identification, seriously hindering the further development and practical application of forward-looking sonar technology. Therefore, the development of efficient enhancement algorithms for forward-looking sonar images in complex scenarios has become one of the key issues that urgently need to be addressed in the current field of underwater robots and ocean exploration.

[0003] Traditional forward-looking sonar image enhancement methods can generally be categorized as either single-frame image processing or multi-frame image fusion. Single-frame image enhancement methods typically improve image quality through filtering, contrast adjustment, and grayscale conversion. However, due to limited available information, achieving ideal enhancement results from severely degraded forward-looking sonar images is difficult. In contrast, multi-frame image fusion-based image enhancement methods can more effectively exploit the complementary information between consecutive or adjacent frames, demonstrating greater potential for detail reconstruction and noise suppression, and have gradually become a research hotspot. However, while existing multi-frame image fusion strategies (such as image averaging and traditional wavelet fusion) can suppress noise to a certain extent, they often result in overly smoothed target areas and low brightness.

[0004] In recent years, with the continuous development of deep learning, data-driven image enhancement methods have achieved remarkable results. Supervised learning methods rely on high-quality paired training data to construct the mapping relationship between input and enhanced images. However, in real underwater environments, high-quality sonar images are extremely expensive to acquire, making it difficult to obtain sufficient training data. Therefore, self-supervised learning methods have gradually become an effective approach for enhancing low-quality forward-looking sonar images. For example, a multi-frame forward-looking sonar image fusion enhancement method designed for small targets directly fine-tunes a pre-trained model trained on remote sensing images, achieving significant visual effects on small target enhancement. However, this method does not fully consider the domain gap between remote sensing images and forward-looking sonar images. (Although remote sensing images also suffer from speckle noise and low resolution, the distribution of remote sensing images differs significantly from that of forward-looking sonar images. Directly applying a pre-trained model on remote sensing images to self-supervised training on forward-looking sonar images often results in oversmoothing and insufficient brightness in the enhancement results.) Furthermore, it lacks the ability to model unknown degradation factors in real underwater environments, making it difficult to effectively improve image brightness when processing complex targets, and thus unable to ensure the accuracy of subsequent target detection tasks. Summary of the Invention

[0005] (1) Technical problems solved

[0006] In response to the above-mentioned shortcomings of the existing technology, the present invention provides a forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion, which can effectively overcome the defects of the existing technology that it is difficult to improve image brightness while suppressing noise, and cannot well preserve the high-frequency details of the target.

[0007] (2) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] Forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion:

[0010] The feature space conversion module maps the original forward-looking sonar image from a low-quality pixel space to a feature space that is more robust to noise and more consistent with heterogeneous images. This narrows the domain gap between sonar images and heterogeneous images, enables the self-supervised model to more effectively adapt to forward-looking sonar data, enhances the self-supervised model's ability to model unknown degradation factors, and provides consistent input for the multi-frame image fusion enhancement network.

[0011] The multi-frame image fusion enhancement network is used to mine and integrate the complementary information between the continuous multi-frame images output by the feature space conversion module, so as to achieve fine reconstruction of image details and improvement of image brightness, and eliminate noise in the image.

[0012] Preferably, mapping the original forward-looking sonar image from a low-quality pixel space to a feature space that is more robust to noise and more consistent with heterogeneous images through a feature space conversion module includes:

[0013] S11. Perform a predefined feature extraction operation on the single-channel original forward-looking sonar image to obtain supplementary information:

[0014] G i (I)=Φ i (I),i∈{HOG,Canny,Haar,WST,GRE};

[0015] Among them, G i (I) is the feature extracted from the original forward-looking sonar image I, Φ i is the feature extraction function, HOG represents directional feature histogram extraction, Canny represents edge detection, Haar represents Haar feature extraction, WST represents wavelet scattering transform, and GRE represents gray level co-occurrence matrix extraction;

[0016] S12. Concatenate all features extracted from the original forward-looking sonar image to form the enhanced input:

[0017] I aug =concat[I,{G i (I)}];

[0018] Among them, I aug To enhance the input, concat[I,{G i (I)}] represents the feature G extracted from the original forward-looking sonar image I i (1) performing splicing;

[0019] S13. Use convolution operation to perform channel compression and information integration on the enhanced input to obtain the fused output:

[0020] I fused =Ψ(I aug );

[0021] Among them, I fused is the fusion output, Ψ(I aug ) represents the enhanced input I aug Perform 1*1 convolution operation.

[0022] Preferably, the mining and integration of complementary information between the continuous multi-frame images output by the feature space conversion module through the multi-frame image fusion enhancement network includes:

[0023] S21, performing feature extraction on each frame of the low-resolution image sequence output by the feature space conversion module through the encoding module, and mapping the low-resolution image sequence to a high-dimensional feature space;

[0024] S22, integrating the features of each frame in the low-resolution image sequence into a global code through a fusion module;

[0025] S23, upsampling and reconstructing the global code through the decoding module to generate the final super-resolution image;

[0026] Among them, the overall architecture of the multi-frame image fusion enhancement network includes an encoding module E, a fusion module G and a decoding module D.

[0027] Preferably, the working process of the multi-frame image fusion enhancement network is expressed by the following formula:

[0028]

[0029] Where Y is the super-resolution image, E(·) represents the encoding operation, G(·) represents the fusion operation, D(·) represents the decoding operation, J is the low-resolution image sequence output by the feature space conversion module, and J K is the K-th frame image in the low-resolution image sequence J, and K is the number of image frames in the low-resolution image sequence J.

[0030] Preferably, when performing self-supervised training, by introducing local consistency loss and gradient consistency loss, it is ensured that the super-resolution image and the image in the low-resolution image sequence have the same change trend in the local area and the details of the target area, and by comparing the difference information of adjacent areas, the high-frequency details of the image in the low-resolution image sequence are effectively retained to prevent excessive smoothing during the enhancement process.

[0031] Preferably, in the self-supervised training, the process of determining the loss function includes:

[0032] S31. Determine a downsampling mean square error loss, which can ensure that the super-resolution image is consistent with the images in the low-resolution image sequence at the global pixel level:

[0033]

[0034] Among them, L down is the downsampling mean square error loss, H(Y) represents the downsampling operation of the super-resolution image Y, and J * is the target image, ||·||2 means calculating the L2 norm, J * (u,v) is the target image J * The pixel value at (u,v), J K(u, v) is the pixel value of the K-th frame image at (u, v) in the low-resolution image sequence J, and median{·} represents the median operation;

[0035] S32. Determine local consistency loss:

[0036]

[0037] Among them, L con is the local consistency loss, Y p 、Y q are the pixel values ​​of adjacent regions p and q in the super-resolution image, are the pixel values ​​of adjacent regions p and q in the median image of the super-resolution image, σ(p) is the set of regions adjacent to region p in the super-resolution image, and P is the number of regions in the super-resolution image;

[0038] S33. Determine the gradient consistency loss:

[0039]

[0040] Among them, L grad is the gradient consistency loss, Indicates the calculation of gradient;

[0041] S34. Combining the downsampling mean square error loss, local consistency loss and gradient consistency loss, we get the comprehensive loss function:

[0042] L total =L down +λ grad L grad +λ con L con ;

[0043] Among them, L total is the comprehensive loss function, λ grad ,λ con are the weight coefficients of gradient consistency loss and local consistency loss respectively.

[0044] (3) Beneficial effects

[0045] Compared with the prior art, the forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion provided by the present invention has the following beneficial effects:

[0046] 1) A self-supervised forward-looking sonar image enhancement framework that does not require paired training data is proposed, which effectively solves the problems of difficulty in acquiring high-quality images in actual underwater environments and the lack of generalization of traditional supervised learning methods;

[0047] 2) A forward-looking sonar image enhancement method combining feature space transformation and multi-frame image fusion was designed. Through cross-domain transformation of feature level and fusion of multi-frame image details, the domain gap between forward-looking sonar images and remote sensing images was effectively narrowed. While suppressing noise, the brightness of the image was significantly improved, and the high-frequency details of the target in the image could be better preserved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0049] Figure 1 It is a schematic diagram of the process of the present invention;

[0050] Figure 2 The feature maps are obtained by extracting several features from the original forward-looking sonar images at different angles of the target.

[0051] Figure 3 A visual diagram of the domain gap between remote sensing images and forward-looking sonar images;

[0052] Figure 4 Schematic diagram of a multi-frame image fusion enhancement network in the present invention;

[0053] Figure 5 Schematic diagram of collecting forward-looking sonar image data sets for three types of targets in the present invention;

[0054] Figure 6 The enhancement effect diagram of different enhancement methods on the forward-looking sonar image dataset of three types of targets;

[0055] Figure 7 This is a diagram showing the enhancement effect of the enhancement method proposed in this invention on a forward-looking sonar image dataset of three types of targets. DETAILED DESCRIPTION

[0056] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] Forward-looking sonar image enhancement methods based on feature space transformation and multi-frame fusion, such as Figure 1 As shown in

[15] , the feature space conversion module is used to map the original forward-looking sonar image from the low-quality pixel space to a feature space that is more robust to noise and more consistent with heterogeneous images, so as to narrow the domain gap between sonar images and heterogeneous images (e.g. Figure 3 As shown in Figure 3, the self-supervised model can more effectively adapt to forward-looking sonar data, enhance the self-supervised model's ability to model unknown degradation factors, and provide consistent input for the multi-frame image fusion enhancement network;

[0058] The multi-frame image fusion enhancement network is used to mine and integrate the complementary information between the continuous multi-frame images output by the feature space conversion module, so as to achieve fine reconstruction of image details and improvement of image brightness, and eliminate noise in the image.

[0059] The feature space conversion module maps the raw forward-looking sonar image from the low-quality pixel space to a feature space that is more robust to noise and more consistent with heterogeneous images, including:

[0060] S11, perform predefined feature extraction operations on the original forward-looking sonar image of a single channel (such as Figure 2 ), get additional information:

[0061] G i (I)=Φ i (I),i∈{HOG,Canny,Haar,WST,GRE};

[0062] Among them, G i (I) is the feature extracted from the original forward-looking sonar image I, Φ i is the feature extraction function, HOG represents directional feature histogram extraction, Canny represents edge detection, Haar represents Haar feature extraction, WST represents wavelet scattering transform, and GRE represents gray level co-occurrence matrix extraction;

[0063] S12. Concatenate all features extracted from the original forward-looking sonar image to form the enhanced input:

[0064] I aug =concat[I,{G i (I)}];

[0065] Among them, I aug To enhance the input, concat[I,{G i (I)}] represents the feature G extracted from the original forward-looking sonar image I i (1) performing splicing;

[0066] S13. Use convolution operation to perform channel compression and information integration on the enhanced input to obtain the fused output:

[0067] I fused =Ψ(I aug );

[0068] Among them, I fused is the fusion output, Ψ(I aug ) represents the enhanced input I aug Perform 1*1 convolution operation.

[0069] like Figure 3 As shown in Figure 3, the above process maps the original forward-looking sonar image from a low-quality pixel space to a feature space that is more robust to noise and more consistent with heterogeneous images, effectively retaining the key structure of the target while suppressing noise, thereby significantly narrowing the domain gap between sonar images and heterogeneous images (such as remote sensing images).

[0070] The complementary information between the continuous multi-frame images output by the feature space conversion module is mined and integrated through the multi-frame image fusion enhancement network. Figure 4 As shown, including:

[0071] S21, performing feature extraction on each frame of the low-resolution image sequence output by the feature space conversion module through the encoding module, and mapping the low-resolution image sequence to a high-dimensional feature space;

[0072] S22, integrating the features of each frame in the low-resolution image sequence into a global code through a fusion module;

[0073] S23, upsampling and reconstructing the global code through the decoding module to generate the final super-resolution image;

[0074] Among them, the overall architecture of the multi-frame image fusion enhancement network includes an encoding module E, a fusion module G and a decoding module D.

[0075] Specifically, the working process of the multi-frame image fusion enhancement network is expressed as follows:

[0076]

[0077] Where Y is the super-resolution image, E(·) represents the encoding operation, G(·) represents the fusion operation, D(·) represents the decoding operation, J is the low-resolution image sequence output by the feature space conversion module, and J K is the K-th frame image in the low-resolution image sequence J, and K is the number of image frames in the low-resolution image sequence J.

[0078] In the technical solution of the present application, when conducting self-supervised training, by introducing local consistency loss and gradient consistency loss, it is ensured that the super-resolution image and the image in the low-resolution image sequence have the same change trend in the local area and the details of the target area. By comparing the difference information of adjacent areas, the high-frequency details of the image in the low-resolution image sequence are effectively retained to prevent excessive smoothing during the enhancement process.

[0079] In self-supervised training, the process of determining the loss function includes:

[0080] S31. Determine a downsampling mean square error loss, which can ensure that the super-resolution image is consistent with the images in the low-resolution image sequence at the global pixel level:

[0081]

[0082] Among them, L down is the downsampling mean square error loss, H(Y) represents the downsampling operation of the super-resolution image Y, and J * is the target image, ||·||2 means calculating the L2 norm, J * (u,v) is the target image J * The pixel value at (u,v), J K (u, v) is the pixel value of the K-th frame image at (u, v) in the low-resolution image sequence J, and median{·} represents the median operation;

[0083] S32. Determine local consistency loss:

[0084]

[0085] Among them, L con is the local consistency loss, Y p 、Y q are the pixel values ​​of adjacent regions p and q in the super-resolution image, are the pixel values ​​of adjacent regions p and q in the median image of the super-resolution image, σ(p) is the set of regions adjacent to region p in the super-resolution image, and P is the number of regions in the super-resolution image;

[0086] S33. Determine the gradient consistency loss:

[0087]

[0088] Among them, L grad is the gradient consistency loss, Indicates the calculation of gradient;

[0089] S34. Combining the downsampling mean square error loss, local consistency loss and gradient consistency loss, we get the comprehensive loss function:

[0090] L total =L down +λ grad L grad +λ con L con ;

[0091] Among them, L total is the comprehensive loss function, λ grad ,λ con are the weight coefficients of gradient consistency loss and local consistency loss respectively.

[0092] The above technical solution, because the downsampling mean square error loss focuses primarily on global pixel information, can easily lead to oversmoothing of high-frequency details, thereby weakening important features in the target area, which is crucial for subsequent target detection tasks. To avoid this problem, during self-supervised training, local consistency loss and gradient consistency loss are introduced to ensure that the super-resolution image and the image in the low-resolution image sequence have consistent change trends in local areas and target area details. By comparing the difference information between adjacent areas, the high-frequency details of the image in the low-resolution image sequence are effectively preserved, preventing oversmoothing during the enhancement process.

[0093] In order to better illustrate the beneficial effects of the technical solution of the present application, a comparative experiment is conducted below on forward-looking sonar image datasets collected for three types of targets using different enhancement methods to verify the effect.

[0094] The experimental platform is a 24GB NVIDIA GeForce GTX 4090. All forward-looking sonar images have accurate pose parameters, so no additional registration calculation is required before multi-frame image fusion. The fusion inference time for 16 images is 0.3s.

[0095] Build a standard water pool for forward-looking sonar image acquisition (such as Figure 5 (as shown in the figure), is used to collect forward-looking sonar image datasets with accurate pose parameters. Forward-looking sonar image datasets were collected for three target types: rubber tires with severe sound absorption, metal torpedo models, and fiberglass frustums. 3200 images were collected for each target type, with 16 images per group. 180 training sets and 10 validation and test sets were randomly selected, with image size of 376 x 376. The sonar model used was a C900-II, operating at a 2.7 MHz frequency, with a horizontal opening angle of 50° and a vertical opening angle of 15°. The sonar was fixed 1.5 meters from the guide cantilever, pitched 15° horizontally (with respect to the rotating platform), and moved 360° around the target.

[0096] To ensure a fair comparison, the proposed enhancement method was compared with other existing enhancement methods, including wavelet fusion-based enhancement methods and Highres-Net-based enhancement methods. The standard deviation (STD) was used to evaluate the smoothness of the enhanced image; a lower standard deviation indicates a smoother image and greater noise suppression. The average gradient (AG) was also used to measure the high-frequency details of the enhanced image; a higher AG indicates richer high-frequency details.

[0097] In terms of visual comparison, Figure 6 The figure shows the enhancement effects of different enhancement methods on the forward-looking sonar image dataset of three types of targets. Figure 6 (a) is the input original forward-looking sonar image, Figure 6 (b) is the enhancement effect diagram based on wavelet fusion. In this enhancement method, the brightness of the target and the noise in the background are increased. The enhancement effect is not obvious for rubber tires with severe sound absorption. The distinction between the target and the background is not high, making it difficult to be identified. Figure 6 (c) is the enhancement effect image based on Highres-Net. Compared with the enhancement method based on wavelet fusion, the noise in the background is better suppressed and the continuity of the target bright area is improved. However, the contrast between the target and the background is not significantly improved. Figure 6 (d) is the enhancement effect diagram of the enhancement method proposed in the present invention, which improves the brightness of the target area while smoothing the background noise. This method has clear boundaries at the edges of the target area, dark area and reflective area, and the details are well preserved.

[0098] Figure 7 The enhancement effect of the enhancement method proposed in this invention on the forward-looking sonar image dataset of three types of targets is shown. Figure 7 (a) represents the original forward-looking sonar image, Figure 7 (b) shows the visualization result after feature space conversion by the feature space conversion module. Figure 7 (c) shows the final enhanced image.

[0099] In terms of enhancement index comparison, the enhancement indicators of different enhancement methods on the forward-looking sonar image dataset of three types of targets are shown in Tables 1, 2, and 3:

[0100] Table 1 Comparison of reinforcement indexes of rubber tires using different reinforcement methods

[0101] Enhancement Method Original image Wavelet-based fusion Based on Highres-Net Methods Standard deviation 58.787 45.956 45.800 37.676 Average gradient 18.091 22.607 23.067 25.328

[0102] Table 2 Comparison of enhancement indicators of different enhancement methods on torpedo model

[0103]

[0104]

[0105] Table 3 Comparison of enhancement indexes of different enhancement methods on the frustum

[0106] Enhancement Method Original image Wavelet-based fusion Based on Highres-Net Methods Standard deviation 17.016 20.416 5.826 3.235 Average gradient 4.9315 7.992 10.324 13.661

[0107] It can be seen from Tables 1 to 3 that the enhancement method proposed in the present invention is superior to the existing enhancement method in terms of both noise suppression and high-frequency detail preservation.

[0108] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion, characterized by: The feature space conversion module maps the original forward-looking sonar image from a low-quality pixel space to a feature space that is more robust to noise and more consistent with heterogeneous images. This narrows the domain gap between sonar images and heterogeneous images, enables the self-supervised model to more effectively adapt to forward-looking sonar data, enhances the self-supervised model's ability to model unknown degradation factors, and provides consistent input for the multi-frame image fusion enhancement network. The multi-frame image fusion enhancement network is used to mine and integrate the complementary information between the continuous multi-frame images output by the feature space conversion module, so as to achieve fine reconstruction of image details and improvement of image brightness, and eliminate noise in the image.

2. The forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion according to claim 1, characterized in that: The feature space conversion module maps the original forward-looking sonar image from a low-quality pixel space to a feature space that is more robust to noise and more consistent with heterogeneous images, including: S11. Perform a predefined feature extraction operation on the single-channel original forward-looking sonar image to obtain supplementary information: MR i (I)=Φ i (I),i∈{HOG,Canny,Haar,WST,GRE}; Among them, G i (I) is the feature extracted from the original forward-looking sonar image I, Φ i is the feature extraction function, HOG represents directional feature histogram extraction, Canny represents edge detection, Haar represents Haar feature extraction, WST represents wavelet scattering transform, and GRE represents gray level co-occurrence matrix extraction; S12. Concatenate all features extracted from the original forward-looking sonar image to form the enhanced input: I aug =concat[I,{G i (I)}]; Among them, I aug To enhance the input, concat[I,{G i (I)}] represents the feature G extracted from the original forward-looking sonar image I i (1) performing splicing; S13. Use convolution operation to perform channel compression and information integration on the enhanced input to obtain the fused output: I fused =Ψ(I aug ); Among them, I fused is the fusion output, Ψ(I aug ) represents the enhanced input I aug Perform 1*1 convolution operation.

3. The forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion according to claim 2, characterized in that: The method of mining and integrating complementary information between the continuous multi-frame images output by the feature space conversion module through the multi-frame image fusion enhancement network includes: S21, performing feature extraction on each frame of the low-resolution image sequence output by the feature space conversion module through the encoding module, and mapping the low-resolution image sequence to a high-dimensional feature space; S22, integrating the features of each frame in the low-resolution image sequence into a global code through a fusion module; S23, upsampling and reconstructing the global code through the decoding module to generate the final super-resolution image; Among them, the overall architecture of the multi-frame image fusion enhancement network includes an encoding module E, a fusion module G and a decoding module D.

4. The forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion according to claim 3, characterized in that: The working process of the multi-frame image fusion enhancement network is expressed as follows: Where Y is the super-resolution image, E(·) represents the encoding operation, G(·) represents the fusion operation, D(·) represents the decoding operation, J is the low-resolution image sequence output by the feature space conversion module, and J K is the K-th frame image in the low-resolution image sequence J, and K is the number of image frames in the low-resolution image sequence J.

5. The forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion according to claim 3, characterized in that: During self-supervised training, local consistency loss and gradient consistency loss are introduced to ensure that the change trends of super-resolution images and images in low-resolution image sequences in local areas and target area details are consistent. By comparing the difference information of adjacent areas, the high-frequency details of images in low-resolution image sequences are effectively retained to prevent over-smoothing during the enhancement process.

6. The forward-looking sonar image enhancement method based on feature space transformation and multi-frame fusion according to claim 5, characterized in that: In the self-supervised training, the process of determining the loss function includes: S31. Determine a downsampling mean square error loss, which can ensure that the super-resolution image is consistent with the images in the low-resolution image sequence at the global pixel level: Among them, L down is the downsampling mean square error loss, H(Y) represents the downsampling operation of the super-resolution image Y, and J * is the target image, ‖·‖2 represents the calculation of L2 norm, J * (u,v) is the target image J * The pixel value at (u,v), J K (u, v) is the pixel value of the K-th frame image at (u, v) in the low-resolution image sequence J, and median{·} represents the median operation; S32. Determine local consistency loss: Among them, L con is the local consistency loss, Y p 、Y q are the pixel values ​​of adjacent regions p and q in the super-resolution image, are the pixel values ​​of adjacent regions p and q in the median image of the super-resolution image, σ(p) is the set of regions adjacent to region p in the super-resolution image, and P is the number of regions in the super-resolution image; S33. Determine the gradient consistency loss: Among them, L grad is the gradient consistency loss, Indicates the calculation of gradient; S34. Combining the downsampling mean square error loss, local consistency loss and gradient consistency loss, we get the comprehensive loss function: L total =L down +λ grad L grad +λ con L con ; Among them, L total is the comprehensive loss function, λ grad ,λ con are the weight coefficients of gradient consistency loss and local consistency loss respectively.

Images