Underwater image enhancement method fusing multi-scale frequency domain and multi-color gamut features
By integrating multi-scale frequency domain and multi-color domain features, an underwater image enhancement method is developed that solves the problem of insufficient utilization of global and local information in complex water environments by existing technologies. This method achieves efficient underwater image color restoration and detail enhancement, and is applicable to scenarios such as port operation and maintenance, ship repair, search and rescue operations, and underwater robot navigation.
Patent Information
- Application Number
- CN202510946256.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-09
- Publication Date
- 2025-11-11
AI Technical Summary
Existing underwater image enhancement technologies struggle to simultaneously capture both global and local information in complex aquatic environments. Furthermore, they are limited by network depth and computational overhead, making high-resolution real-time inference difficult. Moreover, they fail to fully utilize the complementary characteristics of multiple color gamuts, resulting in failure in extremely turbid or unevenly lit scenarios.
By employing a method that integrates multi-scale frequency domain and multi-color domain features, the features are mapped to the complex frequency domain through Fast Fourier Transform, separating low-frequency global brightness from high-frequency texture details. Parallel convolution and residual enhancement blocks are used to deeply mine the complementary characteristics of brightness, hue and saturation, significantly suppressing color bias and restoring natural colors.
It achieves unified optimization of global consistency and local details in complex aquatic environments, significantly improving the color reproduction accuracy and detail clarity of underwater images. It is applicable to scenarios such as port operation and maintenance, ship repair, search and rescue operations, and underwater robot navigation, improving the safety and efficiency of marine engineering tasks.
Smart Images

Figure CN120931529A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of underwater image processing, and more particularly to an underwater image enhancement method that integrates multi-scale frequency domain and multi-color gamut features. Background Technology
[0002] With the rapid growth of the global marine economy and the continuous advancement of the "deep blue" strategy, underwater optical imaging has become a key means of acquiring seabed information. However, the selective absorption of different wavelengths of light by seawater and the multiple scattering by suspended particles can cause severe color shifts, decreased contrast, and blurred details. These degradation phenomena not only weaken the visual effect of the images but also directly affect the efficiency and safety of port operations, ship maintenance, search and rescue operations, and even the autonomous navigation of underwater robots. Specifically: port and wharf maintenance requires high-quality underwater images to promptly detect corrosion of pier foundation piles, cracks in retaining walls, or risks of siltation, avoiding high maintenance costs caused by structural instability; ship bottom inspection and navigation support rely on high-quality underwater images to identify minor defects such as attached organisms and coating peeling, in order to reduce energy consumption and extend ship life; nearshore and deep-sea search and rescue require front-end systems to output images with natural colors and clear details in real time under turbid or low-light conditions, helping rescuers quickly locate distressed targets; intelligent underwater robots in tasks such as pipeline inspection, marine ranch monitoring, and 3D mapping are highly dependent on the quality of visual input. Therefore, researching underwater image enhancement technologies that can output natural colors and clear details in real time in complex aquatic environments has become an urgent need to ensure the safety of offshore operations and improve the overall efficiency of marine engineering tasks.
[0003] Early methods for addressing underwater image degradation largely borrowed from atmospheric dehazing approaches, using prior estimations of background light and transmittance based on illumination attenuation models and factors such as dark channels and scene depth. However, seawater exhibits significantly greater wavelength absorption differences than the atmosphere, drastically reducing the robustness of these priors in complex water environments. Traditional non-physical model methods (such as histogram equalization, Retinex, gamut linear mapping, and filter fusion), while simple to implement and highly real-time, are prone to over-enhancement, detail loss, and artifacts. In recent years, deep learning-based methods have gained attention: CNNs can learn degradation maps end-to-end, improving color and contrast; GANs enhance the naturalness of generated results through adversarial training. However, these networks generally face three major bottlenecks: (i) insufficient utilization of cross-scale structural information, making it difficult to simultaneously consider global and local aspects; (ii) limitations in network depth and computational overhead, hindering high-resolution real-time inference; and (iii) a lack of explicit utilization of multi-gamut complementary properties such as LAB and HSV, resulting in failure in extremely turbid or unevenly lit scenes. Summary of the Invention
[0004] Based on the technical problems mentioned in the background section, this invention provides an underwater image enhancement method that integrates multi-scale frequency domain and multi-color gamut features. The invention synchronously maps the original RGB image to the LAB and HSV spatial domains, utilizing parallel convolution and residual enhancement blocks to deeply mine the complementary characteristics of brightness (L), hue (H), and saturation (S), significantly suppressing color cast and restoring natural colors. Multi-scale frequency domain feature enhancement (FEB): Features are mapped to the complex frequency domain using Fast Fourier Transform (FFT), separating low-frequency global brightness from high-frequency texture details across three scales. Point convolution and large-kernel convolution modulation are applied separately, followed by layer-by-layer upsampling and stitching to form a cross-scale information flow, fully preserving structural contours and subtle textures. Learnable heterogeneous information adaptive fusion: Learnable weights are introduced in both low-frequency / high-frequency and multi-color gamut branches to dynamically balance the contributions of different information sources. Finally, channels are stitched at the highest resolution and integrated via point convolution, achieving unified optimization of global consistency and local details.
[0005] The technical means employed in this invention are as follows: 1. An underwater image enhancement method that integrates multi-scale frequency domain and multi-color gamut features, characterized by comprising the following steps: Step 1: Use the raw images collected by the underwater robot and perform size normalization processing on the raw images to obtain standardized images; Step 2: Input the standardized image into the frequency domain feature enhancement network to generate enhanced frequency domain features; Step 3: Simultaneously convert the standardized image to the International Commission on Illumination Lab color space and the Hue-Saturation-Luminosity color space, and combine it with the original Red-Green-Blue color space. Input these into multiple cascaded color feature enhancement blocks to form three color features. Then, stitch and fuse the three color features to obtain enhanced color features. Step 4: Concatenate and fuse the enhanced color representation with the frequency domain representation to obtain the final joint features; use L1 loss and structural similarity loss for joint supervision to complete network training.
[0006] Furthermore, step 2 includes the following steps: Step 21: Map the input spatial domain features to the complex frequency domain using a Fast Fourier Transform: ; in, and These represent the high-frequency and low-frequency features obtained from the decomposition, respectively. u Represents frequency domain coordinates; Indicates Fourier transform; Step 22: Apply a voltage to the low-frequency characteristics in the frequency domain. Convolutional layers: ; Step 23: Apply a larger receptive field to high-frequency features. Convolutional layers are used to aggregate neighborhood spectral information. ; Step 24: Introduce learnable fusion coefficients and The processed high- and low-frequency features are weighted and summed to obtain the frequency domain fusion representation: ; Step 25: The fused frequency domain features, along with the separately transformed high and low frequency features, are fed together into the inverse Fourier transform to restore the spatial domain. ; in, This represents the inverse Fourier transform.
[0007] Furthermore, the frequency domain feature enhancement network executes "frequency domain enhancement module - downsampling" sequentially from high to low resolution to construct features at three scales: ; ; ; in, This indicates an average pooling operation; Represents a frequency domain feature enhancement block; Then, starting from the lowest resolution, merge layer by layer upwards: ; ; ; in, This indicates bilinear upsampling; Concat represents the concatenation operation, which merges the upsampled features with the downsampled features. Finally, the highest resolution fusion result is passed through another Conv-ReLU layer to obtain the final frequency domain enhancement feature. .
[0008] Furthermore, step 3 includes the following steps: Step 31: Given the original RGB image, map it to the HSV and Lab color gamuts respectively to obtain Lab and HSV color gamut images. and Among them, the RGB branch fully preserves the original hue; in the LAB branch, the L channel highlights the difference in brightness, and the a and b channels reflect complementary colors; in the HSV branch, H corresponds to hue and S corresponds to saturation, which can independently suppress color cast and enhance local contrast. Step 32: Use one on each color gamut branch. Convolutional layers extract preliminary features: ; ; ; in represent Convolutional layers, , , The initial color features representing the three color gamuts respectively Step 33: Introduce several FEB blocks. Each FEB block consists of two Conv-BatchNorm-ReLU models with residual structures. Stack these models sequentially in each branch. n If there are 1 FEB block, the branch output will be: ; ; ; in, Indicates the three branches n Features enhanced by each FEB block; Step 34: Concatenate the features of the three branches enhanced by FEB in the channel dimension, and use a single... Convolution for compression and interaction: ; in This indicates the enhanced color features output by the color feature module.
[0009] Furthermore, in step 4, the frequency domain fused features and the inversely transformed spatial features are concatenated in the channel dimension, and then processed through a single layer. Convolutional integration yields the final output features. : .
[0010] Furthermore, the loss function combines the L1 loss and the structural similarity loss in a weighted manner to obtain the final training objective function; ; ; ; ; in, Indicates L1 loss, Indicates SSIM loss, Represents the total loss function. and These represent the weight hyperparameters for controlling L1 loss and SSIM loss in the total loss, respectively.
[0011] Compared with the prior art, the present invention has the following advantages: This invention achieves synergistic restoration of multiple degradations in underwater images, including color distortion, contrast loss, and detail blurring, by coupling multi-scale frequency domain enhancement and multi-color gamut information enhancement within the same framework. Compared to existing single spatial domain or single color gamut methods, this invention has the following advantages: (1) Accurate color reproduction: The parallel RGB-LAB-HSV branches combined with learnable weighted fusion enable the network to adaptively compensate for color shifts based on absorption differences at different wavelengths, resulting in natural output colors that conform to human visual perception. (2) High detail clarity: The frequency domain enhancement module based on FFT retains the global brightness tone at low frequencies while using large receptive field convolution to enhance high-frequency textures and edges, significantly improving the recognition of weak details; (3) Strong cross-scale consistency: Top-down multi-scale feature decoupling and bottom-up cascade fusion together ensure that global semantics and local structural information are optimized synchronously, avoiding excessive smoothing or amplification of local noise; (4) High robustness and versatility: It maintains stable performance in extremely turbid waters, low light environments and complex lighting conditions, and can be widely used in port operation and maintenance, ship repair, search and rescue operations and underwater robot navigation, significantly improving the safety and efficiency of marine engineering tasks.
[0012] This invention is primarily applied to underwater target detection, autonomous underwater robot navigation, subsea pipeline and structure inspection, marine life monitoring, shipwreck and artifact detection, and underwater vehicle navigation and obstacle avoidance. The underwater image enhancement method designed in this invention, which integrates multi-scale frequency domain and multi-color gamut features, can significantly improve the visibility and color fidelity of underwater images, correct color casts and enhance contrast, highlight details and target outlines, thereby improving the accuracy of target detection and recognition in turbid or low-light environments. Attached Figure Description
[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a schematic diagram of the underwater image enhancement method that integrates multi-scale frequency domain and multi-color gamut features according to the present invention.
[0015] Figure 2 This is a schematic diagram of the frequency domain feature enhancement module based on Fourier transform of the present invention.
[0016] Figure 3 This is a schematic diagram of the multi-gamut color information enhancement module of the present invention. Detailed Implementation
[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0019] like Figure 1-3 As shown, this invention provides an underwater image enhancement method that integrates multi-scale frequency domain and multi-color gamut features, characterized by comprising the following steps: Step 1: Use the raw images collected by the underwater robot and perform size normalization processing on the raw images to obtain standardized images; Step 2: Input the standardized image into the frequency domain feature enhancement network to generate enhanced frequency domain features; Step 2 includes the following steps: Step 21: Map the input spatial domain features to the complex frequency domain using a Fast Fourier Transform: ; in, and These represent the high-frequency and low-frequency features obtained from the decomposition, respectively. u Represents frequency domain coordinates; Indicates Fourier transform; Step 22: Apply a voltage to the low-frequency characteristics in the frequency domain. Convolutional layers: ; Step 23: Apply a larger receptive field to high-frequency features. Convolutional layers are used to aggregate neighborhood spectral information. ; Step 24: Introduce learnable fusion coefficients and The processed high- and low-frequency features are weighted and summed to obtain the frequency domain fusion representation: ; Step 25: The fused frequency domain features, along with the separately transformed high and low frequency features, are fed together into the inverse Fourier transform to restore the spatial domain. ; in, This represents the inverse Fourier transform.
[0020] Further, step 3 involves simultaneously converting the standardized image to both the International Commission on Illumination (ICI) Lab color space and the Hue-Saturation-Luminosity color space, and combining it with the original Red-Green-Blue color space. Multiple cascaded color feature enhancement blocks are then input into these blocks to form three color features. These three color features are then spliced and fused to obtain enhanced color features. Step 3 includes the following steps: Step 31: Given the original RGB image, map it to the HSV and Lab color gamuts respectively to obtain Lab and HSV color gamut images. and Among them, the RGB branch fully preserves the original hue; in the LAB branch, the L channel highlights the difference in brightness, and the a and b channels reflect complementary colors; in the HSV branch, H corresponds to hue and S corresponds to saturation, which can independently suppress color cast and enhance local contrast. Step 32: Use one on each color gamut branch. Convolutional layers extract preliminary features: ; ; ; in represent Convolutional layers, , , The initial color features representing the three color gamuts respectively Step 33: Introduce several FEB blocks. Each FEB block consists of two Conv-BatchNorm-ReLU models with residual structures. Stack these models sequentially in each branch. n If there are 1 FEB block, the branch output will be: ; ; ; in, Indicates the three branches pass through n Features enhanced by each FEB block; Step 34: Concatenate the features of the three branches enhanced by FEB in the channel dimension, and use a single... Convolution for compression and interaction: ; in This indicates the enhanced color features output by the color feature module.
[0021] Step 4: Concatenate and fuse the enhanced color representation with the frequency domain representation to obtain the final joint feature; use L1 loss and structural similarity loss for joint supervision to complete network training. Concatenate the frequency domain fused feature with the inversely transformed spatial feature in the channel dimension, and then pass it through a single layer. Convolutional integration yields the final output features. : .
[0022] In this application, the frequency domain feature enhancement network executes "frequency domain enhancement module - downsampling" sequentially from high to low resolution to construct features at three scales: ; ; ; in, This indicates an average pooling operation; Represents a frequency domain feature enhancement block; Then, starting from the lowest resolution, merge layer by layer upwards: ; ; ; in, This indicates bilinear upsampling; Concat represents the concatenation operation, which merges the upsampled features with the downsampled features. Finally, the highest resolution fusion result is passed through another Conv-ReLU layer to obtain the final frequency domain enhancement feature. .
[0023] In a preferred embodiment, in this application, the loss function is a weighted combination of L1 loss and structural similarity loss to obtain the final training objective function; ; ; ; ; in, Indicates L1 loss, Indicates SSIM loss, Represents the total loss function. and These represent the weight hyperparameters for controlling L1 loss and SSIM loss in the total loss, respectively.
[0024] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0025] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0026] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0027] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0028] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0029] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An underwater image enhancement method that integrates multi-scale frequency domain and multi-color gamut features, characterized in that, Includes the following steps: Step 1: Use the raw images collected by the underwater robot and perform size normalization processing on the raw images to obtain standardized images; Step 2: Input the standardized image into the frequency domain feature enhancement network to generate enhanced frequency domain features; Step 3: Simultaneously convert the standardized image to the International Commission on Illumination Lab color space and the Hue-Saturation-Luminosity color space, and combine it with the original Red-Green-Blue color space, inputting them into multiple cascaded color feature enhancement blocks to form three-way color features; The three color features are spliced and fused to obtain enhanced color features; Step 4: Concatenate and fuse the enhanced color representation with the frequency domain representation to obtain the final joint features; use L1 loss and structural similarity loss for joint supervision to complete network training.
2. The underwater image enhancement method according to claim 1, which integrates multi-scale frequency domain and multi-color gamut features, is characterized in that... Step 2 includes the following steps: Step 21: Map the input spatial domain features to the complex frequency domain using a Fast Fourier Transform: ; in, and These represent the high-frequency and low-frequency features obtained from the decomposition, respectively. u Represents frequency domain coordinates; Indicates Fourier transform; Step 22: Apply a voltage to the low-frequency characteristics in the frequency domain. Convolutional layers: ; Step 23: Apply a larger receptive field to high-frequency features. Convolutional layers are used to aggregate neighborhood spectral information. ; Step 24: Introduce learnable fusion coefficients and The processed high- and low-frequency features are weighted and summed to obtain the frequency domain fusion representation: ; Step 25: The fused frequency domain features, along with the separately transformed high and low frequency features, are fed together into the inverse Fourier transform to restore the spatial domain. ; in, This represents the inverse Fourier transform.
3. The underwater image enhancement method according to claim 1, which integrates multi-scale frequency domain and multi-color gamut features, is characterized in that... The frequency domain feature enhancement network executes "frequency domain enhancement module - downsampling" sequentially from high to low resolution to construct features at three scales: ; ; ; in, This indicates an average pooling operation; Represents a frequency domain feature enhancement block; Then, starting from the lowest resolution, merge layer by layer upwards: ; ; ; in, This indicates bilinear upsampling; Concat represents the concatenation operation, which merges the upsampled features with the downsampled features. Finally, the highest resolution fusion result is passed through another Conv-ReLU layer to obtain the final frequency domain enhancement feature. .
4. The underwater image enhancement method according to claim 1, which integrates multi-scale frequency domain and multi-color gamut features, is characterized in that... Step 3 includes the following steps: Step 31: Given the original RGB image, map it to the HSV and Lab color gamuts respectively to obtain Lab and HSV color gamut images. and Among them, the RGB branch fully preserves the original hue; in the LAB branch, the L channel highlights the difference in brightness, and the a and b channels reflect complementary colors; in the HSV branch, H corresponds to hue and S corresponds to saturation, which can independently suppress color cast and enhance local contrast. Step 32: Use one on each color gamut branch. Convolutional layers extract preliminary features: ; ; ; in represent Convolutional layers, , , The initial color features representing the three color gamuts respectively Step 33: Introduce several FEB blocks. Each FEB block consists of two Conv-BatchNorm-ReLU models with residual structures. Stack these models sequentially in each branch. n If there are 1 FEB block, the branch output will be: ; ; ; in, Indicates the three branches pass through n Features enhanced by each FEB block; Step 34: Concatenate the features of the three branches enhanced by FEB in the channel dimension, and use a single... Convolution for compression and interaction: ; in This indicates the enhanced color features output by the color feature module.
5. The underwater image enhancement method according to claim 1, which integrates multi-scale frequency domain and multi-color gamut features, is characterized in that... In step 4, the frequency domain fused features and the inversely transformed spatial features are concatenated in the channel dimension, and then connected through a single layer. Convolutional integration yields the final output features. : 。 6. The underwater image enhancement method according to claim 1, which integrates multi-scale frequency domain and multi-color gamut features, is characterized in that... The loss function is a weighted combination of L1 loss and structural similarity loss to obtain the final training objective function; ; ; ; ; in, Indicates L1 loss, Indicates SSIM loss, Represents the total loss function. and These represent the weight hyperparameters for controlling L1 loss and SSIM loss in the total loss, respectively.
Citation Information
Cited By
Submarine cable image splicing method based on double-domain decoupling enhancement and multistage fault-tolerant matching
CN121998820A
Foggy day target detection method based on multi-scale time-frequency information enhancement mechanism
CN122243781A
