Underwater image enhancement method and system based on synchronous feature fusion
Through the underwater image enhancement method of synchronous feature fusion, combined with Transformer and CNN, the adaptive detail weighted loss function and modular structure are used to solve the problem of insufficient image detail retention, and high-quality underwater image enhancement is achieved to adapt to complexity Underwater environment.
Patent Information
- Application Number
- CN202411034034.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-30
AI Technical Summary
Existing underwater image enhancement technologies have shortcomings in image detail retention, especially when deep learning-based methods such as CNN and Transformer process underwater images with color distortion, blur and low contrast, the enhanced image details cannot be effectively retained. .
A synchronous feature fusion method is proposed. Combined with Transformer and CNN, a global feature extraction module and a detailed feature extraction module are established by designing adaptive detail weighted loss functions, and local features and global features under different scales are fused. Dynamically adjust the loss weight to enhance the image.
It effectively overcomes the problem of information loss in the prior art, enhances image quality, retains the fine structure and texture characteristics of the original image, adapts to complex scenes in different underwater environments, and improves the visual perception effect of underwater images.
Smart Images

Figure CN119067863A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image processing, and relates to a synchronous feature fusion underwater image enhancement method and system. Background Art
[0002] Underwater images are affected by optical phenomena such as water scattering, which leads to visual quality problems such as color distortion, reduced contrast and image blur. These optical challenges limit the accurate acquisition and clear display of image information in underwater environments, which is not conducive to the development of underwater exploration, underwater robot vision, marine biology research and other tasks. Therefore, underwater image enhancement technology is of great significance for improving image quality and obtaining clear underwater images.
[0003] Existing underwater image enhancement methods can be divided into methods based on physical models, visual priors, and deep learning. Among them, the physical model-based method estimates environmental parameters and transmission parameters through conditional assumptions, prior knowledge and other related theories, and reversely derives the underwater image degradation process. This method relies on manual priors, which is time-consuming and greatly affected by parameters, and cannot adapt to complex underwater real scenes. The method based on visual priors does not need to consider the optical imaging model, and only enhances the image quality by adjusting the pixel value of the image. Although this method is easy to implement, the algorithm obtained has certain limitations due to ignoring the imaging changes in different water environments, and often presents the phenomenon of insufficient or excessive enhancement. Typical networks of deep learning include CNN, GAN and Transformer. By constructing a neural network to learn the mapping relationship between the distorted image and the reference image end-to-end on the data set, the efficiency of underwater image enhancement tasks can be improved. CNN can accurately extract local features of the image through convolution kernel sliding filtering, and has better generalization ability and faster convergence speed. However, this method still has obvious shortcomings: (1) Since the convolution kernel has a limited receptive field, it pays more attention to the local features of the image, which makes CNN lack the perception of global information; (2) The convolution filter has static weights during the calculation process and cannot adaptively enhance the areas with inconsistent attenuation in the input image. GAN can learn and understand the global characteristics and overall distribution of underwater image data to a certain extent by combining the generator and the discriminator. However, the training of GAN is usually complex and unstable, and may face the problem of mode collapse, which means that the quality of the images generated by the generator is degraded to a certain extent. Transformer has excellent performance in several visual tasks due to its advanced global modeling capabilities. However, when faced with color distortion, blur and low-contrast underwater images, Transformer's focus on long-distance dependencies often makes it impossible to retain the details of the enhanced image. Summary of the invention
[0004] The purpose of the present invention is to solve the problem in the prior art that image details cannot be retained when performing image enhancement, and to provide an underwater image enhancement method and system with synchronous feature fusion.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] The present invention proposes a synchronous feature fusion underwater image enhancement method, comprising the following steps:
[0007] Obtain an open dataset of real underwater scenes and design an adaptive detail-weighted loss;
[0008] Establish an underwater image enhancement network model, and train the underwater image enhancement network model based on an open dataset of underwater real scenes and adaptive detail weighted loss;
[0009] The distorted image is input into the trained underwater image enhancement network model, and the enhanced underwater image is output to achieve underwater image enhancement.
[0010] Preferably, the establishment of the underwater image enhancement network model specifically includes a global feature extraction module and a detail feature extraction module.
[0011] Preferably, the global feature extraction module is as follows:
[0012] enter Divide the input X into non-overlapping patches Each patch is regarded as a "token", which is normalized by the layer and then enters the two-layer convolution to obtain the spatial information of the original feature and project the feature map to the feature representation with three times the number of channels of the input X. Then get the feature X′ Patch Perform window-based multi-head self-attention to calculate relative position deviations; finally, obtain the global feature map through layer normalization and MLP in, It is the flag that the image data is represented as a tensor, H represents the number of rows, W represents the number of columns, and C represents the number of channels.
[0013] Preferably, the global feature extraction module is composed of an even number of modules connected in series, and two consecutive global feature extraction modules use conventional and shifted window multi-head self-attention structures respectively, and the output tensor dimension always remains unchanged from the input.
[0014] Preferably, the detail feature extraction module is as follows:
[0015] For input Use a 1×1 convolution kernel to transform the input features and expand the channel to get the first convolution result;
[0016] Based on the result of the first convolution, three convolution kernels of 1×3, 3×3 and 3×1 are used to perform convolution operations on the same input in different directions and sizes, and batch normalization is performed. The outputs of the three layers are directly added together to obtain the detailed features of the image;
[0017] After a 1×1 convolution operation, the channel dimension is aligned with the detail feature map of the image to obtain the final detail feature.
[0018] Preferably, the adaptive detail weighted loss is specifically:
[0019]
[0020] Among them, α, γ and β are hyperparameters. is the adaptive detail weighted loss function, is the perceptual loss function, is the gradient difference loss function.
[0021] Preferably, the adaptive detail weighted loss function as follows:
[0022]
[0023] The perceptual loss function as follows:
[0024]
[0025] The gradient difference loss function as follows:
[0026]
[0027] in, and i are the enhanced image and the real ground image, α i is the loss value of the i-th pixel, ∈ is a constant e -6 ; N is the number of scales at which the image is decomposed, represents the SSIM value at each scale, β i are weight parameters of different scales; and Represent the gradient of the enhanced image and the reference image at each pixel respectively.
[0028] The present invention proposes a synchronous feature fusion underwater image enhancement system, comprising:
[0029] Parameter acquisition module, used to obtain open datasets of underwater real scenes and design adaptive detail weighted loss;
[0030] Model building module, used to build an underwater image enhancement network model, and train the underwater image enhancement network model based on an open dataset of underwater real scenes and adaptive detail weighted loss;
[0031] The data processing module is used to input the distorted image into the trained underwater image enhancement network model, output the enhanced underwater image, and realize underwater image enhancement.
[0032] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of an underwater image enhancement method with synchronous feature fusion when executing the computer program.
[0033] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of an underwater image enhancement method with synchronous feature fusion.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] The present invention proposes a synchronous feature fusion underwater image enhancement method, which combines Transformer with CNN to establish an underwater image enhancement network model by designing an adaptive detail weighted loss, and trains the underwater image enhancement network model based on an open dataset of underwater real scenes and an adaptive detail weighted loss. The distorted image is input into the trained underwater image enhancement network model, and an enhanced underwater image is output to achieve underwater image enhancement. The synchronous feature fusion underwater image enhancement network model not only plays the structural advantages of each, but also integrates local features and global features at different scales, overcoming the problem of effective information loss in the enhancement process of the prior art. At the same time, the adaptive detail weighted loss is designed, and the loss weight can be dynamically allocated according to the loss size between different pixel points, so that the model can pay more attention to those parts with more serious attenuation, and effectively achieve underwater image enhancement.
[0036] The underwater image enhancement system based on synchronous feature fusion proposed in the present invention finally realizes underwater image enhancement by dividing the system into a parameter acquisition module, a model building module and a data processing module. The modularization concept is adopted to make each module independent of each other, which is convenient for unified management of each module. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0038] Figure 1 This is a flow chart of the underwater image enhancement method of the synchronous feature fusion of the present invention.
[0039] Figure 2 It is the overall network architecture diagram of the present invention.
[0040] Figure 3 It is a structural diagram of the global feature extraction module of the present invention.
[0041] Figure 4 It is a structural diagram of the detail feature extraction module of the present invention.
[0042] Figure 5 A diagram of the underwater image enhancement system of the synchronous feature fusion of the present invention.
[0043] Figure 6 The figure is a schematic diagram of the structure of an electronic device of the present invention. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0045] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0047] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0048] The present invention proposes a synchronous feature fusion underwater image enhancement method, such as Figure 1As shown, the following steps are included:
[0049] S1. Obtain an open dataset of underwater real scenes and design an adaptive detail weighted loss;
[0050] The adaptive detail weighted loss is specifically:
[0051]
[0052] Among them, α, γ and β are hyperparameters. is the adaptive detail weighted loss function, is the perceptual loss function, is the gradient difference loss function.
[0053] The adaptive detail weighted loss function as follows:
[0054]
[0055] The perceptual loss function as follows:
[0056]
[0057] The gradient difference loss function as follows:
[0058]
[0059] in, and i are the enhanced image and the real ground image, α i is the loss value of the i-th pixel, ∈ is a constant e -6 ; N is the number of scales at which the image is decomposed, represents the SSIM value at each scale, β i are weight parameters of different scales; and Represent the gradient of the enhanced image and the reference image at each pixel respectively.
[0060] S2. Establish an underwater image enhancement network model, and train the underwater image enhancement network model based on an open dataset of underwater real scenes and an adaptive detail weighted loss;
[0061] The underwater image enhancement network model is established, which specifically includes a global feature extraction module and a detail feature extraction module.
[0062] The global feature extraction module is specifically as follows:
[0063] enter Divide the input X into non-overlapping patches Each patch is regarded as a "token", which is normalized by the layer and then enters the two-layer convolution to obtain the spatial information of the original feature and project the feature map to the feature representation with three times the number of channels of the input X. Then get the feature X′ Patch Perform window-based multi-head self-attention to calculate relative position deviations; finally, obtain the global feature map through layer normalization and MLP in, It is the flag that the image data is represented as a tensor, H represents the number of rows, W represents the number of columns, and C represents the number of channels.
[0064] The global feature extraction module is composed of an even number of modules in series. The two consecutive global feature extraction modules use multi-head self-attention structures with regular and shifted windows respectively, and the output tensor dimension always remains the same as the input.
[0065] The detail feature extraction module is specifically as follows:
[0066] For input Use a 1×1 convolution kernel to transform the input features and expand the channel to get the first convolution result;
[0067] Based on the result of the first convolution, three convolution kernels of 1×3, 3×3 and 3×1 are used to perform convolution operations on the same input in different directions and sizes, and batch normalization is performed. The outputs of the three layers are directly added together to obtain the detailed features of the image;
[0068] After a 1×1 convolution operation, the channel dimension is aligned with the detail feature map of the image to obtain the final detail feature.
[0069] S3. Input the distorted image into the trained underwater image enhancement network model, output the enhanced underwater image, and achieve underwater image enhancement.
[0070] Combine the following Figures 2 to 4 The underwater image enhancement method is described in detail:
[0071] We obtain the large-scale underwater dataset EUVP, which contains more than 16,000 image pairs for training the network model, and the corresponding 515 image pairs as the full reference test set to evaluate the model. In order to test the generalization of the model, we will also use the UIEB dataset and the U45 dataset as non-reference test sets.
[0072] Transformer has good long-range dependency and global feature extraction capabilities, while CNN has strong performance in local detail feature extraction. In order to better utilize the characteristics of both, a synchronous feature fusion underwater image enhancement network framework is designed. The same input is transmitted to the two modules respectively. The spatial size of the feature map is aligned by using window reset and residual connection in the Transformer block and setting the convolution kernel parameters in the CNN block. In addition, 1×1 convolution is used in CNN to ensure that the output feature map is consistent with the Transformer feature map in the channel dimension, so as to facilitate feature fusion.
[0073] like Figure 2 This is the overall network architecture of the present invention. The network follows the U-Net principle and is an encoder-decoder structure. The output of each encoder is fed into the corresponding mirror decoder through a jump connection. Given an underwater distorted image The low-level features of the original image are obtained through the synchronous feature fusion module 256×256 represents the spatial dimension of the feature map, and 3 represents the number of channels. Specifically, the input image The global feature extraction module processes the input image I and outputs the global feature The detail feature extraction module processes the input image I and outputs the detail features The low-level features of the original image are obtained by fusing the two parts of the output Then, the spatial size is reduced by downsampling, while the number of channels is expanded, and then deeper features are obtained through the synchronous feature fusion module. Finally, we continue to use the downsampling operation and the synchronous feature fusion module to obtain the potential features of the image. And use it as the input of the decoder to gradually reconstruct high-quality real ground images. The decoder part consists only of a global feature extraction module. This is because the decoder needs to convert the abstract coding features into the spatial structure of the image. The Transformer can more naturally generate sequence data corresponding to the image space through the self-attention mechanism and position encoding. In order to assist the reconstruction process, the low-level image features of the encoder part are fused with the high-level image features of the symmetric decoder, so that the network can retain the fine structure and texture features of the original image in the process of obtaining the enhanced image, and finally generate an enhanced image with high-quality visual perception.
[0074] like Figure 3This is the structure diagram of the global feature extraction module of the present invention. This is an improvement based on the Swin Transformer. By stacking 1×1 convolution and 3×3 convolution and embedding them into the Transformer block, the model can better retain the spatial information of the image. This module inherits the original infrastructure, adopts a windowing strategy to process images, and achieves efficient processing of large-size images through block and localized attention mechanisms. This enables the model to flexibly model at various spatial scales and has a complex linear calculation method suitable for images of different sizes. For the input First, divide it into non-overlapping patches Each patch is considered a "token", which is an abstract representation of the original RGB image pixels. After layer normalization, it enters two layers of convolution to obtain the spatial information of the original features and project the feature map to its triple-dimensional feature representation. For feature X′ Patch Perform window-based multi-head self-attention to calculate relative position deviations. Then, through layer normalization and MLP, we get the global feature map.
[0075] The global feature extraction layer is composed of an even number of the above single modules in series. The two consecutive global feature extraction modules use the conventional and shifted window multi-head self-attention structures respectively, and the output tensor dimension always remains the same as the input. Specifically, the output of the previous module is As the input of the current module, the input is normalized by layers and a multi-head attention structure based on a regular window is used to obtain a feature map, and then this feature map is added to the input to obtain the output feature of the multi-head attention. After layer normalization and MLP, a new feature map is obtained. This new feature map is consistent with Add together to get the output X of the current module l ;X l As the input of the next module, a feature map is obtained after layer normalization and a multi-head attention structure based on a shifted window, and this feature map is added to the input to obtain the output feature of the multi-head attention. After layer normalization and MLP, a new feature map is obtained. This new feature map is consistent with Add together to get the output X of the current module l+1 The calculation mechanism is as follows:
[0076]
[0077] in, and X lThey represent the output features of (S)W-MSA and MLP of the lth module respectively; W-MSA and SW-MSA represent the use of multi-head self-attention based on regular windows and shifted windows respectively; LN represents layer normalization.
[0078] For a relative position of the input, the output is calculated through the multi-head self-attention mechanism:
[0079]
[0080] Where Q, K, and V represent query, key, and value matrices respectively. T represents the dot product operation between the query matrix and the key matrix, represents the degree of scaling, and softmax represents the softmax function.
[0081] Figure 4 This is the structural diagram of the detail feature extraction module of the present invention. The key mechanism of this module is to utilize the characteristics of convolution kernels of different sizes: the 3×3 convolution kernel provides a wider range of receptive fields, which helps to capture medium-scale features such as texture and shape; the 3×1 and 1×3 convolution kernels operate in the horizontal and vertical directions respectively, and can capture edge and directional features more finely. Especially in underwater image enhancement tasks, there are differences in the degree of attenuation of degraded images between local areas. This diverse convolution kernel size enables the network to more flexibly adapt to image details of different scales and directions, which helps to improve the model's ability to capture subtle features of the image. For the input Use 1×1 convolution kernel to transform its features and expand the channel to get the features
[0082]
[0083] Among them, F 1×1 is the feature obtained by using a 1×1 convolution kernel, * is a two-dimensional convolution operator, X i is the i-th channel of the input feature matrix, δ(·) is the ReLU activation function, and W 4C,C,1,1 is a single weight within the 1×1 convolution kernel, which is used to expand the number of channels of the input X from C to 4C.
[0084] Then, through three parallel layers, three convolution kernels of 1×3, 3×3 and 3×1 are used to perform convolution operations on the same input in different directions and sizes, and batch normalization is set at the end. Then the outputs of the three layers are directly added together to obtain the detailed features of the image.
[0085] F d =δ(BN(F0*W :,:,1,3)+BN(F0*W :,:,3,3 )+BN(F0*W :,:,3,1 ))
[0086] Among them, F d It is the detail feature obtained by using three convolution kernels, W :,:,1,3 , W :,:,3,3 and W :,:,3,1 They represent convolution kernels of size 1×3, 3×3, and 3×1 respectively, and the output has the same number of feature channels as the input.
[0087] Finally, a 1×1 convolution operation is performed to align the channel dimension with the output feature map of the global feature extraction module to obtain the final output feature map.
[0088] F out =F d *W 4C,C,1,1
[0089] Among them, F out It is the output feature obtained by 1×1 convolution operation.
[0090] Furthermore, the present invention designs three loss functions to collaboratively guide network training. The adaptive detail weighted loss uses Softmax normalization to dynamically allocate loss weights according to the loss size between different pixels, aggravates the penalty for outliers, and minimizes the error between the enhanced image and the reference image at the pixel level. The loss between them is defined as follows:
[0091]
[0092] in, and i are the enhanced image and the real ground image, α i is the loss value of the i-th pixel, ∈ is a small constant, which is empirically set to e -6 .
[0093] The perceptual loss function introduces the concept of multi-scale based on SSIM. By decomposing and comparing images at different scales, it better considers the structure and content information of the image. The loss is defined as follows:
[0094]
[0095] Where N is the number of scales at which the image is decomposed, and i Respectively represent the sub-images of the enhanced image and the reference image at different scales, represents the SSIM value at each scale, β i are weight parameters of different scales.
[0096] Gradient difference loss evaluates the difference between the enhanced image and the reference image by comparing their gradient information. Minimizing the gradient difference loss helps improve the model's understanding and retention of image structure, edges, and details, and can better improve the blur and distortion problems caused by underwater environments. We describe this loss as follows:
[0097]
[0098] in, and Represent the gradient of the enhanced image and the reference image at each pixel respectively.
[0099] The final loss function assigns weights to the above three loss functions and linearly combines them as follows:
[0100]
[0101] in, is the final loss function. α, γ, and β are all hyperparameters. After a large number of experiments, they are set to 0.2, 0.3, and 0.5 respectively to balance the impact of different losses on the training process.
[0102] Example 2
[0103] The present invention proposes a synchronous feature fusion underwater image enhancement system, such as Figure 5 As shown, including:
[0104] Parameter acquisition module, used to obtain open datasets of underwater real scenes and design adaptive detail weighted loss;
[0105] Model building module, used to build an underwater image enhancement network model, and train the underwater image enhancement network model based on an open dataset of underwater real scenes and adaptive detail weighted loss;
[0106] The data processing module is used to input the distorted image into the trained underwater image enhancement network model, output the enhanced underwater image, and realize underwater image enhancement.
[0107] Example 3
[0108] See also Figure 6 As shown, the present invention also provides an electronic device 100 for an underwater image enhancement method with synchronous feature fusion; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.
[0109] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the underwater image enhancement method of synchronous feature fusion described in Example 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the electronic device 100, etc. In addition, the memory 101 can include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
[0110] The at least one processor 102 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor, etc. The processor 102 is the control center of the electronic device 100, and uses various interfaces and lines to connect various parts of the entire electronic device 100.
[0111] The memory 101 in the electronic device 100 stores a plurality of instructions to implement a synchronous feature fusion underwater image enhancement method, and the processor 102 can execute the plurality of instructions to implement:
[0112] Obtain an open dataset of real underwater scenes and design an adaptive detail-weighted loss;
[0113] Establish an underwater image enhancement network model, and train the underwater image enhancement network model based on an open dataset of underwater real scenes and adaptive detail weighted loss;
[0114] The distorted image is input into the trained underwater image enhancement network model, and the enhanced underwater image is output to achieve underwater image enhancement.
[0115] Example 4
[0116] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).
[0117] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0118] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.
[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A synchronous feature fusion underwater image enhancement method, characterized in that: The steps include: Obtain an open dataset of real underwater scenes and design an adaptive detail-weighted loss; Establish an underwater image enhancement network model, and train the underwater image enhancement network model based on an open dataset of underwater real scenes and adaptive detail weighted loss; The distorted image is input into the trained underwater image enhancement network model, and the enhanced underwater image is output to achieve underwater image enhancement.
2. The underwater image enhancement method of synchronous feature fusion according to claim 1, characterized in that: The underwater image enhancement network model is established, which specifically includes a global feature extraction module and a detail feature extraction module.
3. The underwater image enhancement method of synchronous feature fusion according to claim 2, characterized in that: The global feature extraction module is specifically as follows: enter Divide the input X into non-overlapping patches Each patch is regarded as a "token", which is normalized by the layer and then enters two layers of convolution to obtain the spatial information of the original feature and project the feature map to a feature representation with three times the number of channels of the input X. Then get the feature X′ Pa t ch Perform window-based multi-head self-attention to calculate relative position deviations; finally, obtain the global feature map through layer normalization and MLP in, It is the flag that the image data is represented as a tensor, H represents the number of rows, W represents the number of columns, and C represents the number of channels.
4. The underwater image enhancement method of synchronous feature fusion according to claim 3 is characterized in that: The global feature extraction module is composed of an even number of modules in series. The two consecutive global feature extraction modules use multi-head self-attention structures with regular and shifted windows respectively, and the output tensor dimension always remains the same as the input.
5. The underwater image enhancement method of synchronous feature fusion according to claim 2, characterized in that: The detail feature extraction module is specifically as follows: For input Use a 1×1 convolution kernel to transform the input features and expand the channel to get the first convolution result; Based on the result of the first convolution, three convolution kernels of 1×3, 3×3 and 3×1 are used to perform convolution operations on the same input in different directions and sizes, and batch normalization is performed. The outputs of the three layers are directly added together to obtain the detailed features of the image; After a 1×1 convolution operation, the channel dimension is aligned with the detail feature map of the image to obtain the final detail feature.
6. The underwater image enhancement method of synchronous feature fusion according to claim 1, characterized in that: The adaptive detail weighted loss is specifically: Among them, α, γ and β are hyperparameters. is the adaptive detail weighted loss function, is the perceptual loss function, is the gradient difference loss function.
7. The underwater image enhancement method of synchronous feature fusion according to claim 6, characterized in that: The adaptive detail weighted loss function as follows: The perceptual loss function as follows: The gradient difference loss function as follows: in, and i are the enhanced image and the real ground image, α i is the loss value of the i-th pixel, and ε is a constant of e -6 ; N is the number of scales at which the image is decomposed, represents the SSIM value at each scale, β i are weight parameters of different scales; and Represent the gradient of the enhanced image and the reference image at each pixel respectively.
8. A synchronous feature fusion underwater image enhancement system, characterized in that: include: Parameter acquisition module, used to obtain open datasets of underwater real scenes and design adaptive detail weighted loss; Model building module, used to build an underwater image enhancement network model, and train the underwater image enhancement network model based on an open dataset of underwater real scenes and adaptive detail weighted loss; The data processing module is used to input the distorted image into the trained underwater image enhancement network model, output the enhanced underwater image, and realize underwater image enhancement.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the underwater image enhancement method with synchronous feature fusion as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the underwater image enhancement method with synchronous feature fusion as claimed in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Adversarial hyperspectral and multispectral remote sensing fusion method
CN116468645A
Underwater image enhancement method based on improved Swinin-Transform
CN117876226A
Cited By
Water body image enhancement method for dynamic monitoring of fishery environment
CN120672625A
