Underwater image enhancement method based on multi-color space and channel frequency division attention
By using multi-color space and channel frequency division attention method in underwater image enhancement, combined with lightweight edge detection and multi-scale feature frequency division attention module, the lack of robustness of underwater image enhancement in the prior art is solved. The problem has achieved high-quality image enhancement effect and provided stronger technical support for marine resource development.
Patent Information
- Application Number
- CN202411132151.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-26
AI Technical Summary
Existing underwater image enhancement algorithms are less robust in complex underwater environments, resulting in artifacts and blurred details in the processed images, affecting subsequent analysis and observation.
The underwater image enhancement method based on multi-color space and channel frequency division attention is adopted, and edge detection is performed through lightweight fusion edge detection module, and the original image and edge image are input into the multi-scale feature frequency division attention module for image enhancement, using Attention mechanism optimizes edge details and reconstructs high-quality underwater images.
It effectively improves the clarity and recognizability of underwater images, adaptively adjusts image color shift, optimizes edge details, generates realistic color high-quality images, and provides more powerful technical support for the development and utilization of marine resources.
Smart Images

Figure CN119027355A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an underwater image enhancement method based on multi-color space and channel frequency division attention. Background Art
[0002] As human society's demand for resources grows, land resources are gradually depleted, and the development of marine resources has become an inevitable trend. Underwater images play a vital role in ocean exploration. However, the absorption and scattering of light by water bodies leads to the degradation of underwater image quality, color distortion, blurring and other problems, which seriously affect subsequent analysis and application. Therefore, it is imperative to develop effective underwater image enhancement algorithms, solve the nonlinear distortion problem in underwater images, improve image clarity and recognizability, and obtain high-quality underwater images.
[0003] At present, traditional underwater image enhancement algorithms are mainly divided into two categories: those based on physical models and those based on non-physical models. The physical model-based method corrects the image by simulating the propagation of light in water, but has poor adaptability to complex water environments. The non-physical model-based method directly adjusts the image pixel value through mathematical transformation or filter, but has low robustness in the changeable underwater environment. In recent years, the deep learning-based method uses data-driven characteristics to learn image enhancement mapping relationships through a large amount of data. It has stronger adaptability and robustness to complex and dynamic underwater environments and has become a research hotspot.
[0004] Although both traditional methods and deep learning methods have made some progress, the processed images may still have problems such as artifacts and blurred details, which affect subsequent image analysis and human observation. Therefore, developing more advanced algorithms to further improve the underwater image enhancement effect, solve the shortcomings of existing methods, and provide more powerful technical support for the development and utilization of marine resources will become the focus of future research. Summary of the invention
[0005] The purpose of this application is to provide an underwater image enhancement method based on multi-color space and channel frequency division attention, which can improve the underwater image enhancement effect.
[0006] To achieve the above objectives, this application provides the following solutions:
[0007] An underwater image enhancement method based on multi-color space and channel frequency division attention, comprising:
[0008] Get underwater original images;
[0009] Inputting the underwater original image into a lightweight fusion edge detection module to perform edge detection to obtain an edge image;
[0010] The underwater original image and the edge image are input into a multi-scale feature frequency division attention module for image enhancement to obtain a clear underwater image.
[0011] Optionally, the lightweight fusion edge detection module includes: a differential network, a global detection network and a fusion module;
[0012] The differential network is used to perform four differential operations on the original image to obtain four original feature maps of different scales;
[0013] The global detection network is used to perform global feature extraction on the four original feature maps of different scales to obtain edge feature maps of four different scales;
[0014] The fusion module is used to fuse the edge feature maps of the four different scales and the underwater original image to obtain an edge image.
[0015] Optionally, the differential network includes: a three-way differential module and a downsampling; the differential module includes a 3×3 pixel differential convolution, an activation function and a 1×1 convolution connected in sequence.
[0016] Optionally, the global detection network includes: a hole module; the hole module includes: a hole convolution layer, an activation function and a 1×1 convolution connected in sequence; the hole convolution layer includes: three hole convolutions with expansion rates of 3, 5 and 7 respectively.
[0017] Optionally, the multi-scale feature frequency division attention module includes: a feature extraction network, a feature encoder, a channel attention module, a feature decoder and a feature reconstruction network;
[0018] The feature extraction network is used to perform channel connection and multi-scale feature extraction on the underwater original image and the edge image to obtain four fused feature maps of different scales;
[0019] The feature encoder is used to perform full relation linear encoding on the four fused feature maps of different scales to obtain four original feature sequences;
[0020] The channel attention module is used to perform feature frequency division attention adjustment on the four original feature sequences to obtain four output feature sequences;
[0021] The feature decoder is used to perform feature remapping on the four output feature sequences to obtain four scaled feature maps;
[0022] The feature reconstruction network is used to fuse the four scaled feature maps from low to high scales to obtain four feature total maps of different scales, and fuse the feature total map of the highest scale with the underwater original image to obtain a clear underwater image.
[0023] Optionally, the feature extraction network includes: a scale-one convolution layer, a scale-two convolution layer, a scale-three convolution layer, a scale-four convolution layer and a channel connection layer;
[0024] The channel connection layer is used to perform channel connection on the underwater original image and the edge image to obtain an original fusion feature map;
[0025] The scale-one convolution layer is used to perform two convolutions on the original fused feature map to obtain a scale-one fused feature map;
[0026] The scale-2 convolution layer is used to perform two convolutions and one downsampling on the fused feature map of scale-1 to obtain a fused feature map of scale-2;
[0027] The scale-three convolutional layer is used to perform two convolutions and one downsampling on the scale-two fused feature map to obtain a scale-three fused feature map;
[0028] The scale-four convolutional layer is used to perform two convolutions and one downsampling on the scale-three fused feature map to obtain a scale-four fused feature map.
[0029] Optionally, the channel attention module uses instance normalization to calculate the attention output of the subspace, splices the attention outputs of all subspaces together and performs linear transformation to obtain multi-head attention output, and uses a multi-layer perceptron and layer normalization to generate four output feature sequences based on the multi-head attention output.
[0030] Optionally, the feature encoder uses four fully connected layers to encode the position information of the four fused feature maps of different scales respectively, so as to obtain four original feature sequences; the feature decoder uses four groups of upsampling, convolutional layers, normalization layers and activation functions to perform feature remapping on the four output feature sequences respectively, so as to obtain four scaled feature maps.
[0031] Optionally, the feature reconstruction network includes: a scale one fusion layer, a scale two fusion layer, a scale three fusion layer, a scale four fusion layer and a final fusion layer;
[0032] The scale-four fusion layer is used to process the scale-four scaled feature map to obtain a scale-four feature map;
[0033] The scale-three fusion layer is used to upsample the feature map of scale-four and fuse it with the scaled feature map of scale-three and then perform two convolutions to obtain the feature map of scale-three;
[0034] The scale-2 fusion layer is used to upsample the feature map of scale-3 and fuse it with the scaled feature map of scale-2, and then perform two convolutions to obtain the feature map of scale-2;
[0035] The scale-one fusion layer is used to upsample the feature map of scale-two and fuse it with the scaled feature map of scale-one and then perform two convolutions to obtain the feature map of scale-one;
[0036] The final fusion layer is used to fuse the feature map of scale one with the underwater original image to obtain a clear underwater image.
[0037] Optionally, the underwater image enhancement method based on multi-color space and channel frequency division attention further includes: displaying the underwater clear image.
[0038] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0039] The present application provides an underwater image enhancement method based on multi-color space and channel frequency division attention. By inputting the underwater original image into a lightweight fusion edge detection module for edge detection, the expression ability of edge features can be effectively enhanced, and image information can be captured with a larger receptive field to obtain a high-quality, noise-free edge image; by inputting the underwater original image and the edge image into a multi-scale feature frequency division attention module for image enhancement, the image color deviation can be adaptively adjusted, and the edge details can be optimized using the attention mechanism to reconstruct a high-quality underwater image with realistic colors and clear details. Therefore, the underwater image enhancement method based on multi-color space and channel frequency division attention provided by the present application can improve the underwater image enhancement effect and provide more powerful technical support for the development and utilization of marine resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 A flow chart of the underwater image enhancement method based on multi-color space and channel frequency division attention provided in this application;
[0042] Figure 2 The pixel differential convolution calculation flow chart provided for this application;
[0043] Figure 3 The calculation flow chart of the dilated convolution provided for this application;
[0044] Figure 4 Block diagram of the lightweight fusion edge detection module provided for this application;
[0045] Figure 5 Block diagram of the multi-scale feature frequency division attention module provided for this application;
[0046] Figure 6 Workflow diagram of the underwater image enhancement system based on multi-color space and channel frequency division attention provided in this application. DETAILED DESCRIPTION
[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0048] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0049] In an exemplary embodiment, the present application provides an underwater image enhancement method based on multi-color space and channel frequency division attention, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, Figure 1 As shown, the method includes the following steps 1 to 4.
[0050] Step 1: Obtain the original underwater image.
[0051] Step 2: Input the underwater original image into the lightweight fusion edge detection module to perform edge detection to obtain an edge image.
[0052] like Figure 4 As shown, the lightweight fusion edge detection module includes: a differential network, a global detection network and a fusion module; the differential network is used to perform four differential operations on the original image to obtain four original feature maps of different scales; the global detection network is used to perform global feature extraction on the four original feature maps of different scales to obtain four edge feature maps of different scales; the fusion module is used to fuse the edge feature maps of four different scales with the underwater original image to obtain an edge image.
[0053] The differential network includes: a three-time differential module and a downsampling; the differential module includes a 3×3 pixel differential convolution, an activation function and a 1×1 convolution connected in sequence. The global detection network includes: a hole module; the hole module includes: a hole convolution layer, an activation function and a 1×1 convolution connected in sequence; the hole convolution layer includes: three hole convolutions with expansion rates of 3, 5 and 7 respectively.
[0054] Traditional edge detection algorithms such as the Sobel operator, Roberts operator, Laplacian operator, Marr-Hildreth algorithm, etc., are prone to amplify noise and are extremely sensitive to noise, which can easily cause blurred details and broken edges. This application applies deep learning to edge detection and lightweights it, improving accuracy while increasing computational efficiency. This application replaces the traditional convolution (Formula 1.1) with pixel differential convolution (Formula 1.2) and hole convolution (Formula 1.3). Pixel differential convolution combines the advantages of traditional edge operators and deep learning methods, improves traditional convolution, and does not directly perform convolution operations on the original pixel values. Instead, it first calculates the difference between the pixels in the k×k matrix, and then performs element-wise multiplication with the weight of the convolution kernel to generate output features. For the pixel differential convolution process, see Figure 2 The dilated convolution is a convolution operation that can increase the receptive field. By introducing holes (also called the dilation rate d) between the convolution kernels, the effective receptive field of the convolution kernel can be expanded, thereby improving the model's ability to capture details such as edges. For the dilated convolution process with a dilation rate of 2, see Figure 3 . Below are the formulas for the three types of convolution:
[0055]
[0056]
[0057] Among them, x l and x' l are all input pixels, (x l ,x' l ) represents the difference calculation, w l is the weight of the k×k convolution kernel, θ represents the parameters (weight and bias) of the convolution kernel, represents a type of differential convolution, x[i,j] represents the pixel value of the input feature map at position (i,j), X[ir m ,jr n ] represents the pixel index corresponding to the convolution kernel position (m, n) on the input feature map, and w[m, n] represents the weight corresponding to the convolution kernel position (m, n) on the input feature map. 1 ,x' 1 ),(x 2 ,x' 2),…,(x n ,x' n )} is a set of pixel pairs selected from the current local patch, and n≤k×k.
[0058] Such pixel differential convolution can explicitly encode gradient information and combine the feature extraction capabilities of convolution to capture subtle changes and details in the image, more effectively capture the gradient information of the image, and enhance the network's sensitivity to edge features. However, differential convolution mainly focuses on local pixel differences and has difficulty capturing high-level information of the global image. It also has a small receptive field and is easily affected by image noise, which can detect many edges. Therefore, a dilated convolution is added to expand its receptive field, capture long-distance dependent information, and improve its anti-disturbance ability.
[0059] The lightweight fusion edge detection module used in this application uses the differential module and the hole module to simplify and lightweight the entire module, thereby achieving accurate extraction of image edge information. The specific edge detection steps are as follows.
[0060] In order to capture edge information of different scales, the underwater original image (I∈R H*W*3 , where H and W represent the height and width of the image respectively) after four differential operations, each differential operation includes three differential modules and one downsampling, and finally four original feature maps of different scales are generated i≤3; i=4, where C represents the number of channels after differential convolution). The differential module uses a 3×3 pixel differential convolution kernel, combined with an activation function and a 1×1 convolution, which can effectively enhance the expression of edge features.
[0061] Due to the local characteristics of differential convolution, a hole module is added after the difference in each stage to expand its receptive field, so that it can capture image information with a larger receptive field. However, the hole convolution introduces holes between the convolution kernels, which can easily cause the model to over-focus on certain areas and ignore other areas, and even cause the gradient vanishing problem. In order to solve this problem, the hole module of this application is fused with three hole convolutions with different expansion rates (d=3, 5, 7), and the above problems are effectively alleviated by "hole interlacing". In addition, the activation function and 1×1 convolution are combined to effectively extract the global feature information in the original feature map and remove noise interference, generating four edge feature maps of different scales (D i ∈R H*W*N , i = 1, 2, 3, 4, where N represents the number of channels after the dilated convolution).
[0062] The four edge feature maps of different scales contain different levels of detail information. In order to obtain the final accurate and complete edge image, these four edge feature maps are sent to the fusion module. The fusion module interweaves information of different scales: it not only retains the accurate grasp of the overall contour of the large-scale feature map, but also integrates the retention of detailed edge information of the small-scale feature map. Finally, the fusion module outputs a high-quality, noise-free edge image (B∈R H*W*3 ), clearly showing the edge information of the image.
[0063] Step 3: Input the underwater original image and the edge image into the multi-scale feature frequency division attention module for image enhancement to obtain a clear underwater image.
[0064] like Figure 5 As shown, the multi-scale feature frequency division attention module includes: a feature extraction network, a feature encoder, a channel attention module, a feature decoder and a feature reconstruction network. The feature extraction network is used to perform channel connection and multi-scale feature extraction on the underwater original image and the edge image to obtain four fused feature maps of different scales. The feature encoder is used to perform full-relation linear encoding on the four fused feature maps of different scales to obtain four original feature sequences. The channel attention module is used to perform feature frequency division attention adjustment on the four original feature sequences to obtain four output feature sequences. The feature decoder is used to perform feature remapping on the four output feature sequences to obtain four scaled feature maps. The feature reconstruction network is used to fuse the four scaled feature maps from low to high scales to obtain four feature total maps of different scales, and fuse the highest scale feature total map with the underwater original image to obtain a clear underwater image.
[0065] The feature extraction network includes: a scale-one convolution layer, a scale-two convolution layer, a scale-three convolution layer, a scale-four convolution layer and a channel connection layer; the channel connection layer is used to perform channel connection on the underwater original image and the edge image to obtain an original fused feature map; the scale-one convolution layer is used to perform two convolutions on the original fused feature map to obtain a fused feature map of scale one; the scale-two convolution layer is used to perform two convolutions and one downsampling on the fused feature map of scale one to obtain a fused feature map of scale two; the scale-three convolution layer is used to perform two convolutions and one downsampling on the fused feature map of scale two to obtain a fused feature map of scale three; the scale-four convolution layer is used to perform two convolutions and one downsampling on the fused feature map of scale three to obtain a fused feature map of scale four.
[0066] Specifically, the original image I∈R H*W*3 and edge image B∈R H*W*3 After channel connection, we get an original fusion feature map D∈RH*W*6 , after two convolutions, we get the fusion feature map of scale one (E 1 ∈R H*W*M , where M represents the number of channels of the original image after feature extraction), and then after two convolutions and one downsampling, a low-scale fusion feature map is obtained. i=2, 3, 4).
[0067] The feature encoder uses four fully connected layers to encode the position information of the four fused feature maps of different scales respectively to obtain four original feature sequences.
[0068] Specifically, after obtaining the fusion feature maps of each scale, since its position information is difficult to combine, a full relation linear projection is designed, and its position information is encoded using full connection, its channel remains unchanged, and the image is straightened to obtain four original feature sequences
[0069] The channel attention module uses instance normalization to calculate the attention output of the subspace, splices the attention outputs of all subspaces together and performs linear transformation to obtain multi-head attention output, and uses a multi-layer perceptron and layer normalization to generate four output feature sequences based on the multi-head attention output.
[0070] Specifically, the query vector of the attention mechanism is obtained according to the four original feature sequences:
[0071] K=SW K V=SW V (1.4)
[0072] in, K∈R e*N , V∈R N*C ,and S∈R C*e By S i Obtained by connecting through the channel dimension, N=M+2M+4M+8M.
[0073] The query vector is input into the channel attention module, and the output of the attention is obtained according to formula (1.4):
[0074]
[0075] Among them, IN is instance normalization and SoftMax represents the activation function.
[0076] This channel attention is performed by encoding the channel axis in advance through the full relation, which can guide the network to pay more attention to the pre-given edge high-frequency parts and channels with more severe attenuation. Instance normalization is to normalize the entire batch of images, which is different from batch normalization. It can make the gradient propagation smooth and simplify the learning process.
[0077] Channel attention is composed of the original Q i The , K and V matrices are linearly mapped into four different subspaces, and scaled dot product attention calculations are performed independently in each subspace. Finally, the attention outputs of all subspaces are concatenated together and linearly transformed again to obtain the final multi-head attention output:
[0078] FDMHA i =(CA i 1 +CA i 2 +.........,+CA i N ) / N+Q i (1.6)
[0079] The four output feature sequences of the channel attention module can be obtained by formula 1.7:
[0080] O i =FDMHA i +MLP(LN(FDMHA i ))(1.7)
[0081] Among them, MLP is a multi-layer perceptron and LN is layer normalization.
[0082] The feature decoder uses four groups of upsampling, convolutional layers, normalization layers and activation functions to perform feature remapping on the four output feature sequences respectively to obtain four scaled feature maps.
[0083] Specifically, the output feature sequence After upsampling, convolution layer, normalization layer and activation function, the four output feature sequences are scaled to a part of the current scale feature map to obtain four scaled feature maps. i=2, 3, 4).
[0084] The feature reconstruction network includes: a scale-one fusion layer, a scale-two fusion layer, a scale-three fusion layer, a scale-four fusion layer and a final fusion layer; the scale-four fusion layer is used to process the scaled feature map of scale four to obtain the feature map of scale four; the scale-three fusion layer is used to upsample the feature map of scale four and fuse it with the scaled feature map of scale three and then perform two convolutions to obtain the feature map of scale three; the scale-two fusion layer is used to upsample the feature map of scale three and fuse it with the scaled feature map of scale two and then perform two convolutions to obtain the feature map of scale two; the scale-one fusion layer is used to upsample the feature map of scale two and fuse it with the scaled feature map of scale one and then perform two convolutions to obtain the feature map of scale one; the final fusion layer is used to fuse the feature map of scale one with the underwater original image to obtain a clear underwater image.
[0085] Specifically, the feature map of the current scale is the feature map of the previous scale Upsampled and scaled feature map with current scale After fusion, it is obtained by two convolutions. Finally, the feature map of scale one (G 1 ) is fused with the original underwater image (I) to compensate for the loss of information and alleviate the gradient vanishing problem. After two convolutions and an activation function, the final clear underwater image is output.
[0086] Step 4: Display the underwater clear image.
[0087] In this embodiment, the lightweight fusion edge detection module and the multi-scale feature frequency division attention module are trained together as an underwater image enhancement model, and the loss function L used in the training is total Including RGB loss, LAB loss, HSV loss, SSIM loss, UIQM loss. The first three losses can be calculated according to different color spaces to more effectively restore the color problem of the image, especially the red channel with the most severe attenuation. SSIM loss is to prevent the image from losing its original structural information and visual sense during the training process. UIQM loss focuses on enhancing the color, clarity and contrast, making the enhanced image more colorful, with clearer details, and significantly improving the overall visual effect.
[0088]
[0089] Among them, α, β, γ, δ, μ, and θ are all hyperparameters, which are set to 0.1, 1, 0.01, 1, 0.1, and 100 respectively. The hyperparameters can be in the same order of magnitude after multiplying with each loss value. In addition, the conversion relationship between RGB and LAB color space is The conversion relationship between RGB and HSV color space is:
[0090] The model is trained using EUVP's scenes dataset and dark dataset plus the UIEBD training dataset and data augmentation, and tested on the EUVP test dataset. It is compared with other algorithms and proves that this application has achieved good results.
[0091] In an exemplary embodiment, due to the complex underwater environment, the underwater images collected by the camera sensor have problems such as color bias and blur. In order to improve the underwater image enhancement effect, the present application also provides an underwater image enhancement system based on multi-color space and channel frequency division attention. The system is composed of a camera, a microcomputer and a display screen. The system uses a camera to collect underwater environment images, inputs the collected underwater images into a microcomputer, and is processed by a processor of the microcomputer. The processor runs an image enhancement algorithm to enhance the image, and the enhanced image is transmitted to the display screen for real-time display. The image enhancement algorithm consists of a lightweight fusion edge detection module and a multi-scale feature frequency division attention module. A hybrid lightweight edge detection network is used to extract the edge information of objects in the underwater original image. Subsequently, the underwater original image and the edge image generated by preprocessing are input into the multi-scale feature frequency division attention network to output an enhanced underwater clear image.
[0092] like Figure 6 As shown, the specific workflow of the underwater image enhancement system based on multi-color space and channel frequency division attention provided by the present application includes the following steps 1) to 4).
[0093] 1) Obtain the original underwater image through the camera.
[0094] 2) Input the underwater original image into the lightweight fusion edge detection module and output the edge image.
[0095] 3) The underwater original image and the edge image are simultaneously input into the multi-scale feature frequency division attention module. Specifically: first, the underwater original image and the edge image are superimposed to extract their features, and their features are encoded to obtain ordered feature information. These features are then input into the feature frequency division attention module (i.e., channel attention module) to adaptively adjust the image color deviation, and use the attention mechanism to optimize the edge details. Finally, its output is feature decoded and fused with the underwater original image for feature reconstruction, outputting high-quality underwater images with realistic colors and clear details.
[0096] 4) The output underwater clear image is transmitted to the display screen for display.
[0097] In an exemplary embodiment, the present application further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0098] In an exemplary embodiment, the present application further provides a computer-readable storage medium storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.
[0099] In an exemplary embodiment, the present application further provides a computer program product, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.
[0100] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant legal provisions.
[0101] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0102] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0103] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0104] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. An underwater image enhancement method based on multi-color space and channel frequency division attention, It is characterized in that include: Get underwater original images; Inputting the underwater original image into a lightweight fusion edge detection module to perform edge detection to obtain an edge image; The underwater original image and the edge image are input into a multi-scale feature frequency division attention module for image enhancement to obtain a clear underwater image.
2. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 1, It is characterized in that The lightweight fusion edge detection module includes: a differential network, a global detection network and a fusion module; The differential network is used to perform four differential operations on the original image to obtain four original feature maps of different scales; The global detection network is used to perform global feature extraction on the four original feature maps of different scales to obtain edge feature maps of four different scales; The fusion module is used to fuse the edge feature maps of the four different scales and the underwater original image to obtain an edge image.
3. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 2, It is characterized in that The differential network includes: a three-dimensional differential module and a downsampling; the differential module includes a 3×3 pixel differential convolution, an activation function and a 1×1 convolution connected in sequence.
4. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 2, It is characterized in that The global detection network includes: a hole module; the hole module includes: a hole convolution layer, an activation function and a 1×1 convolution connected in sequence; the hole convolution layer includes: three hole convolutions with expansion rates of 3, 5 and 7 respectively.
5. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 1, It is characterized in that The multi-scale feature frequency division attention module includes: a feature extraction network, a feature encoder, a channel attention module, a feature decoder and a feature reconstruction network; The feature extraction network is used to perform channel connection and multi-scale feature extraction on the underwater original image and the edge image to obtain four fused feature maps of different scales; The feature encoder is used to perform full relation linear encoding on the four fused feature maps of different scales to obtain four original feature sequences; The channel attention module is used to perform feature frequency division attention adjustment on the four original feature sequences to obtain four output feature sequences; The feature decoder is used to perform feature remapping on the four output feature sequences to obtain four scaled feature maps; The feature reconstruction network is used to fuse the four scaled feature maps from low to high scales to obtain four feature total maps of different scales, and fuse the feature total map of the highest scale with the underwater original image to obtain a clear underwater image.
6. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 5, It is characterized in that The feature extraction network includes: a scale one convolution layer, a scale two convolution layer, a scale three convolution layer, a scale four convolution layer and a channel connection layer; The channel connection layer is used to perform channel connection on the underwater original image and the edge image to obtain an original fusion feature map; The scale-one convolution layer is used to perform two convolutions on the original fused feature map to obtain a scale-one fused feature map; The scale-2 convolution layer is used to perform two convolutions and one downsampling on the fused feature map of scale-1 to obtain a fused feature map of scale-2; The scale-three convolutional layer is used to perform two convolutions and one downsampling on the scale-two fused feature map to obtain a scale-three fused feature map; The scale-four convolutional layer is used to perform two convolutions and one downsampling on the scale-three fused feature map to obtain a scale-four fused feature map.
7. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 5, It is characterized in that The channel attention module uses instance normalization to calculate the attention output of the subspace, splices the attention outputs of all subspaces together and performs linear transformation to obtain multi-head attention output, and uses a multi-layer perceptron and layer normalization to generate four output feature sequences based on the multi-head attention output.
8. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 5, It is characterized in that The feature encoder uses four fully connected layers to encode the position information of the four fused feature maps of different scales respectively to obtain four original feature sequences; The feature decoder uses four groups of upsampling, convolutional layers, normalization layers and activation functions to perform feature remapping on the four output feature sequences respectively to obtain four scaled feature maps.
9. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 5, It is characterized in that The feature reconstruction network includes: a scale one fusion layer, a scale two fusion layer, a scale three fusion layer, a scale four fusion layer and a final fusion layer; The scale-four fusion layer is used to process the scale-four scaled feature map to obtain a scale-four feature map; The scale-three fusion layer is used to upsample the feature map of scale-four and fuse it with the scaled feature map of scale-three and then perform two convolutions to obtain the feature map of scale-three; The scale-2 fusion layer is used to upsample the feature map of scale-3 and fuse it with the scaled feature map of scale-2, and then perform two convolutions to obtain the feature map of scale-2; The scale-one fusion layer is used to upsample the feature map of scale-two and fuse it with the scaled feature map of scale-one and then perform two convolutions to obtain the feature map of scale-one; The final fusion layer is used to fuse the feature map of scale one with the underwater original image to obtain a clear underwater image.
10. The underwater image enhancement method based on multi-color space and channel frequency division attention according to claim 1, It is characterized in that Also includes: The underwater clear image is displayed.
