A dynamic underwater image enhancement method and device based on prior knowledge

Abstract

The application discloses a kind of dynamic underwater image enhancement method and device based on prior knowledge, image enhancement is carried out to underwater image based on the image enhancement network model constructed, the image enhancement network model includes dynamic network module based on atmospheric light, prior-based multiscale network module and transmission rate guided dynamic network module. Prior information is obtained by dynamic network module based on atmospheric light and transmission rate guided dynamic network module, in prior-based multiscale network module, sequentially through first multiscale U type network, transmission rate guided feature extraction network and second multiscale U type network, and enhanced image is obtained. The application can dynamically adjust the parameters of the network according to the input image, so that the method can be applied to real underwater scenes, can adaptively select appropriate parameters for various water types, and has better generalization ability and robustness.

Description

Technical Field

[0001] This application belongs to the field of underwater scene artificial intelligence application, and in particular relates to a dynamic underwater image enhancement method and device based on prior knowledge. Background Technology

[0002] Underwater image processing has broad application prospects, such as in underwater robotics, marine exploration, and submarine cable inspection. However, because different wavelengths of light travel at different speeds underwater, and green and blue light have the shortest wavelengths, underwater photos are often biased towards blue, green, or yellow, which masks the original colors of marine plants and animals. Furthermore, the scattering and refraction of light cause underwater images to suffer from low saturation and blurriness.

[0003] Current underwater image enhancement methods can be broadly categorized into two types: traditional underwater image enhancement methods and deep learning-based underwater image enhancement methods. Traditional underwater image enhancement methods include prior-based and model-free methods. Prior-based methods utilize rich prior information and estimate the parameters of the underwater image imaging model to generate enhanced images. While these methods fully leverage prior information, they are not very practical in complex scenes and often lead to over-enhancement, thus making them unsuitable for real-world underwater scenarios. Model-free methods typically rely on the spatial relationships between pixel values ​​in the original underwater image to improve brightness, contrast, and saturation, such as gray world and white balance methods. However, these methods often neglect image details and depth information, leading to artifacts and over-enhancement.

[0004] Deep learning has introduced novel strategies for underwater image enhancement tasks. These methods can achieve complex, non-systematic end-to-end modeling or combine physical priors with neural networks to address existing problems. These methods possess excellent feature representation capabilities thanks to the large amounts of data they utilize for training. However, a major challenge is the high cost and difficulty of acquiring large-scale underwater datasets, forcing most methods to use small-scale datasets.

[0005] In summary, the challenges in underwater image enhancement are as follows: 1. Due to the scattering and absorption of light by water, underwater images often suffer from color distortion and low contrast. Furthermore, water quality and light transmission distance also affect image clarity, making underwater images blurry. 2. The scarcity of high-quality paired training samples limits the performance of deep learning models. 3. Model generalization is a crucial task in underwater image enhancement; it is important but often overlooked. Summary of the Invention

[0006] The purpose of this application is to provide a dynamic underwater image enhancement method and apparatus based on prior knowledge to solve the above-mentioned technical problems existing in the prior art. It can adapt to underwater images in various scenarios, can be applied in complex scenarios, and has good robustness.

[0007] To achieve the above objectives, the technical solution of this application is as follows:

[0008] A dynamic underwater image enhancement method based on prior knowledge enhances underwater images based on a constructed image enhancement network model. The image enhancement network model includes a dynamic network module based on atmospheric light, a multi-scale network module based on prior knowledge, and a transmission rate-guided dynamic network module. The dynamic underwater image enhancement method based on prior knowledge includes:

[0009] The underwater image to be enhanced is input into the atmospheric light-based dynamic network module. In the atmospheric light-based dynamic network module, the prior information of atmospheric light is extracted through dynamic convolution and global average pooling.

[0010] The underwater image to be enhanced is input into the transmission rate-guided dynamic network module. The transmission rate-guided dynamic network module performs dynamic convolution and global average pooling to extract transmission prior information.

[0011] The underwater image to be enhanced and the corresponding transmission rate map are input into a priori-based multi-scale network module. The image is then passed through a first multi-scale U-shaped network, a transmission rate-guided feature extraction network, and a second multi-scale U-shaped network in sequence to obtain the enhanced image.

[0012] The first multi-scale U-shaped network and the second multi-scale U-shaped network obtain convolution kernel parameters by processing atmospheric light prior information, and the transmission rate-guided feature extraction network obtains convolution kernel parameters by processing transmission prior information.

[0013] Furthermore, both the atmospheric light-based dynamic network module and the transmission rate-guided dynamic network module include a first unit and a second unit, each unit including two dynamic convolutions and a global average pooling.

[0014] Furthermore, the first multi-scale U-shaped network and the second multi-scale U-shaped network have the same structure, including a feature extraction block, a convolutional network, and a feature extraction block and a convolutional network. Each layer of the first multi-scale U-shaped network and the second multi-scale U-shaped network is connected by fast connections. The feature extraction block includes a superconvolution, a multi-scale feature extraction block, and a superconvolution. The convolutional network includes three superconvolutions. The transfer rate-guided feature extraction network includes a superconvolution, a connection layer, a multi-scale feature extraction block, and a superconvolution.

[0015] Furthermore, the superconvolution includes a residual block and a fully connected layer, and the residual block is represented by the following formula:

[0016] f′ c =Conv(ReLU(Conv(f) c )+f c

[0017] Among them, f c For the input features, f′ c For output features;

[0018] In the superconvolution of the first and second multi-scale U-shaped networks, the atmospheric light prior information is used to obtain the parameters of the convolution kernel in the residual block through the fully connected layer;

[0019] In the hyperconvolution of the transmission rate-guided feature extraction network, the transmission prior information is used to obtain the parameters of the convolution kernel in the residual block through the fully connected layer.

[0020] Furthermore, the connection layer is represented by the following formula:

[0021] f″=f′·f m +f′

[0022] Where f′ represents the input features of the connection layer, and f″ represents the output features of the connection layer. m This represents the transmission rate map corresponding to the underwater image to be enhanced.

[0023] Furthermore, the multi-scale feature extraction block includes three convolutional branches at different scales and a channel attention module, and the multi-scale feature extraction block performs the following operations:

[0024] The input features f of the multi-scale feature extraction block are processed by the channel attention module and can be expressed by the following formula:

[0025] f1=Conv(ReLU(Conv(GAP(f))));

[0026] f2=Conv(ReLU(Conv(GMP(f))));

[0027] f CA =f1+f2

[0028] Among them, f CA The output features of the channel attention module are defined by GAP (Global Average Pooling) and GMP (Global Max Pooling).

[0029] The input features of the multi-scale feature extraction block are processed by three convolutional branches at different scales, and the output features of the three convolutional branches at different scales are respectively correlated with f. CAMultiply and then add to obtain the output features of the multi-scale feature extraction block.

[0030] Furthermore, the dynamic underwater image enhancement method based on prior knowledge also includes: training the image enhancement network model;

[0031] Training the image augmentation network model includes:

[0032] The network parameters of the dynamic network module based on atmospheric light and the dynamic network module guided by transmission rate are fixed, and the prior-based multi-scale network module is trained.

[0033] By fixing the network parameters of the prior-based multi-scale network module and the transmission rate-guided dynamic network module, a dynamic network module based on atmospheric light is trained.

[0034] By fixing the network parameters of the prior-based multi-scale network module and the atmospheric light-based dynamic network module, a transmission rate-guided dynamic network module is trained.

[0035] Furthermore, the joint loss function used when training the image enhancement network model is as follows:

[0036] L(J,I) gt )=L1+λ1L SSIM +λ2L per

[0037] Where λ1 and λ2 are weight parameters, L SSIM For the mean squared error loss, L per L1 is the perceptual loss, L1 is the reconstruction loss, J is the output image of the image augmentation network model, and I is the image. gt For reference only.

[0038] This application also proposes a dynamic underwater image enhancement device based on prior knowledge, including a processor and a memory storing a number of computer instructions, which, when executed by the processor, implement the steps of the above method.

[0039] This application proposes a dynamic underwater image enhancement method and apparatus based on prior knowledge. This method dynamically adjusts the network parameters according to the input image, enabling its application in real-world underwater scenarios. It fully utilizes prior knowledge of underwater imaging, combining this knowledge with a deep learning model to enhance the model's feature extraction capabilities and robustness. It can be trained on small datasets and achieves good generalization performance, overcoming the challenge of insufficient underwater image datasets. It can be applied to various underwater operational scenarios, such as underwater robots, submarine cable repair, and marine surveying. Attached Figure Description

[0040] Figure 1 This is a flowchart of the dynamic underwater image enhancement method based on prior knowledge proposed in this application.

[0041] Figure 2 This is a schematic diagram of the image enhancement network model of this application.

[0042] Figure 3 This is a schematic diagram of the dynamic network module in an embodiment of this application.

[0043] Figure 4 This is a schematic diagram of a multi-scale network module based on prior knowledge, as described in an embodiment of this application.

[0044] Figure 5 This is a schematic diagram of a multi-scale feature extraction block in an embodiment of this application.

[0045] Figure 6 This is a schematic diagram of a transmission rate-guided feature extraction network according to an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] In one embodiment, such as Figure 1 As shown, a dynamic underwater image enhancement method based on prior knowledge is proposed. Underwater images are enhanced based on a constructed image enhancement network model, which includes a dynamic network module based on atmospheric light, a multi-scale network module based on prior knowledge, and a transmission rate-guided dynamic network module. This embodiment of a dynamic underwater image enhancement method based on prior knowledge includes:

[0048] Step S1: Input the underwater image to be enhanced into the atmospheric light-based dynamic network module. In the atmospheric light-based dynamic network module, the atmospheric light prior information is extracted through dynamic convolution and global average pooling.

[0049] This embodiment constructs an image enhancement network model as follows: Figure 2 As shown, an image enhancement network model is used to enhance underwater images. The image enhancement network model includes an atmospheric light-based dynamic network module (ADS), a priori-based multi-scale network module (PMS), and a transmission rate-guided dynamic network module (TDS).

[0050] The atmospheric light-based dynamic network module adaptively extracts atmospheric light prior information from underwater images using dynamic convolution and global average pooling. This prior information is then processed to provide parameters for the convolution kernels in the multi-scale U-shaped network within the prior-based multi-scale network module. This allows the entire image enhancement network model to adjust its parameters according to the degradation level of the input image, improving the network's generalization ability.

[0051] In one specific embodiment, the dynamic network module based on atmospheric light, such as Figure 3 As shown, it mainly consists of two modules: a dynamic convolution module (denoted as DC) and a global average pooling module (denoted as GAP). Its structure includes a first unit and a second unit, each containing two dynamic convolutions and one global average pooling. The dynamic network module based on atmospheric light is represented by the following formula:

[0052]

[0053] in, This represents the input features of the dynamic network module based on atmospheric light, here representing the underwater image to be enhanced; f ADS This represents the output feature, i.e., prior information about atmospheric light.

[0054] Step S2: Input the underwater image to be enhanced into the transmission rate-guided dynamic network module. In the transmission rate-guided dynamic network module, dynamic convolution and global average pooling are performed to extract transmission prior information.

[0055] In this embodiment, the transmission rate-guided dynamic network module adaptively extracts transmission prior information from underwater images using dynamic convolution and global average pooling. This transmission prior information is then processed to provide parameters for the convolutional kernels in the transmission rate-guided feature extraction network within the prior-based multi-scale network module. The transmission rate-guided dynamic network module encourages the network to focus more on the areas of most severe quality degradation, enabling the network to adaptively select appropriate parameters based on water quality conditions.

[0056] Specifically, such as Figure 3 As shown, the transmission rate-guided dynamic network module has the same modules and structure as the atmospheric light-based dynamic network module, and its calculation formula is as follows:

[0057]

[0058] Among them, f TDS This is to output features, i.e., to transmit prior information; The input image for the transmission rate-guided dynamic network module is the underwater image to be enhanced.

[0059] Step S3: Input the underwater image to be enhanced and the corresponding transmission rate map into the prior-based multi-scale network module. In the prior-based multi-scale network module, the image is sequentially passed through the first multi-scale U-shaped network, the transmission rate-guided feature extraction network, and the second multi-scale U-shaped network to obtain the enhanced image.

[0060] This embodiment is based on a priori multi-scale network modules, such as Figure 4 As shown, the network includes a first multi-scale U-shaped network, a transmission rate-guided feature extraction network, and a second multi-scale U-shaped network. The underwater image to be enhanced is sequentially passed through the first multi-scale U-shaped network, the transmission rate-guided feature extraction network, and the second multi-scale U-shaped network to obtain the enhanced image. The first and second multi-scale U-shaped networks obtain convolution kernel parameters by processing atmospheric light prior information, while the transmission rate-guided feature extraction network obtains convolution kernel parameters by processing transmission prior information.

[0061] In one specific embodiment, the first multi-scale U-shaped network and the second multi-scale U-shaped network have the same structure, including a feature extraction block, a convolutional network, and another feature extraction block. The layers of the first and second multi-scale U-shaped networks are connected by fast connections. A fast connection refers to the sum of the input features of each layer in the decoder stage and the output features of the corresponding layer in the encoder stage and the output features of the preceding layers in the decoder stage. Fast connections can fuse low-level positional information with deep semantic information, enabling the network to have a larger receptive field while also paying attention to the detailed information of edge features.

[0062] The feature extraction blocks include hyperconvolution (HRB), multi-scale feature extraction blocks (MFE), and hyperconvolution (HRB). The convolutional network consists of three hyperconvolutions (HRB).

[0063] The hyperconvolutional HRB consists of a residual block and a fully connected layer. The formula for calculating the residual block is as follows:

[0064] f′ c =Conv(ReLU(Conv(f) c )+f c

[0065] Among them, f c For the input features, f′ c For the output features, the Conv parameters here are determined by f ADS After passing through the fully connected layer (FC), it is decided that f will... ADS The parameters of the convolution kernel are obtained by passing the kernel through a fully connected layer (FC).

[0066] For example, atmospheric light prior information f ADSThe feature size is batch_size × 128 × 1 × 1 (batch_size is the batch size). Then, the features are repeated (repeat operation), resulting in n_block × n_each_block feature vectors of size batch_size × 128 × 1 × 3 × 3 (where n_block is the number of convolutional blocks, and n_each_block is the number of convolutional kernels in each block). Then, this feature vector is subjected to an EqualLinear operation (i.e.,...). Figure 4 (FC in the model). The feature group vector is an n_block×n_each_block×batch_size×1×64×1×1 matrix. This matrix is ​​transformed by FC into batch_size out_channel×in_channel×kernel_size×kernel_size as parameters of the convolution kernel, where out_channel represents the output dimension of the convolution, in_channel represents the input dimension of the convolution, and kernel_size represents the size of the convolution kernel.

[0067] Among them, multi-scale feature extraction blocks (MFE) are as follows: Figure 5 As shown, it includes three convolutional branches of different scales (i.e., 1×1, 3×3, and 5×5) and a channel attention module. The channel attention module is used to generate the weights corresponding to the convolutions of different scales.

[0068] The channel attention module is calculated as follows:

[0069] f1=Conv(ReLU(Conv(GAP(f))));

[0070] f2=Conv(ReLU(Conv(GMP(f))));

[0071] f CA =f1+f2

[0072] Among them, f CA The output features of the channel attention module are represented by GAP (Global Average Pooling), GMP (Global Max Pooling), and Conv (Convolution), where the kernel size is 1×1. ReLU is the activation function.

[0073] Furthermore, the input features of the multi-scale feature extraction block are processed through three convolutional branches of different scales. Each convolutional branch contains Conv, ReLU, and Conv, with different kernel sizes for the Conv convolutions: 1×1, 3×3, and 5×5. Finally, the output features of the three convolutional branches of different scales are respectively compared with f. CA Multiply and then add to obtain the output features of the multi-scale feature extraction block.

[0074] In one specific embodiment, a transmission rate-guided feature extraction network such as Figure 6 As shown, it includes: hyperconvolution HRB, connection layer TC, multi-scale feature extraction block MFE, and hyperconvolution HRB.

[0075] The calculation formula for the transmission rate-guided feature extraction network is as follows:

[0076] f′=HRB(f);

[0077] f″=f′·f m +f′;

[0078] f″′=HRB(MFE(f″);

[0079] Where f′ represents the input feature of the connection layer, i.e., the output feature of the first hyperconvolution HRB; f″ represents the output feature of the connection layer, f m The image represents the transmissivity map corresponding to the underwater image to be enhanced, and f″′ represents the final output feature of the transmissivity-guided feature extraction network.

[0080] Unlike multi-scale U-shaped networks, the parameters of the two superconvolutional kernels in the transmission rate-guided feature extraction network are determined by the transmission prior information f. TDS The Conv parameter of the hyperconvolutional HRB is determined by f. TDS It is obtained through a fully connected layer in the hyperconvolutional HRB.

[0081] It should be noted that the transmittance map f of the underwater image to be enhanced m Using the underwater dark channel prior (UDCP) algorithm, the transmission rate map can be directly calculated from the input image, which will not be elaborated here.

[0082] This embodiment's technical solution can adaptively generate network parameters based on the water feature characteristics of the input image, enabling the structure to be applied to real underwater scenes and enhancing underwater images with different water feature characteristics. These different water feature characteristics refer to different types of atmospheric light and different types of transmission rates.

[0083] In one specific embodiment, in order to train the image enhancement network model, a joint loss function is constructed to guide the model to produce enhancement results with minimal color artifacts and blur, and to make the details as close as possible to the reference image.

[0084] The joint loss function is:

[0085] L(J,I) gt )=L1+λ1L SSIM +λ2L per

[0086] Where λ1 and λ2 are weight parameters, L SSIM For the mean squared error loss, L per L1 is the perceptual loss, and L1 is the reconstruction loss. J is the output image of the image augmentation network model, i.e. Figure 2 f in PMS I gt This is a reference image, or a real image.

[0087] Among them, the perceived loss L per :

[0088] L per (J, I) gt ) = L MSE (VGG 3，8，15 (J), VGG 3，8，15 (I gt ))

[0089] Among them, L MSE VGG represents the mean squared error (MSE). 3，8，15 This indicates the features extracted from layers 3, 8, and 15 using a pre-trained VGG16 network.

[0090] Reconstruction loss L1:

[0091]

[0092] Where n = H × W is the total number of pixels, and the reconstruction loss L1 is used to eliminate unnecessary interference in the background color.

[0093] Mean squared error loss L SSIM :

[0094]

[0095] Where μ J , It is the average value of all pixels in the image patch, σ J , C1 and C2 are constants, where C1 represents the variance of the image pixel values. This application uses SSIM loss to focus more on structural details. The subscript J represents the output image of the image enhancement network model, and I represents the variance of the image pixel values. gt For reference only.

[0096] The prior-based multi-scale network module described in this application benefits from an atmospheric light-based dynamic network module and a transmission rate-guided dynamic network module. It can adaptively generate network parameters based on the water feature characteristics of the input image, enabling the structure to be applied to real underwater scenes and enhancing underwater images with different water feature characteristics. These different water feature characteristics refer to different types of atmospheric light and different types of transmission rates.

[0097] To enhance underwater images using an image augmentation network model and improve its generalization performance and robustness, this application designs a hierarchical training method and a hyperparameter optimization scheme to optimize the hyperparameters of the dynamic network module based on atmospheric light and the dynamic network module guided by transmission rate.

[0098] In one specific embodiment, training the image enhancement network model includes:

[0099] The network parameters of the dynamic network module based on atmospheric light and the dynamic network module guided by transmission rate are fixed, and the prior-based multi-scale network module is trained.

[0100] By fixing the network parameters of the prior-based multi-scale network module and the transmission rate-guided dynamic network module, a dynamic network module based on atmospheric light is trained.

[0101] By fixing the network parameters of the prior-based multi-scale network module and the atmospheric light-based dynamic network module, a transmission rate-guided dynamic network module is trained.

[0102] This embodiment adopts a hierarchical training approach, dividing the training samples into multiple batches. One batch is used to train a multi-scale network module based on priors, then another batch is used to train a dynamic network module based on atmospheric light, and then another batch is used to train a transmission rate-guided dynamic network module. This process is repeated until the training termination condition is met.

[0103] The training of the prior-based multi-scale network module is represented as follows:

[0104]

[0105] in, This represents a priori-based multi-scale network module, where ω represents the parameters of the priori-based multi-scale network module. This represents a dynamic network module based on atmospheric light, where θ is its parameter, and A c This represents the estimated atmospheric light. This indicates a rate-driven dynamic network module, where φ is its parameter, and T c This is a graph showing the estimated transmission rate. c (x) is the input image of the network. This is the enhanced image.

[0106] Calculate all losses:

[0107]

[0108] Where L represents the joint loss function, I gt Indicates a reference image. I represents the image generated by the network. c (x) represents the input image.

[0109] Then update the prior-based multi-scale network module. The parameter ω:

[0110]

[0111] in, ρ is the gradient of the joint loss, and ρ0 is the update step size.

[0112] Training a dynamic network module based on atmospheric light is represented as follows:

[0113]

[0114] in, For the enhanced image, This refers to altered atmospheric light. This indicates the parameters used to generate the main network using the ADS structure, where θ represents the parameters of the ADS branches. This indicates the parameters used to generate the main network using the TDS structure, where φ represents the parameters of the TDS branches, and T... c This represents the original transmission rate graph, where ω is a parameter of the main network. Represent the main network. Then calculate the joint loss.

[0115]

[0116] Where L is the total loss function, For the network output, I gt For reference image, To modify the output image after the atmospheric light image, I c (x) represents the original input image. Since this method generates a set of underwater images with the same content but different water type characteristics by changing the atmospheric light, and the reference image for this set of images is the same, it is necessary to calculate the loss sum.

[0117] Then update the dynamic network module based on atmospheric light. The parameter θ:

[0118]

[0119] in, ρ1 is the gradient of the joint loss, and ρ1 is the update step size.

[0120] The training transfer rate-guided dynamic network module is represented as follows:

[0121]

[0122] in, For the enhanced image, The atmospheric light has been altered. Then the joint loss is calculated.

[0123]

[0124] in The input image is modified after changing the transmission rate graph. Then, the transmission rate-guided dynamic network module is updated. Parameter φ:

[0125]

[0126] in, ρ is the gradient of the joint loss, and ρ2 is the update step size.

[0127] Specifically, in the atmospheric light-based dynamic network module, the color of the input image is first adjusted by randomly changing the atmospheric light, based on the underwater image imaging model. This adjustment is then used as input to train the atmospheric light-based dynamic network module and update its hyperparameters. This allows the dynamic structure to learn more information about color, thus adaptively adjusting for different color differences, resulting in images enhanced by the model that possess rich and vivid colors.

[0128] Specifically, the imaging model is as follows:

[0129] I c (x)=J c (x)e -βd(x) +(1-e -βd(x) A c c∈{r, g, b}

[0130] Where x represents the spatial location of each pixel, I c It is the observed image, J c This is the restored haze-free image, A c d(x) represents the global atmospheric light, d(x) is the scene depth at pixel x, and β is the channel-level attenuation coefficient that depends on the water quality.

[0131] Tc(x)=e -βd(x) This is a transmittance map, representing the percentage of spoke brightness in the scene. It indicates the percentage of scene irradiance that reaches the camera after reflection from point x. In an underwater scene, it represents the percentage that reaches the camera after reflection from point x, which also reflects the type of water.

[0132] Specifically, the formula for adjusting the color of the input image is as follows:

[0133]

[0134]

[0135] in, The image after adjusting the color of atmospheric light. The adjusted atmospheric light, This indicates a change in atmospheric light. This represents a random change in atmospheric light, where The coefficients are randomly generated within the range (0.3, 0.6).

[0136] Specifically, in the transfer rate-guided dynamic network module, the transfer rate map of the input image is first estimated, and then multiplied by a randomly generated parameter matrix. Based on the underwater image imaging model, the transfer rate of the input image is adjusted, and this is used as input to train the transfer rate-guided dynamic network module and update the corresponding hyperparameters. This model-enhanced dynamic structure encourages the network to focus more on areas of most severe quality degradation and can enhance images captured in various water qualities.

[0137] Specifically, the formula for calculating the transmission rate of the adjusted input image is as follows:

[0138]

[0139]

[0140] in, To adjust the transmission rate of the image, For the adjusted transmission rate, This indicates a change in the transmission rate. This indicates that the transmission rate is adjusted randomly, where The coefficients are randomly generated within the range (0.6, 1.1).

[0141] In another embodiment, this application also provides a dynamic underwater image enhancement device based on prior knowledge, including a processor and a memory storing a plurality of computer instructions, which, when executed by the processor, implement the steps of the above-described dynamic underwater image enhancement method based on prior knowledge.

[0142] Specific limitations regarding the prior knowledge-based dynamic underwater image enhancement device can be found in the limitations of the prior knowledge-based dynamic underwater image enhancement method described above, and will not be repeated here. Each module in the aforementioned prior knowledge-based dynamic underwater image enhancement device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0143] The memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory stores a computer program that can run on the processor, which implements the network topology layout method in this embodiment of the invention by running the computer program stored in the memory.

[0144] The memory may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores the program, and the processor executes the program upon receiving an execution instruction.

[0145] The processor may be an integrated circuit chip with data processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0146] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A priori knowledge-based dynamic underwater image enhancement method, based on the constructed image enhancement network model, the underwater image is enhanced, characterized in that, The image enhancement network model comprises a dynamic network module based on atmospheric light, a prior-based multi-scale network module and a dynamic network module guided by transmission rate, the prior-based dynamic underwater image enhancement method comprises: inputting the underwater image to be enhanced into the dynamic network module based on atmospheric light, performing dynamic convolution and global average pooling in the dynamic network module based on atmospheric light, and extracting atmospheric light prior information; inputting the underwater image to be enhanced into the dynamic network module guided by transmission rate, performing dynamic convolution and global average pooling in the dynamic network module guided by transmission rate, and extracting transmission rate prior information; inputting the underwater image to be enhanced and the corresponding transmission rate graph into the prior-based multi-scale network module, sequentially performing a first multi-scale U-shaped network, a transmission rate guided feature extraction network and a second multi-scale U-shaped network in the prior-based multi-scale network module, and obtaining an enhanced image; wherein the first multi-scale U-shaped network and the second multi-scale U-shaped network obtain convolution kernel parameters by processing the atmospheric light prior information, and the transmission rate guided feature extraction network obtains convolution kernel parameters by processing the transmission rate prior information; the first multi-scale U-shaped network and the second multi-scale U-shaped network have the same structure, comprising a feature extraction block, a convolution network, a feature extraction block and a convolution network, and the layers of the first multi-scale U-shaped network and the second multi-scale U-shaped network are connected through fast connection; the feature extraction block comprises a hyper convolution, a multi-scale feature extraction block and a hyper convolution connected in sequence, and the convolution network comprises three hyper convolutions; the transmission rate guided feature extraction network comprises a hyper convolution, a connection layer, a multi-scale feature extraction block and a hyper convolution connected in sequence.

2. The priori knowledge based dynamic underwater image enhancement method according to claim 1, characterized in that, The dynamic network module based on atmospheric light and the dynamic network module guided by transmission rate both comprise a first unit and a second unit, and each unit comprises two dynamic convolutions and one global average pooling.

3. The priori knowledge based dynamic underwater image enhancement method according to claim 1, wherein, The hyper convolution comprises a residual block and a full connection layer, and the residual block adopts the formula as follows: ； wherein, is an input feature, is an output feature; In the hyper convolution of the first multi-scale U-shaped network and the second multi-scale U-shaped network, the atmospheric light prior information obtains the parameters of the convolution kernel in the residual block through the full connection layer; In the hyper convolution of the transmission rate guided feature extraction network, the transmission rate prior information obtains the parameters of the convolution kernel in the residual block through the full connection layer.

4. The priori knowledge based dynamic underwater image enhancement method according to claim 1, wherein, The connection layer adopts the formula as follows: ； wherein, represents an input feature of the connection layer, represents an output feature of the connection layer, represents a transmission rate map corresponding to the underwater image to be enhanced.

5. The priori knowledge based dynamic underwater image enhancement method according to claim 1, wherein, The multi-scale feature extraction block comprises three convolution branches of different scales and a channel attention module, and performs the following operations: Input features of the multiscale feature extraction block After the channel attention module, the formula is expressed as follows: ； ； ； wherein, is an output feature of the channel attention module, is a global average pooling, is a global max pooling; The input features of the multi-scale feature extraction block pass through three convolution branches of different scales, and the output features of the three convolution branches of different scales are respectively multiplied and then added to obtain the output features of the multi-scale feature extraction block. The input features of the multi-scale feature extraction block pass through three convolution branches of different scales, and the output features of the three convolution branches of different scales are respectively multiplied and then added to obtain the output features of the multi-scale feature extraction block.

6. The priori knowledge based dynamic underwater image enhancement method according to claim 1, wherein, The prior-based dynamic underwater image enhancement method further comprises training the image enhancement network model; When training the image enhancement network model, the following steps are included: fixing the network parameters of the dynamic network module based on atmospheric light and the dynamic network module guided by transmission rate, and training the prior-based multi-scale network module; fixing the network parameters of the prior-based multi-scale network module and the dynamic network module guided by transmission rate, and training the dynamic network module based on atmospheric light; By fixing the network parameters of the prior-based multi-scale network module and the atmospheric light-based dynamic network module, a transmission rate-guided dynamic network module is trained.

7. The priori knowledge based dynamic underwater image enhancement method according to claim 6, characterized in that, The joint loss function used when training the image enhancement network model is as follows: ； wherein, and are weight parameters, is a mean squared error loss, is a perceptual loss, is a reconstruction loss, is an output image of the image enhancement network model, is a reference image.

8. A priori knowledge based dynamic underwater image enhancement device, comprising a processor and a memory having stored therein a plurality of computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images