An underwater image enhancement method based on an improved ARLTGAN model

Abstract

The application relates to the technical field of image enhancement, and discloses an underwater image enhancement method based on an improved ARLTGAN model, which comprises the following steps: obtaining an original image shot by an underwater robot, performing normalization preprocessing, and forming a to-be-processed image; loading an underwater image enhancement model, wherein the underwater image enhancement model is generated by training an improved ARLTGAN model and is used for realizing low-quality degraded underwater image I LR denoising, pixel enhancement, and outputting a high-resolution image I output ; the underwater image enhancement model comprises a generator and a discriminator; the to-be-processed image is input into the underwater image enhancement model, and a high-resolution image I output is output. The improved ARLTGAN model is introduced, the network architecture of the generator and the discriminator is innovated, a multi-source loss function is set, high-quality enhancement of the underwater image and highlighting of key defect features are realized, and the technical defects that the existing underwater image processing technology is not suitable for complex underwater environments are overcome.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement, in particular to an underwater image enhancement method based on an improved ARLTGAN model. BACKGROUND

[0002] In the field of safety detection of underwater structures of hydropower projects, underwater detection robots (ROV / AUV) have gradually replaced traditional manual inspection and become the core tool for defect detection of key parts such as dam body, gate and pressure pipeline. However, the complex optical characteristics of the underwater environment lead to three major problems in the images collected by the robot:

[0003] First, color distortion. The absorption coefficient of water to red light is much higher than that of blue and green light, which makes the image show a "blue-green color cast" with overloading of blue and green channels and attenuation of red channel information, which hides the color difference characteristics of the defect on the structure surface.

[0004] Second, detail blur. The light scattering effect of suspended particles leads to image edge diffusion and reduced contrast, and key defects such as millimeter-level cracks and tiny corrosion spots are easily overwhelmed by background noise.

[0005] Third, poor scene adaptability. The image degradation degree varies significantly under different water depths and flow rates, and a single enhancement algorithm cannot cover a variety of detection scenarios.

[0006] Existing underwater image enhancement techniques still have obvious limitations and cannot meet the high-precision detection requirements. Traditional non-deep learning methods (such as wavelet transform denoising, Retinex theory color correction, and histogram equalization) have low computational cost, but they do not model the underwater optical degradation mechanism and can only perform global gray scale adjustment or single noise suppression. For example, wavelet transform can smooth the crack edges during denoising, and the color correction effect of Retinex theory on strong speckle noise scenes is unstable, which cannot accurately preserve the defect detail features and cannot adapt to the complex underwater degradation environment of hydropower stations.

[0007] Deep learning methods based on a single support also have inherent technical bottlenecks: pure CNN methods rely on local convolution kernels to extract features, and although they can optimize detail restoration through residual connection and dense connection, they are limited by the receptive field range and cannot effectively capture the global spatial relationship of underwater structures, resulting in the problem of "local clarity but global structure fragmentation" in the enhanced image; pure Transformer methods capture long-range context information through self-attention mechanisms, which can improve the global structure integrity, but lack sensitivity to local minor defects - the global attention calculation is easily disturbed by underwater uniform noise, and the attention operation complexity of large-size feature maps is high, which has poor real-time performance on embedded detection devices; traditional GAN methods improve image visual authenticity through adversarial training, but have two major defects:

[0008] First, the discriminator mostly uses traditional pooling layer downsampling, which is easy to lose weak feature responses such as crack edges and corrosion spots.

[0009] Second, the loss function mostly relies on a single adversarial loss or a VGG series network feature content loss, which is difficult to balance the relationship between "noise elimination", "color correction" and "detail preservation", resulting in over-sharpening or blurred details in the generated image.

[0010] As disclosed in the patent document CN115861094B, a lightweight GAN underwater image enhancement model is disclosed, which includes a generation network and a discrimination network; the generation network includes an encoder Encoder and a decoder Decoder, the encoder Encoder includes a depth separable convolution module and an attention module, the decoder Decoder includes a plurality of up-sampling convolution modules; the discrimination network obtains images through a Markov discriminator PatchGAN, and the model can process synthetic underwater images and real underwater images. However, the adaptability to complex underwater environments is poor, and the image processing effect for complex underwater degradation needs to be improved.

[0011] In addition, the existing feature fusion method has poor adaptability: some methods try to combine the advantages of CNN and Transformer, but mostly use fixed weight feature splicing or simple weighted fusion, without considering the feature differences of underwater images in different areas. For example, the defect area needs to strengthen local detail extraction, and the background area needs to optimize global color correction, and the fixed fusion strategy cannot dynamically adapt to such regional specificity requirements, resulting in a significant decline in enhancement effect in complex underwater degradation scenarios. At the same time, there is an engineering adaptation requirement for image enhancement technology in underwater detection of hydropower stations: existing algorithms are mostly trained based on general underwater datasets, without optimization for scene features specific to hydropower stations such as "concrete dam texture", "metal gate corrosion texture", "pipeline weld structure", etc., and the model generalization ability is insufficient; and the embedded hardware resources of detection robots are limited, and existing large parameter models cannot meet the real-time processing requirements.

[0012] In summary, the current underwater image enhancement technology has shortcomings in "local detail-global structure collaborative enhancement", "dynamic scene adaptability" and "engineering real-time performance", which has become a key bottleneck restricting the accurate detection of underwater structure defects in hydropower stations, and it is urgent to build a new type of underwater image enhancement technology scheme that takes into account multi-scale feature interaction capability, dynamic weight adaptation mechanism and lightweight engineering properties. SUMMARY

[0013] To solve the above prior art in "local-global feature coordination", "dynamic scene adaptation", "engineering real-time" and the technical problems of speckle noise, blue-green color deviation, low contrast and detail blur in underwater detection images, the present application provides an underwater image enhancement method based on an improved ARLTGAN model, including the following steps:

[0014] An original image taken by an underwater robot is obtained, normalized preprocessing is performed, and a to-be-processed image is formed; the original image has speckle noise, blue-green color deviation, low contrast and detail blur characteristics;

[0015] An underwater image enhancement model is loaded, which is generated by training an improved ARLTGAN model and is used to realize low-quality degraded underwater image I LR de-noising and pixel enhancement, and output a high-resolution image I output ; the underwater image enhancement model includes a generator and a discriminator;

[0016] The to-be-processed image is input to the underwater image enhancement model, and a high-resolution image I output is output.

[0017] Further, before loading the underwater image enhancement model, the underwater image enhancement model is constructed, including the following contents:

[0018] An improved ARLTGAN model is defined;

[0019] An underwater original degraded image dataset is prepared, normalized preprocessing is performed, and an underwater standard degraded image sample set I input is formed;

[0020] The underwater standard degraded image sample set I input is used to train an improved ARLTGAN model to generate the underwater image enhancement model; the underwater original degraded image sample set I input includes real high-quality underwater images I HR and low-quality degraded underwater images I LR ;

[0021] During the training, a hybrid loss function L G and a training strategy are introduced.

[0022] Further, the generator includes a shallow feature module, a deep feature module and an image reconstruction module, which are used to output a high-resolution image I output ; the discriminator includes a multi-scale convolution downsampling module, a feature attention aggregation module and a probability output module, which are used to extract a global feature vector of the high-resolution image I output , and judge the high-resolution image I outputProbability of authenticity.

[0023] Further, the construction method of the generator comprises the following contents:

[0024] The shallow feature module is constructed by using a 3*3 convolution layer and a LeakyReLU activation function to extract the basic features F input of the underwater standard degradation image sample set I shallow and transmit to the deep feature module; the basic features F shallow include basic texture and contour features;

[0025] The deep feature module is constructed by using 6 enhanced residual dense blocks RRDB-II, 6 lightweight visual Transformer modules LightViT-Plus and intelligent adaptive feature fusion AFF-II modules; the RRDB-II is used to extract local features in the basic features F shallow , the LightViT-Plus is used to capture global features in the basic features F shallow , the AFF-II module is used to dynamically generate attention weights α, and the local features and the global features are weighted and fused to form deep features F deep transmitted to the image reconstruction module;

[0026] The image reconstruction module is constructed by using a progressive upsampling structure and a residual connection to reconstruct the deep features F deep into a high-resolution image and output an enhanced image I output .

[0027] Further, the RRDB-II and the LightViT-Plus adopt a group cascading manner, and each group is connected to an AFF-II module after cascading; the group cascading manner is that two RRDB-II and one LightViT-Plus form a group of cascades;

[0028] Wherein, the RRDB-II is built-in with multiple dense blocks, and the structure of each dense block is residual connection + cross-layer channel attention;

[0029] The LightViT-Plus adopts a small window attention mechanism and introduces a local cross-attention layer to interact the features in the window with the edge features of adjacent windows.

[0030] The AFF-II module adopts a 2-layer 1*1 convolution and a Sigmoid activation function to dynamically generate attention weights α and weightedly fuse the local features and the global features.

[0031] Further, the output of the shallow feature module is F shallow , ;

[0032] The output of the deep feature module is F deep , and the expression is: deep =F AFF-Ⅱ , ;

[0033] The output of the image reconstruction module is I output , and the expression is:

[0034] ;

[0035] wherein F RRDB-Ⅱ is the output of the RRDB-II module, F LightViT-Plus is the output of the LightViT-Plus module, F AFF-Ⅱ is the output of the AFF-II module, and PixelShuffle represents pixel reorganization.

[0036] Further, the progressive up-sampling structure is to adjust the deep feature F dimension through a 1×1 convolution layer, realize super-resolution enlargement through 2 times pixel reorganization PixelShuffle, and then output the feature map after super-resolution enlargement through a 3×3 convolution layer; the residual connection is to add the basic feature F shallow adjusted in dimension through a 1×1 convolution.

[0037] Further, the process of judging the authenticity probability of the high-resolution image I output includes the following contents:

[0038] After the multi-scale convolution down-sampling module receives the high-resolution image I output , it is convolved through 4-level 3×3 convolution units, and after each convolution, it is sequentially processed through a BatchNorm layer, a LeakyReLU activation function, and a local response normalization LRN layer. The feature map F d after convolution is input into the feature attention aggregation module;

[0039] The feature attention aggregation module generates a spatial attention map through a 1×1 convolution and a Sigmoid activation function, weights the feature map F d input by the multi-scale convolution down-sampling module, and then compresses the spatial dimension through global average pooling to generate a global feature vector F global , which is input into the probability output module;

[0040] After the probability output module receives the global feature vector F global , it adopts a 2-layer fully connected layer FC, accesses a LeakyReLU activation function in the middle, and finally outputs an image authenticity probability P real through a Sigmoid function, wherein the image authenticity probability Preal The expression is: ;

[0041] When the true probability P of the output image real When the value is close to 1, the high-resolution image I is determined. output To produce a true, high-quality image; when the output image has a true probability P real When the value is close to 0, the high-resolution image I is determined to be... output The image generated by the generator;

[0042] In this context, FC1 and FC2 represent the first fully connected layer and the second fully connected layer, respectively.

[0043] Furthermore, the loss function L G Including combating losses Gadv Content loss content and perceived loss The loss function L G The expression is: ;

[0044] Wherein, the adversarial loss Gadv The expression is: ;

[0045] The content loss content The expression is: ;

[0046] The perceived loss The expression is: ;

[0047] in, This is the content loss weighting coefficient. P represents the weighting coefficient for perceived loss. data Represents a true high-quality image I HR The probability distribution, G(I) LR D( is the enhanced image output by the generator) ) represents the probability of authenticity output by the discriminator. This represents the desired operation on the dataset. This is the feature extraction function for ResNet-50. The dimension of the feature map for layer 4.

[0048] Furthermore, the training strategy includes the following:

[0049] The AdamW optimizer is used, with an initial learning rate set to , learning rate is decayed to 0.45 times of the previous epoch every 15 training epochs, the total training epoch is 80, the batch size is set to 2, and an early stopping mechanism is set;

[0050] using the underwater standard degraded image sample set I input training is performed, when the peak signal-to-noise ratio (PSNR) of the underwater standard degraded image sample set I input stops training when the peak signal-to-noise ratio (PSNR) is improved by less than or equal to 0.02 dB for 4 consecutive epochs.

[0051] The present application has the following beneficial effects: The present application introduces an improved ARLTGAN model, sets a "2 RRDB-II + 1 LightViT-Plus" grouping cascade architecture, RRDB-II adds channel attention reinforcement to local defects, LightViT-Plus uses small window cross attention to reduce costs and cooperate with global and edge information; upgrade the AFF-II module to dynamically adapt to feature fusion with double convolution + BatchNorm; construct a double attention discriminator without a pooling layer to preserve weak features and improve discrimination ability; fuse improved Wasserstein, ResNet-50 content, and LPIPS perception loss to ensure quality; at the same time, through small size input and mixed precision training, it is still able to capture long-distance global context information while reducing computational complexity; through the whole process scheme of "data preprocessing optimization - network architecture innovation - loss function design - engineering deployment adaptation", it realizes high-quality enhancement and key defect feature highlighting of underwater images, adapts to complex underwater image processing application scenarios, and overcomes the technical defects of existing underwater image processing technology that is not suitable for complex underwater environments. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is the flow chart of the underwater image enhancement method based on the improved ARLTGAN model provided by the present application;

[0053] Figure 2 is the underwater image enhancement effect comparison chart of the improved ARLTGAN model and the U-Net and SwinTransformer models provided by the present application. DETAILED DESCRIPTION

[0054] The technical solutions of the present application are further described below, but the scope of protection is not limited to the description.

[0055] The present application embodiment provides an underwater image enhancement method based on an improved ARLTGAN model, as shown in Figure 1 , including the following steps:

[0056] Step S100, obtain the original image shot by the underwater robot, perform normalization preprocessing to form a to-be-processed image; the original image has speckle noise, blue-green color deviation, low contrast and detail blur characteristics;

[0057] The normalization preprocessing mentioned in step S100 includes: first adjusting the brightness distribution through adaptive histogram equalization (CLAHE), then preliminarily removing large-scale noise through median filtering (3*3), and finally performing processing according to the normalization preprocessing rule in step S200.

[0058] Step S200, load the underwater image enhancement model, wherein the underwater image enhancement model is generated by training an improved ARLTGAN model, and is used to realize denoising and pixel enhancement on the low-quality degraded underwater image I LR to output a high-resolution image I output ; the underwater image enhancement model includes a generator and a discriminator.

[0059] Before loading the underwater image enhancement model, the underwater image enhancement model is constructed, including the following contents:

[0060] Define an improved ARLTGAN model.

[0061] Prepare an underwater original degraded image dataset, perform normalization preprocessing to form an underwater standard degraded image sample set I input ;

[0062] Train the improved ARLTGAN model using the underwater standard degraded image sample set I input to generate the underwater image enhancement model; the underwater original degraded image sample set I input includes real high-quality underwater images I HR and low-quality degraded underwater images I LR .

[0063] During the training, a hybrid loss function L G and a training strategy are introduced.

[0064] The normalization preprocessing mentioned in step S200 includes: generating high-quality reference images as training labels by using bilinear interpolation up-sampling (2 times), adjusting contrast by ±18%, and adjusting saturation by ±12% on the underwater original degraded image dataset; designing a data enhancement strategy, including adjusting brightness in the range of ±8% to ±22%, performing ±15% intensity disturbance on the red channel, and implementing a combination operation of vertical flipping and 90° rotation; cropping all images to a size of 224*224 pixels, normalizing to the [-1, 1] interval with a mean of 0.45 and a standard deviation of 0.42, and dividing the underwater original degraded image dataset into a training set, a validation set and a test set in a ratio of 7:2:1.

[0065] The data enhancement strategy further comprises adding Gaussian noise (variance 0.02-0.05) and salt and pepper noise (density 0.01-0.03) to simulate the underwater speckle noise scene and improve the noise generalization ability of the model.

[0066] The generator adopts a three-level architecture of "shallow feature extraction-multiscale deep feature fusion-super-resolution reconstruction", specifically including a shallow feature module, a deep feature module and an image reconstruction module, for outputting a high-resolution image I output ;

[0067] The construction method of the generator comprises the following contents:

[0068] The shallow feature module is constructed by using a 3x3 convolution layer (channel number 64) and a LeakyReLU activation function (negative slope 0.15) to extract the basic features F input of the underwater standard degraded image sample set I shallow and transmit them to the deep feature module; the basic features F shallow include basic texture and contour features; the output of the shallow feature module is F shallow , (1)

[0069] The deep feature module is constructed by using 6 enhanced residual dense blocks RRDB-II, 6 lightweight visual Transformer modules LightViT-Plus and an intelligent adaptive feature fusion AFF-II module; the RRDB-II is used to extract local features in the basic features F shallow , the LightViT-Plus is used to capture global features in the basic features F shallow , and the AFF-II module is used to dynamically generate attention weights α, weight the local features and the global features, and form deep features F deep transmitted to the image reconstruction module;

[0070] The RRDB-II and the LightViT-Plus adopt a grouping cascading mode, and each group is connected after accessing 1 AFF-II module, and the grouping cascading mode is that 2 RRDB-II and 1 LightViT-Plus form a group of cascades;

[0071] The RRDB-II has multiple dense blocks built-in, and the structure of each dense block is residual connection + cross-layer channel attention; the dense blocks are 4; the expression of a single residual block is:

[0072] (2)

[0073] wherein, For the channel attention mechanism, the key channel response is strengthened through compression and excitation operations; F input For the input features of the residual block, BN represents batch normalization, and ReLU is an activation function.

[0074] The dense block built-in the RRDB-II adopts a "dense connection + cross-layer skip connection" structure, and the feature calculation expression of a single dense block is:

[0075] (3)

[0076] And a BN layer and a ReLU activation function are connected after each convolutional layer.

[0077] The LightViT-Plus adopts a small window attention mechanism and introduces a local cross-attention layer to interact the features within the window with the edge features of adjacent windows; the calculation expression of the attention mechanism is:

[0078] (4)

[0079] wherein, , Q and K are the query and key matrices of the edge features of adjacent windows, respectively, is the edge feature weight coefficient (value 0.35); d k is the dimension of Q / K, and T represents a transposed matrix.

[0080] The specific processing procedure of the LightViT-Plus light-weight visual Transformer module includes:

[0081] Firstly, feature blocking: the input feature map is divided into non-overlapping blocks according to 5x5 windows, and each block is flattened into a feature vector;

[0082] Secondly, local cross-attention calculation: for each window feature vector, query (Q), key (K), and value (V) matrices are generated, and the features of 2 pixels at the edge of adjacent windows are extracted to form and According to the attention calculation expression, the attention weight is obtained, and the local cross-attention feature is obtained by weighting the V matrix;

[0083] Thirdly, multi-head attention fusion: 6 attention heads are calculated in parallel, the output features of each head are spliced, and then fused through a 1x1 convolutional layer (channel number 64) to obtain multi-head attention features .

[0084] Finally, layer normalization and MLP processing: the multi-head attention features are processed through a layer normalization layer and a multi-layer perceptron (MLP) to obtain the output features of the LightViT-Plus module Layer normalization (LN) is performed, and then the deep feature F is processed by a MLP module composed of two 1x1 convolution layers (with channel numbers of 256 and 64, respectively) and a GELU activation function, and finally output , and the expression is:

[0085] (5)

[0086] The AFF-II module adopts two 1x1 convolution layers (with channel numbers of 128 and 64, respectively) and a Sigmoid activation function to dynamically generate attention weights α, and the local features and global features are fused by weighting.

[0087] The calculation expression of the attention weights α is:

[0088] (6)

[0089] The calculation expression of the weighted fusion is:

[0090] (7)

[0091] The output of the deep feature module is F deep , F deep =F AFF-Ⅱ .

[0092] (8)

[0093] wherein, F RRDB-Ⅱ is the output of the RRDB-II module, F LightViT-Plus is the output of the LightViT-Plus module, and F AFF-Ⅱ is the output of the AFF-II module.

[0094] An image reconstruction module is constructed by using a progressive upsampling structure and a residual connection, which is used to reconstruct the deep feature F deep into a high-resolution image, and output an enhanced image I output .

[0095] The progressive upsampling structure is to adjust the dimension of the deep feature F by a 1x1 convolution layer (with a channel number of 128), realize super-resolution enlargement by a 2-pixel reorganization PixelShuffle, and then output a feature map after super-resolution enlargement by a 3x3 convolution layer (with a channel number of 3); and the residual connection is to add the basic feature F shallow after adjusting the dimension by a 1x1 convolution (with a channel number of 3).

[0096] The output of the image reconstruction module is I output , and the expression is:

[0097] (9)

[0098] wherein, PixelShuffle represents pixel reorganization.

[0099] The discriminator adopts a three-level structure of "multi-scale convolution down-sampling-feature attention aggregation-probability output", and does not set a traditional pooling layer, and specifically includes a multi-scale convolution down-sampling module, a feature attention aggregation module and a probability output module, which are used to extract a global feature vector of the high-resolution image I output and judge a reality probability of the high-resolution image I output .

[0100] The process of judging the reality probability of the high-resolution image I output includes the following contents:

[0101] After the multi-scale convolution down-sampling module receives the high-resolution image I output , 4-level 3x3 convolution units are used for convolution (channel number 32→64→128→256, step 2), and after each level of convolution, BatchNorm layer, LeakyReLU activation function (negative slope 0.2) and local response normalization LRN layer are sequentially used for processing, and the feature map F d after the convolution is input into the feature attention aggregation module; the output of the multi-scale convolution down-sampling module is F d , and the expression is:

[0102] (10)

[0103] The multi-scale convolution down-sampling module is also provided with a channel attention branch, and a channel weight is calculated by an SE module for each level of convolution output feature map, and the weighting formula is:

[0104] (11)

[0105] wherein, is a global average pooling operation.

[0106] The feature attention aggregation module introduces a spatial attention mechanism, generates a spatial attention map by 1x1 convolution (channel number 1) and Sigmoid activation function, and performs weighting on the feature map F d input from the multi-scale convolution down-sampling module to obtain F att , and then compresses the spatial dimension by global average pooling to generate a global feature vector F global , which is input into the probability output module; the output of the feature attention aggregation module is the global feature vector F global , and the expression is:

[0107] (12)

[0108] (13)

[0109] wherein, is the spatial size of the feature map;

[0110] The probability output module receives the global feature vector F global , and then adopts two full connection layers FC (the number of neurons is 128 and 1 respectively), and a LeakyReLU activation function (the negative slope is 0.2) is connected in the middle, and finally outputs the image authenticity probability P real through a Sigmoid function. real The expression of the image authenticity probability P

[0111] (14)

[0112] When the output image authenticity probability P real is close to 1, it is determined that the high-resolution image I output is a real high-quality image; when the output image authenticity probability P real is close to 0, it is determined that the high-resolution image I output is an image generated by the generator.

[0113] wherein, FC1 and FC2 represent the first full connection layer and the second full connection layer respectively.

[0114] The loss function L G includes an adversarial loss Gadv , a content loss content and a perception loss , and the expression of the loss function L G is:

[0115] (15)

[0116] The adversarial loss Gadv is calculated by using an improved Wasserstein distance, and the expression is:

[0117] (16)

[0118] The content loss content is calculated based on the layer4 layer features of the pre-trained ResNet-50 network, and the expression is:

[0119] (17)

[0120] the perceptual loss The LPIPS index is used to calculate the perceptual difference between the generated image and the real image, and the expression is:

[0121] (18)

[0122] wherein, is a content loss weight coefficient, is a perceptual loss weight coefficient; P data represents the probability distribution of the real high-quality image I HR , G(I LR ) is the enhanced image output by the generator, D( ) is the authenticity probability output by the discriminator, represents the expected operation on the data set, is a ResNet-50 feature extraction function, is the layer4 feature map dimension.

[0123] The LPIPS index calculation uses a pre-trained AlexNet network, selects conv2, conv3, and conv4 layer features, and sets the difference weights of each layer feature to 0.3, 0.4, and 0.3, respectively. The total perceptual loss is the weighted sum of the difference of each layer feature.

[0124] The training strategy includes the following:

[0125] An AdamW optimizer is used ( , with a weight decay of 0.001), the initial learning rate is set to , the learning rate is decayed to 0.45 times of the previous epoch every 15 training epochs (epochs), the total training epochs are 80, the batch size is set to 2, and an early stopping mechanism is set.

[0126] The underwater standard degraded image sample set I input is used for training, and when the peak signal-to-noise ratio PSNR of the underwater standard degraded image sample set I input is improved by ≤0.02dB for 4 consecutive epochs (epochs), the training is stopped.

[0127] In the training process, a gradient clipping strategy is used, the upper limit of the gradient norm is set to 1.5 to avoid the problem of gradient explosion, and mixed precision training (FP16) is used to improve the training efficiency.

[0128] Step S300, input the to-be-processed image to the underwater image enhancement model, and output a high-resolution image I output .

[0129] The embodiment of the application further provides a performance verification method of the underwater image enhancement model, and specifically comprises the following contents:

[0130] PSNR, SSIM and LPIPS are used as objective evaluation indexes, wherein the image is first normalized to the [0, 1] interval when PSNR is calculated, SSIM is calculated by using a 3*3 Gaussian window (standard deviation 1.2), and LPIPS is calculated by using default parameters.

[0131] The embodiment of the application innovatively introduces the ARLTGAN model. The core lies in designing a "2 RRDB-II+1 LightViT-Plus" grouping cascade architecture, RRDB-II strengthens local defects by channel attention, LightViT-Plus reduces cost by using small window cross attention and cooperates with global and edge information; upgrading the AFF-II module to dynamically adapt to feature fusion by double convolution+BatchNorm; constructing a double-attention discriminator without a pooling layer to retain weak features and improve the discrimination ability; fusing improved Wasserstein, ResNet-50 content and LPIPS perception loss to guarantee quality; at the same time, through small size input and mixed precision training, the long distance global context information can still be captured while reducing the amount of calculation.

[0132] In the ARLTGAN model, 6 RRDB-Ⅱ and 6 LightViT-Plus are used, and a grouping cascade structure of “2 RRDB-Ⅱ+1 LightViT-Plus” is used. Compared with the traditional RRDB and LightViT modules, the parameter amount is reduced by about 25%, the real-time processing speed is increased by 20%, and the feature interference is reduced through grouping cooperation. The 2 RRDB-Ⅱ focuses on local defect feature extraction such as cracks and corrosion spots, and then the 1 LightViT-Plus captures the global structure, and the millimeter-level crack detection rate is increased to more than 90%. At the same time, the RRDB-Ⅱ module can dynamically enhance the defect feature channel and suppress noise, and the feature extraction signal-to-noise ratio is increased by 8-10dB in the speckle noise scene; the LightViT-Plus uses a 5×5 window + local cross-attention to accurately capture the edge correlation of adjacent regions and improve the edge information retention rate; the AFF-Ⅱ module uses a double-layer convolution dynamic weight, and in the “strong scattering + color deviation” scene, the fused features have a PSNR increase of 1.2-1.5dB and a SSIM increase of 0.08-0.12 compared with the traditional AFF module, and the adaptability to complex scenes is significantly enhanced. In terms of discriminators, the traditional pooling layer is removed and multi-scale convolution downsampling (32→64→128→256 channels) is used, the weak feature retention rate is increased by 30%, combined with the newly added spatial-channel dual attention aggregation, the discrimination accuracy of the discriminator for “real-generated images” is increased by 12%, avoiding the generated image from appearing “over-correction” or “color deviation residual”.

[0133] In terms of loss function settings, the improved ARLTGAN model adds LPIPS perceptual loss (based on pre-trained AlexNet), reduces the perceptual similarity index LPIPS to below 0.15, and the generated image is more natural in subjective vision, and the structure deviation (such as crack length, corrosion area) is reduced to within 5%; the adversarial loss is upgraded to an improved Wasserstein loss with gradient penalty, the training loss fluctuation is reduced to ±0.2, the convergence speed is increased by 20%, and “mode collapse” is avoided; the content loss is replaced by ResNet-50 feature guidance (instead of VGG-19), which is more accurate in representing global structure, and the color restoration degree is increased by 18-22% in the blue-green color deviation scene.

[0134] The data processing and training strategy are optimized, and the model generalization and engineering adaptation capability are strengthened. In the data enhancement, Gaussian noise (variance 0.02-0.05), salt and pepper noise (density 0.01-0.03) and red channel ±15% disturbance are added, which simulates the real underwater speckle and color deviation scene, and the attenuation rate of the enhancement effect in the unknown noise scene is reduced to within 10%; the input size is optimized to 224x224, combined with FP16 mixed precision training, the single image inference time is reduced to 0.5s, the GPU memory occupation is reduced to 16GB, which is suitable for the embedded hardware of the underwater detection robot; the early stopping mechanism (PSNR continuous 4 rounds of increase ≤0.02dB stop) shortens the training time by 20%, suppresses overfitting, and reduces the generalization error by 8-10%.

[0135] The underwater image enhancement method based on the improved ARLTGAN model provided by the application has the following advantages: Figure 2 (a)-(d) are shown in Figure 2 (a) The three images arranged from top to bottom are three original images taken underwater by a underwater robot; Figure 2 (b) The three images arranged from top to bottom are the corresponding effect images of the three original images in Figure 2 (a) processed by the U-Net model; Figure 2 (c) The three images arranged from top to bottom are the corresponding effect images of the three original images in Figure 2 (a) processed by the SwinTransformer model; Figure 2 (d) The three images arranged from top to bottom are the corresponding effect images of the three original images in Figure 2 (a) processed by the underwater image enhancement method based on the improved ARLTGAN model provided by the application; Figure 2 From the comparison of the image effects in (a)-(d), it is obvious that the speckle noise, blue-green color deviation, low contrast and detail blur problems of the underwater original image processed by the underwater image enhancement method based on the improved ARLTGAN model provided by the application are better.

[0136] In summary, the underwater image enhancement method based on the improved ARLTGAN model provided by the application has the following advantages: in the underwater scene verification of the hydropower station, the peak signal-to-noise ratio PSNR of the improved ARLTGAN model is improved from 28.5dB to 30.2dB, and the structural similarity index SSIM is improved to 0.91, not only realizing the leap-forward improvement of defect detection accuracy, but also solving the pain points of poor adaptability and difficult engineering landing of traditional methods in complex environments, providing a high-precision, high-reliability and landable technical solution for underwater structure safety detection.

[0137] The above disclosure is merely a specific embodiment of the present application, but the present application is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the scope of the present application.

Claims

1. An underwater image enhancement method based on an improved ARLTGAN model, characterized in that, The method comprises the following steps: An original image taken by an underwater robot is acquired, normalized and preprocessed to form a to-be-processed image; the original image has speckle noise, blue-green color deviation, low contrast and fuzzy details; Load an underwater image enhancement model generated by an improved ARLTGAN model for training to realize low-quality degraded underwater image I LR De-noising, pixel enhancement, output high-resolution image I output The underwater image enhancement model includes a generator and a discriminator; the generator includes a shallow feature module, a deep feature module and an image reconstruction module, and is used for outputting a high-resolution image I output The construction method of the generator includes the following contents: A shallow feature module is constructed using a 3×3 convolutional layer and the LeakyReLU activation function to extract underwater standard degraded image sample set I. input Basic features F shallow Transmitted to the deep feature module; the basic feature F shallow Includes basic texture and contour features; The deep feature module is constructed by using 6 enhanced residual dense blocks (RRDB-II), 6 light-weight visual transformer modules (LightViT-Plus) and an intelligent adaptive feature fusion (AFF-II) module. The RRDB-II is used for extracting local features in the basic feature F shallow The LightViT-Plus is used for capturing global features in the basic feature F shallow The AFF-II module is used for dynamically generating attention weights α, weighting and fusing the local features and the global features to form a deep feature F deep which is transmitted to the image reconstruction module. An image reconstruction module is constructed by using a progressive upsampling structure and a residual connection, and is used to reconstruct the deep feature F deep into a high-resolution image, and output the high-resolution image I output ; inputting the to-be-processed image into the underwater image enhancement model, and outputting a high-resolution image I output ; The RRDB-II and the LightViT-Plus are connected in a grouping cascade mode, and each group is connected to an AFF-II module after cascading; the grouping cascade mode is that two RRDB-II and one LightViT-Plus form a group cascade; The RRDB-II is internally provided with multiple dense blocks, and the structure of each dense block is residual connection + cross-layer channel attention; The LightViT-Plus adopts a small window attention mechanism and introduces a local cross-attention layer to interact the features in the window with the edge features of adjacent windows; The AFF-II module adopts two layers of 1x1 convolution and a Sigmoid activation function to dynamically generate attention weights alpha, and weightedly fuses the local features and the global features.

2. The underwater image enhancement method based on the improved ARLTGAN model according to claim 1, wherein, Before loading the underwater image enhancement model, the underwater image enhancement model is constructed, comprising the following contents: An improved ARLTGAN model is defined. Preparing an underwater original degraded image dataset, performing normalization preprocessing, forming the underwater standard degraded image sample set I input ; with the underwater standard degraded image sample set I input training the improved ARLTGAN model to generate the underwater image enhancement model; the underwater standard degraded image sample set I input including real high-quality underwater images I HR and low-quality degraded underwater images I LR ; wherein, in the course of the training, a hybrid loss function L is introduced G with the training strategy.

3. The underwater image enhancement method based on the improved ARLTGAN model according to claim 1, wherein, The discriminator comprises a multi-scale convolution down-sampling module, a feature attention aggregation module and a probability output module, and is used for extracting a global feature vector of the high-resolution image I output , judging a reality probability of the high-resolution image I output .

4. The underwater image enhancement method based on the improved ARLTGAN model according to claim 1, wherein, The output of the shallow feature module is F shallow , ; The output of the deep feature module is F deep , F deep =F AFF-Ⅱ , ; The output of the image reconstruction module is I output , and the expression is: ； where F RRDB-Ⅱ is the output of the RRDB-II module, F LightViT-Plus is the output of the LightViT-Plus module, F AFF-Ⅱ is the output of the AFF-II module, and PixelShuffle denotes pixel reshuffling.

5. The underwater image enhancement method based on the improved ARLTGAN model according to claim 4, wherein, The progressive up-sampling structure is to adjust the deep feature through a 1x1 convolution layer After the dimension, the super-resolution is enlarged through a 2-pixel reorganization PixelShuffle, and then a 3x3 convolution layer is used to output the feature map after the super-resolution enlargement; the residual connection is the basic feature F shallow is added.

6. The underwater image enhancement method based on the improved ARLTGAN model according to claim 3, wherein, determining the high resolution image I output The process of authenticity probability comprises the following: The multi-scale convolution downsampling module receives the high-resolution image I output After that, the feature map F is convolved by 4-level 3x3 convolution units, and after each level of convolution, the feature map F is sequentially processed by a BatchNorm layer, a LeakyReLU activation function and a local response normalization (LRN) layer. d The feature attention aggregation module is input. The feature attention aggregation module generates a spatial attention map through a 1x1 convolution and a Sigmoid activation function, and generates a feature map F input from the multi-scale convolution downsampling module d The weighted feature map is compressed in spatial dimension by global average pooling to generate a global feature vector F global , and input into the probability output module; The probability output module receives the global feature vector F global After that, a 2-layer fully connected layer FC is adopted, a LeakyReLU activation function is accessed in the middle, and finally a Sigmoid function is used to output the image authenticity probability P real The expression of the image authenticity probability P real is as follows: ; When the output image real probability P real is close to 1, it is determined that the high resolution image I output is a real high quality image; When the output image real probability P real close to 0, then it is determined that the high resolution image I output is the image generated by the generator; FC1 and FC2 represent a first layer full connection layer and a second layer full connection layer respectively.

7. The underwater image enhancement method based on the improved ARLTGAN model according to claim 2, wherein, the loss function including an adversarial loss , a content loss , and a perceptual loss , the loss function is expressed as: ; Wherein the adversarial loss The expression is: ; The content loss The expression is: ; The perceptual loss The expression is: ; wherein, is a content loss weight coefficient, is a perceptual loss weight coefficient; P data denotes a probability distribution of real high-quality images I HR , G(I LR ) is an enhanced image output by the generator, D( ) is a reality probability output by the discriminator, denotes an expected operation on a data set, is a ResNet-50 feature extraction function, is a layer4 layer feature map dimension.

8. The underwater image enhancement method based on the improved ARLTGAN model according to claim 2, wherein, The training strategy comprises the following contents: The AdamW optimizer is adopted, the initial learning rate is set to The learning rate is decayed to 0.45 times of the previous epoch every 15 training epochs, the total training epochs are 80, the batch size is set to 2, and an early stopping mechanism is set. using the underwater standard degraded image sample set I input training is performed, when the peak signal-to-noise ratio PSNR of the underwater standard degraded image sample set I input is improved by ≤0.02 dB for 4 consecutive rounds, the training is stopped.

Images