Multi-branch underwater image enhancement system for non-uniform illumination and color distortion

Abstract

The present application relates to the technical field of image enhancement, in particular to a multi-branch underwater image enhancement system for non-uniform illumination and color distortion, comprising: a data set for storing underwater images, model training and testing; a brightness correction module for receiving an original image, converting the original image into an L channel of a CIELab color space, further decomposing the L channel into a reflection component R and an illumination component I based on the Retinex theory, performing brightness correction using a double-encoder-decoder structure, and outputting an image P after brightness correction; a double-branch underwater image enhancement model GLNet based on global-local features, receiving the image P, starting from global features and local features respectively, using a double-branch network to process color feature and contrast feature related information respectively, comprehensively repairing underwater image brightness, contrast and color on three levels, and realizing image enhancement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement, in particular to a multi-branch underwater image enhancement system for non-uniform illumination and color distortion. BACKGROUND

[0002] At present, the benefits of the ocean in resources, environment, space and other aspects are increasingly apparent, and the development and utilization of marine resources have gradually been widely valued by countries around the world. However, due to the interference of light and underwater environment, the underwater optical image inevitably has degradation problems such as blurring and color distortion. The degraded image loses the original texture and color details, making image feature extraction and detection very difficult, which further exacerbates the difficulty of visual task application in underwater environment. Therefore, the purpose of underwater image enhancement technology is to improve the quality of underwater images so as to more effectively utilize image feature information. Underwater image enhancement has numerous practical applications, including target detection, marine environment monitoring, shipwreck detection, etc. By utilizing image enhancement technology to improve the quality of underwater images, the present application has important theoretical significance and practical application value.

[0003] In order to improve the poor lighting conditions in the water, underwater imaging equipment usually needs to be equipped with auxiliary light (AL) to improve the lighting conditions during shooting, thereby improving the quality of the collected underwater images. However, since the auxiliary light usually has a relatively high intensity energy and is relatively close to the object, the illuminated area and the non-illuminated area usually exhibit different brightness distribution and color distribution characteristics, and this non-uniform illumination effect further exacerbates the complexity of the underwater image enhancement problem. Therefore, the present application proposes a multi-branch underwater image enhancement system for non-uniform illumination and color distortion. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art and provide a multi-branch underwater image enhancement system for non-uniform illumination and color distortion. In view of the non-uniform illumination problem caused by the auxiliary light, the L channel of the CIELab color space which is sensitive to illumination features is processed, the L channel is further decomposed into a reflection component R and an illumination component I based on the Retinex theory, and a double-encoder-decoder structure is used for brightness correction to repair the brightness distortion problem caused by the auxiliary light. On this basis, in order to solve the two problems of inherent color distortion and contrast reduction in the underwater environment, the global features and local features are respectively started, and a double-branch network is used to process color feature and contrast feature related information respectively, so as to realize comprehensive repair of underwater image brightness, contrast and color on three levels.

[0005] The present application is implemented by the following technical scheme: a multi-branch underwater image enhancement system for non-uniform illumination and color distortion comprises:

[0006] a dataset for storing underwater images, model training and testing;

[0007] a brightness correction module, configured to receive the original image, convert the original image into an L channel of a CIELab color space, further decompose the L channel into a reflection component R and an illumination component I based on a Retinex theory, perform brightness correction by using a double-encoder-decoder structure, and output an image P after brightness correction;

[0008] a dual-branch underwater image enhancement model GLNet based on global-local features, configured to receive the image P, start from global features and local features respectively, use a dual-branch network to process color feature and contrast feature related information respectively, comprehensively repair the underwater image from three aspects of brightness, contrast and color, and realize image enhancement.

[0009] The dataset uses an underwater comprehensive image dataset containing underwater real images and underwater synthetic images.

[0010] The brightness correction module includes three parts: a Retinex decomposition network, an illumination correction network ICNet, and a brightness enhancement network.

[0011] The Retinex decomposition network is configured to receive the L channel and decompose the L channel into the reflection component R and the illumination component I.

[0012] The illumination correction network ICNet is configured to restore the illumination component I affected by an auxiliary light source to an illumination component I' not affected by the auxiliary light source.

[0013] The brightness enhancement network uses a U-Net network as the brightness enhancement network, and realizes enhancement of overall brightness of the underwater image by adjusting the L channel.

[0014] The Retinex decomposition network includes five cascaded 3x3 convolution layers and a nonlinear activation function, and the nonlinear activation function uses ReLU.

[0015] The encoder in the illumination correction network ICNet includes an encoder R and an encoder I, the encoder R takes the reflection component R as input, the encoder I takes the illumination component I as input, and the encoder R and the encoder I gradually expand the receptive field by gradually downsampling operation, capture multi-scale context information from local to global, and realize feature extraction.

[0016] The decoder finally obtains the brightness-corrected illumination component I' by layer-by-layer reconstruction.

[0017] Each stage of the decoder introduces a feature-level linear modulation module FiLM, which utilizes features F from the encoder R path.R and features F in the encoder I path I Generating a pair of affine transformation parameters and .

[0018] The bottleneck layer of the dual encoder-decoder introduces a spatial attention module SABlock, and the bottleneck layer features of the illumination component I and the reflectance component R are fused and sent to the spatial attention module SABlock to obtain a spatial attention matrix.

[0019] The Retinex decomposition network adopts a loss function L cons for measuring the difference between the reflectance components with and without an auxiliary light source;

[0020] The loss function L rec is used to measure the consistency before and after decomposition;

[0021] The illumination correction network ICNet adopts a loss function L ICNet .

[0022] The dual-branch underwater image enhancement model GLNet based on global-local features receives an input image P, which is first decomposed into a global color mean representing the overall color bias and a local centered image P containing texture details centered ;

[0023] The dual-branch underwater image enhancement model GLNet based on global-local features includes an upper branch and a lower branch;

[0024] The upper branch receives a multi-dimensional statistical feature vector representing global color information as input, and through a lightweight multi-layer perceptron network, predicts the gain g and bias parameter b for color correction, and then calculates the corrected new target color mean ;

[0025] The output end of the upper branch is designed to have a quantifiable guide signal ;

[0026] The lower branch receives the centered image as input and receives the guide signal output by the upper branch ;

[0027] The lower branch introduces a channel attention module CABlock, which acts on the guide signal to generate an attention weight for each channel, and through dynamic weighting, the channel attention module CABlock realizes adaptive fusion of the global color prior;

[0028] The dual-branch underwater image enhancement model GLNet based on global-local features linearly adds the outputs of the upper and lower branches and then outputs.

[0029] The dual-branch underwater image enhancement model GLNet based on global-local features adopts a linear combination of a loss function L l and a perception loss L per as a total loss function L total .

[0030] Compared with the prior art, the present application has the following beneficial effects:

[0031] The present application aims to solve the problem of non-uniform illumination caused by the auxiliary light source of the underwater ROV. The L channel of the CIELab color space sensitive to the illumination feature is processed, the L channel is further decomposed into a reflection component R and an illumination component I based on the Retinex theory, and a dual-encoder-decoder structure is used for brightness correction to repair the brightness distortion problem caused by the auxiliary light source. On this basis, in order to solve the two problems of inherent color distortion and contrast reduction in the underwater environment, the global feature and the local feature are respectively started, and the dual-branch network is used to process the color feature and the contrast feature related information respectively, so as to realize the comprehensive repair of the brightness, contrast and color of the underwater image on three levels.

[0032] The present application proposes a dual-branch underwater image enhancement model GLNet based on global-local features, which aims to solve the brightness distortion problem and solve the classic color distortion and contrast reduction problems in the underwater degradation. GLNet effectively realizes the efficient decoupling of the color distortion problem and the contrast reduction problem through the establishment of the upper and lower double layers, so as to realize targeted processing. The upper branch analyzes the multi-dimensional statistical feature vector containing RGB and CIELab space information through a lightweight multilayer perceptron, accurately predicts the gain and bias for color correction, and efficiently corrects the global color offset. The lower branch is a feature extraction network based on the idea of dense reuse, which focuses on processing the centralized image containing only texture details. Its characteristics lie in the innovative introduction of the color prior of the upper layer output as a guide signal, and the adaptive fusion through the channel attention module CABlock. The network adopts a dense connection strategy, reuses the outputs of multiple core processing modules, and sends them to the final reconstruction module. This design not only realizes the direct reuse of deep and shallow features of the network, significantly enhances the transmission efficiency of information flow, effectively alleviates the gradient disappearance problem, and more ensures that the network can deeply capture and restore the complex texture and details of the image under the accurate guidance of the color prior. Finally, through the cooperative work of the upper and lower branches, GLNet realizes excellent enhancement effect while maintaining light weight.

[0033] This invention effectively solves the intrinsic degradation problems (color distortion and reduced contrast) caused by the underwater environment itself and the secondary degradation problems (brightness distortion caused by non-uniform illumination) caused by the auxiliary light source carried by the ROV through a phased and decoupled processing logic. First, through the brightness correction branch, the light spot artifacts caused by the auxiliary light source are accurately removed, so that the underwater degraded image with the auxiliary light source is restored to the underwater degraded image without AL. On this basis, the dual-branch network GLNet solves the color distortion and reduced contrast problems in the underwater degraded image without the auxiliary light source, and achieves the final restoration. By decomposing a complex restoration task into multiple well-defined and relatively independent sub-tasks, this system not only reduces the learning difficulty of a single model, but also achieves synergistic optimization and precise breakthrough of multiple degradation problems through orderly guidance between stages, thereby achieving enhanced fidelity far exceeding that of a single end-to-end network. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of an underwater composite image;

[0035] Figure 2 This is a schematic diagram of the brightness correction process;

[0036] Figure 3 This is a diagram of the Retinex decomposed network architecture;

[0037] Figure 4 This is a diagram of the illuminance correction network architecture;

[0038] Figure 5 This is the architecture diagram of the spatial attention mechanism SABlock;

[0039] Figure 6 This is a diagram of the GLNet network architecture, a two-branch underwater image enhancement model based on global-local features.

[0040] Figure 7 This is the architecture diagram of the CABlock channel attention module;

[0041] Figure 8 This is the overall network architecture diagram of the system in this application;

[0042] Figure 9 This is a qualitative comparison chart of the underwater real image test dataset from Example 2;

[0043] Figure 10 Example 2: Qualitative comparison of underwater synthetic image test dataset. Detailed Implementation

[0044] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0045] Embodiment 1

[0046] With reference to Figure 8 The multi-branch underwater image enhancement system for non-uniform illumination and color distortion includes a data set, a brightness correction module, and a global-local feature-based double-branch underwater image enhancement model GLNet.

[0047] The data set is used to store underwater images, and to train and test the model.

[0048] Specifically, the data set adopts an underwater comprehensive image data set containing underwater real images and underwater synthetic images.

[0049] Under the influence of an auxiliary light (AL), underwater images usually exhibit significant non-uniform illumination characteristics, specifically, the center region of the light source is overexposed, and the brightness of the edge region sharply decays, which seriously affects the color distribution and visual quality of the underwater images. In order to effectively solve the brightness distortion problem caused by the auxiliary light, it is necessary to decouple the brightness information from the complex color information and perform targeted processing. However, in the traditional RGB color space, the color information in the three channels is highly coupled with the brightness information, and it is difficult to independently adjust the brightness without changing the original color distribution. In contrast, the L channel in the CIELab color space can decouple the brightness feature information alone, realizing the independent processing of the brightness feature.

[0050] Thus, for the underwater synthetic image, the present embodiment adopts an underwater synthetic image dataset UIDW auxiliary light source. Unlike traditional underwater image enhancement methods based on algorithm engineering and physical models, underwater image enhancement methods based on deep learning aim to automatically extract features from training data and learn nonlinear mapping functions, which also leads to underwater image enhancement methods based on deep learning relying on a large amount of underwater image data. However, due to the complex marine environment, it is difficult to obtain underwater images in real marine environments. This limitation has prompted the development of underwater image synthesis technology. Underwater image synthesis technology converts high-quality images on land into degraded images with underwater environmental characteristics through underwater imaging physical models or style transfer technology, and this technology has been widely used in the field of underwater image enhancement. In addition, underwater synthetic images can customize the degree of degradation and the type of degradation, and are therefore commonly used for targeted model training. Compared with other underwater synthetic datasets, the dataset adopted in the present application covers underwater synthetic images under the influence of auxiliary light sources, as shown in Figure 1 The first row of images is a synthetic image without auxiliary light source, and the second row of images is a synthetic image under the influence of auxiliary light source. As can be seen, the underwater images synthesized in the UIDW auxiliary light source cover the degradation problems such as color distortion and contrast reduction commonly seen in marine environments. In addition, due to the non-uniform illumination caused by the auxiliary light source, the original feature distribution of the underwater synthetic image is further disturbed, and the corresponding underwater image enhancement is more challenging.

[0051] The brightness correction module is configured to receive the original image, convert the original image into an L channel of a CIELab color space, further decompose the L channel into a reflection component R and an illumination component I based on the Retinex theory, perform brightness correction using a double-encoder-decoder structure, and output an image P without auxiliary light source after brightness correction.

[0052] Based on this, referring to Figure 2 , after determining the processing object, the brightness correction module can further decompose the L channel, and the decomposition formula is as follows:

[0053] ;

[0054] wherein L represents the L channel to be processed, and R and I represent the reflection component and the illumination component, respectively. The reflection component R describes the inherent characteristics of the target object, and the illumination component I represents various brightness on the object. Considering that the reflection component R usually remains unchanged under different brightness conditions, it can be considered that the auxiliary light source mainly affects the illumination component I. Therefore, by correcting the illumination component, the non-uniform illumination effect of the auxiliary light source on the underwater image can be effectively removed. The brightness correction module includes three parts: a Retinex decomposition network, an illumination correction network ICNet, and a brightness enhancement network;

[0055] The Retinex decomposition network is used to receive the L channel, decompose the L channel into a reflection component R and an illumination component I;

[0056] The illumination correction network ICNet is used to restore the illumination component I affected by the auxiliary light source to the illumination component I' not affected by the auxiliary light source;

[0057] Considering that the underwater image usually has a degradation problem of reduced brightness, the brightness enhancement network, which adopts a U-Net network as the brightness enhancement network, adjusts the L channel to enhance the overall brightness of the underwater image, so as to finally realize the processing flow for the degradation problem related to the brightness of the underwater image under the influence of the auxiliary light source.

[0058] The L channel is input into the Retinex decomposition network to estimate the illumination component I and the reflection component R. Further, the Retinex decomposition network adopts a lightweight fully convolutional network, and the network architecture is as shown in Figure 3 The Retinex decomposition network includes five cascaded 3x3 convolutional layers and a nonlinear activation function, and the nonlinear activation function adopts ReLU.

[0059] ReLU is used as the main nonlinear activation function. It is worth noting that, in order to avoid the information loss and artifact problems that may be introduced by the change of feature map size, the convolutional layers of the entire Retinex decomposition network are set with stride=1 and padding=1, so as to ensure that the spatial resolution of the feature map is always consistent with the input during the entire forward propagation process.

[0060] Referring to Figure 4 , the illumination correction network ICNet adopts a U-Net-like encoder-decoder structure; further, the encoder includes two encoders, an encoder R and an encoder I, the encoder R takes the reflection component R as input, and the encoder I takes the illumination component I as input, the encoder R and the encoder I gradually expand the receptive field through gradual downsampling operations, so as to be able to capture multi-scale context information from local to global, which is conducive to the network to understand the illumination distribution of the entire scene, and finally realize feature extraction;

[0061] The decoder finally obtains the brightness-corrected illumination component I' through layer-by-layer reconstruction. Similar to the U-Net structure, the original image is ensured to be effectively repaired in the reconstruction process through the jump connection, so that the fine illumination texture not affected by the auxiliary light source can be effectively repaired in the reconstruction process.

[0062] To accurately eliminate the spot artifact introduced by the auxiliary light source, the application designs a targeted illumination correction network ICNet to obtain the illumination component I' not affected by the auxiliary light source through two components output by the Retinex decomposition network. The network architecture of the illumination correction network ICNet is as shown in Figure 4 Unlike the traditional network which only relies on a single illumination component for correction, the illumination correction network ICNet adopts a dual-flow input and multi-mechanism collaborative architecture design. First, the illumination correction network ICNet introduces the reflection component R as auxiliary input. Since the reflection component R only reflects the inherent properties of objects in the scene, and the spot formed by the auxiliary light source is usually smooth, lacking of physically structured brightness abnormal area. Inputting the reflection component R as content prior into the illumination correction network ICNet can help the network learn to distinguish which is the brightness change of the object itself and which is the artifact caused by the auxiliary light source, so as to remove the spot introduced by the auxiliary light source while retaining the natural lighting details consistent with the scene structure to the maximum extent. In addition, in order to more finely utilize the guiding information of the reflection component R, a feature-level linear modulation module FiLM is introduced at each stage of the decoder. The feature-level linear modulation module FiLM utilizes the features F R from the encoder R path and the features F I in the encoder I path to generate a pair of affine transformation parameters and . The modulation process is represented as follows:

[0063] ;

[0064] wherein, and are the scaling and bias vectors learned from F R by a small neural network, denotes element-wise multiplication, and this mechanism realizes the adaptive dynamic adjustment of the illumination component I features. Thus, the structural information of R is deeply and dynamically integrated into each step of the reconstruction process of I, thereby realizing fine guidance and correction.

[0065] The bottleneck layer of the dual-encoder-decoder introduces a spatial attention module SABlock, and the network architecture is as shown in Figure 5 . Considering that the bottleneck layer usually contains highly abstract semantic information, it is the best position to identify global lighting artifacts. After the bottleneck layer features of the illumination component I and the reflection component R are fused and sent into the spatial attention module SABlock module, the spatial attention matrix is obtained. The process is represented as follows:

[0066] ;

[0067] ;

[0068] wherein, , are bottleneck layer features of the illumination component I and the reflection component R respectively, is the feature after channel concatenation of the two, represents element-wise multiplication. The spatial position of the auxiliary light source light spot can be adaptively given a higher weight, guiding the network to concentrate more computing resources for correcting these high weight areas, realizing spatial adaptive focusing of the correction process. is the output feature map after the spatial attention SABlock guidance.

[0069] Since the reflection component R is not affected by the auxiliary light source, its value is considered unchanged in the presence and absence of the auxiliary light source, so the Retinex decomposition network adopts the loss function L cons for measuring the difference between the reflection component in the presence and absence of the auxiliary light source;

[0070] Since the Retinex decomposition network is a process of mutual inverse for the illumination component I and the reflection component R, the information before and after Retinex decomposition should be consistent, so the loss function L rec is used to measure the consistency before and after decomposition;

[0071] The illumination correction network ICNet adopts the loss function L ICNet . For the illumination correction network ICNet, the label during training is the underwater image without the influence of the auxiliary light source, in order to describe the effect of brightness correction, the illumination component I output by ICNet should be close to the L channel of the label after recombination with the reflection component R. The loss function L ICNet of ICNet is as follows:

[0072] ;

[0073] wherein, is the illumination component without the influence of AL.

[0074] The final original image is output by the brightness correction module to obtain the image P, which is the auxiliary light source-free image after brightness correction.

[0075] The dual-branch underwater image enhancement model GLNet based on global-local features receives the image P, and starts from global features and local features respectively, adopts a dual-branch network to process color feature and contrast feature related information respectively, to realize comprehensive repair of underwater image brightness, contrast and color three levels, and then realize image enhancement.

[0076] Traditional deep learning based underwater image enhancement methods usually use a single end-to-end network to solve both the global color distortion and the local contrast reduction of underwater images at the same time. However, this mixed processing method not only makes the model architecture more complex and increases the computational burden, but also causes mutual influence when processing multiple features at the same time, that is, when processing spatial feature information such as texture, the color feature will be affected, resulting in impure color correction. Therefore, the present application proposes a dual-branch underwater image enhancement model GLNet based on global-local features, which separates the processing of global color problems and local detail problems, avoids the interference of local texture on global color correction, and makes the processing more accurate.

[0077] Most underwater images are dominated by relatively single and uniform color distribution, so the global average of underwater images can be considered to represent the overall color information. Based on this, the network architecture of the dual-branch underwater image enhancement model GLNet based on global-local features is as shown in Figure 6 .

[0078] The dual-branch underwater image enhancement model GLNet based on global-local features receives an input image P, which is first decomposed into a global color mean value representing overall color deviation and a local centered image P containing texture details centered ; wherein the calculation formula of the centered image P centered is as follows:

[0079] .

[0080] The dual-branch architecture has its own functions. Further, the dual-branch underwater image enhancement model GLNet based on global-local features includes an upper branch and a lower branch.

[0081] The upper branch focuses on the statistical features representing the global color. The upper branch receives a multi-dimensional statistical feature vector (RGB mean / variance, CIELab ab mean / variance) representing the global color information as input, and accurately predicts the gain g and bias parameter b for color correction through a lightweight multi-layer perceptron (MLP) network. Using these two parameters, the network can calculate the corrected new target color mean value , and the calculation formula is:

[0082] ;

[0083] The upper branch effectively solves the problem of global color distortion. In order to ensure that the repaired color information is not damaged when processing the lower local contrast reduction problem, a quantifiable guide signal is designed at the output end of the upper branch, and the calculation formula is as follows:

[0084] ;

[0085] The lower branch focuses on contrast and detail restoration in the spatial domain. It takes the centered image as input and receives a guiding signal from the upper branch. However, if the global boot signal is simply... Applying this to all local features could potentially damage the restored color information. Therefore, a channel attention module (CABlock) is introduced during the reception of the guiding signal. Its network architecture is as follows: Figure 7 As shown. The channel attention module CABlock acts on the guiding signal. This allows for the generation of an attention weight for each channel. Through this dynamic weighting, the channel attention module CABlock achieves adaptive fusion of global color priors, enabling the network to maintain overall tone correction consistency while making differentiated fine-tuning for different regions and feature channels. This enhances contrast while maximizing the preservation of local original color information, avoiding overcorrection and artifacts.

[0086] The GLNet dual-branch underwater image enhancement model, based on global-local features, combines the outputs of the upper and lower branches (i.e., the mean color of the new target after correction by the upper branch). The components (after contrast correction of the lower branch) are linearly added together and then output to obtain the final restored image with better visual effect.

[0087] To fully train the proposed global-local feature-based dual-branch underwater image enhancement model GLNet, a loss function L is used. l and perceived loss L per The linear combination of L is used as the total loss function. total The formula is as follows:

[0088] ;

[0089] In this embodiment, the hyperparameter for measuring the weight is set to 0.1.

[0090] To reconstruct the underwater image, a loss function L is applied. l To maintain the clarity of visual quality boundaries and details. l This can be represented as the distance between the reconstructed image and the reference label, and its calculation formula is as follows:

[0091] ;

[0092] In the formula, N is the number of pixels in the reconstructed image. the i-th pixel value of the output image of the global-local feature based dual-branch underwater image enhancement model GLNet, the i-th pixel value of the output image of the global-local feature based dual-branch underwater image enhancement model GLNet,

[0093] only loss function L l may cause loss of details in the image, so a perceptual loss is combined to enhance the details. The perceptual loss is calculated using a VGG-16 network pre-trained on the ImageNet dataset. represents the 7th convolutional layer in the VGG-16 network, and measures the difference between the reconstruction result J and the reference label R as follows:

[0094] .

[0095] Here, H and W are the height and width of the image, respectively, and represent the pixel values of the reconstruction result J and the reference label R at the i-th row and j-th column pixel, respectively.

[0096] The method first converts the picture from the traditional RGB color space to the CIELab color space and focuses on the L channel sensitive to brightness to decouple the brightness information and the color information and avoid affecting the original color distribution when adjusting the brightness. Subsequently, based on the Retinex theory, the L channel is further decomposed into an illumination component I and a reflection component R through a lightweight network to separate the I component closely related to the light source characteristics, making the processing object more specific and specific to the I component adjustment. For this purpose, the present application designs an illumination correction network ICNet using a unique dual-encoder-decoder architecture. ICNet innovatively accepts both the illumination I and the reflection R as inputs, guides the model to modify I by taking R as a content prior, to distinguish the inherent light and shadow of the image scene from the AL caused spot artifacts. In addition, ICNet also incorporates a spatial attention module SABlock and a feature modulation module FiLM to ensure that the natural light details consistent with the scene structure are maximally preserved while the auxiliary light source spots are efficiently removed. Through the above effective strategies and the synergistic effect of each strategy, the present application effectively solves the brightness distortion problem of underwater images.

[0097] Embodiment 2

[0098] In order to fully train the system model proposed in the application, the embodiment adopts an underwater comprehensive image dataset containing underwater real images and underwater synthetic images for model training and subsequent model testing. Among them, the underwater synthetic images adopt the UIDW auxiliary light source dataset mentioned in embodiment 1, which includes a real land image dataset, an underwater synthetic image dataset without auxiliary light source and an underwater synthetic image dataset with auxiliary light source. The UIDW auxiliary light source is divided into 8000 training groups and 2000 test groups, each group containing corresponding “land image-underwater synthetic image without auxiliary light source-underwater synthetic image with auxiliary light source”, which can solve the problem of model underfitting caused by insufficient underwater images. However, it is difficult to simulate the complex conditions of underwater real environment in underwater synthetic environment. Therefore, the embodiment selects three classic underwater real image datasets UIEB, EUVP and LUSI for model training. Specifically, 800 UIEB dataset images, 2000 EUVP dataset images and 2000 LSUI dataset images are selected for training.

[0099] The system model proposed in the application is a multi-branch network structure, which is also a multi-stage network structure. Its training is divided into multiple steps. Specifically, the Retinex decomposition network in the brightness correction branch, the illumination correction network ICNet and the brightness enhancement network need to be trained first. The trained model is no longer trained in subsequent model training, but is directly used for inference according to the trained weight. The loss function L cons , L rec and L ICNet proposed in embodiment 1 are the loss functions for model training in the brightness correction module. After the brightness correction module training is completed, the network model for color distortion correction and contrast correction is a complete structure, and the overall weight is lightened. It can be directly trained. The total loss function L total in embodiment 1 is the training loss function of the double-branch network GLNet. After the above process, the overall training process of the system model is completed, and the repaired underwater image can be obtained.

[0100] The application adopts the PyTorch framework to realize the training and testing of the model, and both are realized on the NVDIA RTX 4090 GPU. Among them, the batch size is set to 64, the AdamW optimizer is adopted, and the cosine annealing learning rate scheduler is used. The initial learning rate is set to , and the minimum learning rate is Since the illumination correction network ICNet is relatively complex, the training round of the ICNet is set to 500, and the training round of other models is set to 100. In the data preprocessing stage, the image size is uniformly adjusted to 256*256 pixels, and data enhancement techniques such as cropping, flipping, rotating, transposing and scaling are applied to enhance the training effect.

[0101] Referring to Figure 8 , the overall architecture of the system model proposed by the application is shown, and the core network is the illumination correction network ICNet and the double-branch network GLNet. In order to visualize the change process of the feature map in the two core architectures, the detailed structure parameters of the two network architectures are listed in Table 1 and Table 2.

[0102] Table 1 ICNet structure parameters

[0103]

[0104] Table 2 GLNet structure parameters

[0105]

[0106] Based on the multiple degradation problems existing in underwater images, the application proposes corresponding correction branches, among which, compared with the two classic degradation problems of color distortion and contrast reduction, the application also focuses on solving the non-uniform illumination problem caused by the ROV auxiliary light source. In order to prove the generalization ability of the system model, the embodiment is tested on the underwater comprehensive image dataset. Specifically, according to different color deviation conditions (blue, green, blue-green), 200 representative underwater real images are selected from 3 real datasets, and 200 underwater images are selected from the synthetic dataset for testing the repair effect of the application on traditional underwater degradation problems. In addition, in order to verify the repair effect of the application on the non-uniform illumination problem caused by the auxiliary light source, 100 synthetic images with different auxiliary light source conditions are selected from the UIDW auxiliary light source dataset for testing.

[0107] In order to verify the performance of the system model, the system model is compared with 7 kinds of traditional underwater image enhancement methods based on algorithm engineering and physical model, and the qualitative comparison results are shown in Figure 9 and Figure 10 . As shown in Figure 9 and Figure 10 , the images in (a), (b), (c), (d), (e), (f), (g), (h), (i) and (j) are the original, reference label, CLAHE, DCP, GC, ICM, UCM, UDCP, ULAP and the corresponding images of the application, respectively. Among them, Figure 9is the enhancement effect of eight underwater image enhancement methods on the underwater real image test data set, and the selected images cover the common color distortion and contrast reduction problems of underwater degradation. It can be seen that, compared with the other seven classical algorithms, the system has sufficient advantages in dealing with color distortion and contrast reduction. The repaired image does not appear red artifacts and overcompensation phenomenon. In addition, the texture details are further enhanced. Figure 10 The focus is on the enhancement effect of eight underwater image enhancement methods on the underwater synthetic image test data set with auxiliary light source. The selected images are affected by the non-uniform illumination of the auxiliary light source, and local brightness distortion occurs, which seriously affects the visual effect. It can be more obviously seen that, for this brightness distortion problem caused by underwater ROV, traditional underwater image enhancement methods cannot handle it at all, while the system effectively solves the brightness distortion problem according to the brightness correction branch, and the repaired image has good effect in terms of color naturalness, contrast, saturation and other aspects. Figure 9 and Figure 10 It is fully proved that the system has excellent processing capability in the face of non-uniform illumination and color distortion problems.

[0108] However, the qualitative comparison result is greatly affected by subjectivity, and subjective evaluation alone does not have sufficient objectivity. Therefore, the invention also introduces the classical objective evaluation index in the field of underwater image enhancement to evaluate the invention method and seven comparison methods. The quantitative evaluation index includes two reference evaluation indexes: peak signal-to-noise ratio PSNR and structural similarity SSIM, and two non-reference evaluation indexes: UIQM and UCIQE. Among them, PSNR and SSIM are used to calculate the similarity between the model repaired image and the reference label from multiple dimensions, so the larger the SSIM and PSNR, the stronger the model repaired image capability. UIQM and UCIQE have been widely used in evaluating underwater image quality. The higher the UIQM value, the more the image conforms to the visual perception of the human eye; the higher the UCIQE value, the better the balance of the image in terms of chroma, saturation and contrast. The quantitative evaluation results are shown in Table 3. The upward arrow on the right side of the evaluation index represents that the evaluation result is proportional to the numerical value, that is, the larger the numerical value, the better the enhancement effect according to the index. It can be seen that the system is significantly better than the other seven traditional underwater image enhancement methods in the four evaluation indexes.

[0109] Table 3 Comparison results of objective indexes of each method on the test data set

[0110]

[0111] The above merely describes optional embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structural transformation made according to the content of the present application, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion, characterized in that, include: The dataset is used to store underwater images for model training and testing. The brightness correction module receives the original image, converts it into the L channel of the CIELab color space, further decomposes the L channel into a reflection component R and an illuminance component I based on Retinex theory, performs brightness correction using a dual encoder-decoder structure, and outputs the brightness-corrected image P. The GLNet underwater image enhancement model, based on global-local features, takes an image P and processes color and contrast features from both global and local features. It uses a dual-branch network to comprehensively repair the brightness, contrast, and color of the underwater image, thereby achieving image enhancement. The dual-branch underwater image enhancement model GLNet based on global-local features receives an image P, which is first decomposed into a global color mean representing the overall color cast and a local centralized image P containing texture details centered ; The GLNet underwater image enhancement model, based on global-local features, consists of an upper branch and a lower branch. The upper layer branch receives a multi-dimensional statistical feature vector representing global color information as input, and predicts gain g and bias parameter b for color correction through a light multi-layer perception network, and then calculates the corrected new target color mean ; Output of upper layer branch designed quantifiable pilot signal ; The lower branch takes the central image as input and receives the guidance signal output by the upper branch ; The lower layer branch introduces a channel attention block CABlock, which acts on the guided signal An attention weight is generated for each channel, and the channel attention block CABlock realizes adaptive fusion of the global color prior through dynamic weighting. The GLNet underwater image enhancement model, based on global-local features, linearly adds the outputs of the upper and lower branches before outputting the final result.

2. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 1, wherein, The dataset is an underwater composite image dataset that includes both real underwater images and synthetic underwater images.

3. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 1, wherein, The brightness correction module consists of three parts: a Retinex decomposition network, an illuminance correction network ICNet, and a brightness enhancement network; The Retinex decomposition network is used to receive the L channel and decompose the L channel into a reflection component R and an illuminance component I. The illuminance correction network ICNet is used to restore the illuminance component I affected by the auxiliary light source to the illuminance component I' unaffected by the auxiliary light source; The brightness enhancement network uses the U-Net network as the brightness enhancement network, and enhances the overall brightness of the underwater image by adjusting the L channel.

4. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 3, wherein, The Retinex decomposition network comprises five cascaded 3x3 convolutional layers and a nonlinear activation function, wherein the nonlinear activation function is ReLU.

5. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 4, wherein, The encoder in the illumination correction network ICNet includes encoder R and encoder I. Encoder R takes the reflection component R as input, and encoder I takes the illumination component I as input. Encoder R and encoder I gradually expand the receptive field through gradual downsampling operation, and capture multi-scale contextual information from local to global to achieve feature extraction. The decoder reconstructs layer by layer to finally obtain the illuminance component I' after brightness correction.

6. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 5, wherein, Each stage of the decoder introduces a feature-level linear modulation module FiLM that utilizes features F R and F in the encoder I path I to generate a pair of affine transformation parameters and .

7. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 5, wherein, The bottleneck layer of the dual encoder-decoder introduces a spatial attention module SABlock. After the bottleneck layer features of the illuminance component I and the reflection component R are fused, they are sent to the spatial attention module SABlock to obtain the spatial attention matrix.

8. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 7, wherein, The Retinex decomposition network adopts a loss function L cons For measuring the difference of the reflection component with and without an auxiliary light source; The loss function L is used rec for measuring the consistency before and after decomposition; The illumination correction network ICNet adopts a loss function L ICNet .

9. The multi-branch underwater image enhancement system oriented to non-uniform illumination and color distortion of claim 1, wherein, A dual-branch underwater image enhancement model GLNet based on global-local features adopts a linear combination of loss functions L l and perception loss L per as the total loss function L total .

Images