A deep-water image enhancement method based on multi-color space coupling

Abstract

The present invention relates to the field of underwater image enhancement technology, and provides a deep-water image enhancement method based on multi-color space coupling, including determining an optimal channel, a moderately attenuated channel, and a heavily attenuated channel based on pixel average values ​​in the RGB color space, performing asymmetric histogram clipping pre-correction on the optimal channel, and designing a loss function to make the attenuated channel approach the optimal channel through iterative compensation; in the Lab color space, using an unsupervised pixel clustering model PCM to cluster and segment the luminance component L into several pixel blocks, performing local contrast enhancement and guided filtering-based noise reduction processing on each pixel block; finally, using an adaptive exponential function to perform nonlinear stretching on the chrominance components a and b to improve color naturalness. The present invention effectively solves the limitations of single color space processing of traditional methods through a multi-color space collaborative enhancement mechanism, and improves the color correction accuracy, brightness enhancement and noise reduction balance, and naturalness optimization effect of deep-water images.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image enhancement, and in particular to a deep-water image enhancement method based on multi-color space coupling. Background Art

[0002] Optical imaging technology plays a vital role in the information collection process in underwater environments. In marine science research, underwater optical imaging technology provides visual data support for scientific research activities such as seabed geological exploration and biological population monitoring. In engineering applications, underwater optical imaging technology provides a visual system for autonomous underwater vehicles (AUVs) to achieve navigation, positioning, and intelligent decision-making. However, due to the wavelength-dependent selective absorption and scattering effects of the water medium, underwater images commonly suffer from degradation phenomena such as color distortion, brightness attenuation, and blurred details, which seriously hinder the effective extraction of visual information.

[0003] Due to wavelength-dependent selective absorption and scattering of light by water, deep-water images commonly exhibit degradation phenomena such as color distortion, brightness attenuation, and blurred detail. Long-wavelength red light is most easily absorbed, resulting in an overall bluish-green tint in the image. Short-wavelength blue-green light also attenuates when penetrating deep water, resulting in an overall darkening of the image and reduced contrast. Scattering effects also cause a loss of texture detail and weakening of high-frequency information. Current mainstream underwater image enhancement methods have significant limitations. Most methods focus on processing in a single color space, operating solely on the RGB space to address color distortion. However, the coupled luminance and chrominance components in RGB space make it difficult to independently optimize the luminance distribution. While the Lab space can separate luminance and chrominance components, it cannot effectively correct channel histogram shifts, making it difficult to balance color correction and brightness enhancement. The restored image is prone to residual color casts or brightness imbalances. Traditional color correction directly uses the optimal channel with the least attenuation to compensate for other channels. However, the optimal channel of deep-water images still contains extreme pixel values. Moreover, when using symmetrical histogram cropping, the marine environment disrupts the histogram symmetry, exacerbating the channel distribution shift, amplifying pre-correction errors, and resulting in color block accumulation after compensation.

[0004] In terms of channel compensation mechanism, fixed intensity compensation does not take into account the difference in pixel-level attenuation, and the global mean compensation formula uses a fixed coefficient, resulting in insufficient compensation for areas with heavier attenuation and overcompensation for areas with lighter attenuation, causing local detail loss or artificial artifacts, and the channel similarity is significantly lower than the natural image reference value. Traditional contrast enhancement uses global operations, and histogram equalization ignores local brightness feature differences, resulting in high-frequency noise being amplified and edge details being blurred. When chroma adjustment uses a linear stretching function, it is unable to adaptively process the dynamic range of the chroma component in the Lab space, resulting in abrupt color transitions near the neutral gray axis, loss of color levels in low-saturation areas, and chroma diffusion in highlight areas. Existing technologies have failed to collaboratively solve the coupled degradation problems of color distortion, brightness attenuation, blurred details, and loss of naturalness. The root cause lies in the lack of a collaborative enhancement mechanism for multiple color spaces, the lack of adaptive correction methods for asymmetric histograms of deep-water images, and the lack of local brightness feature clustering and noise suppression strategies.

[0005] Therefore, it is of great significance to design a deep-water image enhancement method that integrates the advantages of multiple spaces and has pixel-level adaptive capabilities. Summary of the Invention

[0006] To solve the problems existing in the background technology, the present invention provides a deep-water image enhancement method based on multi-color space coupling, comprising the following steps:

[0007] S1. Perform color distortion correction on deep-water images in the RGB color space: determine the optimal channel, moderately attenuated channel, and heavily attenuated channel based on the pixel average value; perform asymmetric histogram cropping pre-correction on the optimal channel; design a loss function and use iterative compensation to make the attenuated channel approach the optimal channel in terms of channel structure and pixel distribution;

[0008] S2. Clustering and enhancing the luminance component in the Lab color space: Convert the color-distortion-corrected image to the Lab color space; cluster the luminance component L into several pixel blocks using the unsupervised pixel clustering model PCM; perform local contrast enhancement and guided filtering-based noise reduction on each pixel block;

[0009] S3. Naturalness optimization in Lab color space: Use adaptive exponential function to perform nonlinear stretching on chromaticity components a and b to improve color naturalness.

[0010] Furthermore, the specific process of S1 includes:

[0011] S11. Determine the optimal channel: Calculate the average pixel values ​​of the three RGB channels and sort them from the largest to the smallest average values ​​to determine the optimal channel. , medium attenuation channel , heavy attenuation channel ; The pixel average calculation formula is:

[0012] ;

[0013] in, is the height of the input deep-water image; is the width of the input deep-water image; Indicates channel Pixel value of Indicates the red, green, or blue channel; is the summation operator; and are the row and column indices of the pixel respectively;

[0014] S12, optimal channel pre-correction: using the histogram mode as the dividing point, cropping a fixed proportion of pixels starting from the beginning on the left side of the mode to obtain , cropping a fixed ratio of pixels from the tail on the right side of the mode to obtain ; The optimal channel Smaller than The pixel value is set to , greater than The pixel value is set to , to remove extreme pixel values;

[0015] S13, attenuation channel compensation: Design loss function and use iterative compensation formula to compensate for moderate attenuation channel and heavy attenuation channels Compensation is performed, and the iteration stopping condition is that the loss function value is less than 0.01 or the number of iterations reaches 200; the loss function and compensation formula are:

[0016] ;

[0017] in, The optimal channel The average pixel value of Attenuation channel or The average value of pixels; is the reference channel similarity of natural images; express and similarity; Indicates a moderately attenuated channel or a heavily attenuated channel;

[0018] ;

[0019] in, is the pixel value of the attenuation channel after compensation; is the original attenuation channel pixel value; Used for global compensation control; Used for local compensation control; is the optimal channel pixel value after pre-correction.

[0020] Furthermore, the specific process of S2 includes:

[0021] S21, brightness component normalization: linearly map the brightness component L from the interval [0,100] to [0,1];

[0022] S22. Pixel Clustering Model PCM Processing: The luminance component L is clustered and segmented into N pixel blocks using the unsupervised pixel clustering model PCM. PCM includes an input layer, a feature extraction layer, and an output layer. The feature extraction layer consists of two encoder-decoder structures, each encoder or decoder consisting of a 1x1 convolutional layer, a batch normalization layer, and an activation function linear structure. The maximum number of cluster categories N = 50, and the middle channel dimension mid = 2048. PCM outputs the pixel clustering probability space and through Function determines pixel category space :

[0023] ;

[0024] in, Represents clustering operation; is the nth pixel block; N = 50 is the maximum number of cluster categories;

[0025] ;

[0026] in, Dimension Pixel clustering probability space; The function returns the category index corresponding to the maximum probability;

[0027] S23, local brightness enhancement: for each pixel block Perform contrast enhancement based on box filtering and noise reduction based on guided filtering:

[0028] ;

[0029] in, is the enhanced pixel value; It is a box filter operation; Pixel block Pixel value of is the local window size of the box filter; Increase weight for high frequency components; and is an arithmetic operator; is the multiplication operator;

[0030] ;

[0031] in, is the pixel value after noise reduction; To guide the filtering operation; is the local window size of the guided filtering; is the regularization parameter.

[0032] Furthermore, the nonlinear stretching formula for the chromaticity components a and b in S3 is:

[0033] ;

[0034] in, is the enhanced chroma pixel value; is the original pixel value of chroma component a or b; is a natural constant; Used to control the intensity of chroma stretching.

[0035] The beneficial effects achieved by the present invention are:

[0036] First, the present invention adopts a multi-color space coupling mechanism to address the problem of deep-water image enhancement, breaking through the limitations of the traditional method of single color space. By solving the color distortion problem in the RGB space and processing the brightness and naturalness problems in the Lab space, synergistic enhancement is achieved. This spatial coupling strategy effectively utilizes the attribute advantages of different color spaces: the RGB space is convenient for channel distribution correction, while the Lab space can decouple the brightness and chromaticity components for independent optimization, thereby significantly improving the overall image quality while retaining color authenticity, and solving the comprehensive degradation problems of color offset, brightness attenuation and blurred details that are common in underwater images.

[0037] Second, the present invention carries out a triple optimization design for RGB color space processing. First, an asymmetric histogram clipping method is proposed to pre-correct the optimal channel. This method uses the histogram mode as the dividing point and clips a fixed proportion of pixels on both sides of the mode, which overcomes the defect of traditional symmetric clipping that relies on the symmetry of the histogram and avoids further deviation of the asymmetric distribution data caused by the marine environment. Secondly, a channel similarity loss function is introduced, and the high similarity between natural image channels is used as the optimization goal to guide the attenuation channel to approach the optimal channel in terms of pixel distribution and structural characteristics. Finally, an iterative compensation mechanism is designed to control the overall color cast correction through the global compensation term, and the local compensation term dynamically adjusts the compensation intensity according to the degree of pixel attenuation to form a closed-loop optimization. These three designs work together to enable the color correction process to accurately restore the natural color distribution and avoid color block accumulation or detail loss caused by insufficient or excessive compensation.

[0038] Third, in the Lab space processing, the present invention designs a pixel clustering model PCM and a local enhancement strategy for brightness component enhancement. The PCM model adopts an unsupervised dual encoder-decoder architecture, and for the first time realizes deep feature extraction through two differentiated dimensionality increase-dimensionality reduction operations: the first time extracts basic features in the original brightness space, and the second time mines abstract features in the pixel clustering probability space. This hierarchical feature processing mechanism greatly improves the clustering accuracy and ensures the accurate aggregation of pixels with similar brightness and texture. Based on the clustering results, a collaborative local enhancement of box filtering and guided filtering is adopted: the box filtering separates high and low frequency components to achieve targeted contrast enhancement, and the guided filtering uses its edge-preserving characteristics to suppress noise amplification. This combined strategy not only enhances the expression of local details, but also maintains the integrity of the edge structure, so that the global brightness enhancement process avoids the generation of artificial artifacts.

[0039] Fourth, for the naturalness optimization of chromaticity components, the present invention proposes an adaptive exponential function. This function performs nonlinear mapping with the neutral gray axis of Lab space as the symmetry center, and its exponential characteristics automatically adapt to the enhancement requirements of different chromaticity areas: strengthening the color level in low-saturation areas and suppressing chromaticity diffusion in highlight areas. This adaptive mechanism makes the color transition of subjects such as corals and fish smoother, while enhancing the color difference distinction of detail edges, effectively overcoming the detail blurring problem caused by traditional linear methods. Combined with the aforementioned color correction and brightness enhancement, it ultimately achieves a comprehensive improvement in color authenticity, brightness distribution, detail clarity, and visual naturalness, allowing the texture structure and boundary contours of complex underwater scenes to be clearly presented. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 : is a diagram of the color distortion phenomenon of a typical deep-water image in the UIEB dataset in Example 1, where (a) is a green-colored image and (b) is a blue-colored image.

[0041] Figure 2 This is a detail preservation diagram comparing the RGB three channels of the deep water image in Example 1.

[0042] Figure 3 3 is a schematic diagram comparing the traditional symmetrical cropping method in Example 1 and the asymmetrical histogram cropping method in this embodiment.

[0043] Figure 4 3 is a schematic diagram of the color distortion correction result of the RGB color space in Example 1.

[0044] Figure 5 This is a network architecture diagram of the pixel clustering model PCM in Example 1.

[0045] Figure 6 Schematic diagram of the PCM feature extraction layer in Example 1.

[0046] Figure 7 Schematic diagram of the PCM clustering result based on the luminance component L in Example 1.

[0047] Figure 8 It is the adaptive exponential function image in Example 1.

[0048] Figure 9 This is a schematic diagram of the final enhancement effect of the present invention in Example 1.

[0049] Figure 10 This is a comparison chart of the experimental results of different methods in Example 2 on the UIEB dataset. DETAILED DESCRIPTION

[0050] The technical solutions of the present invention will be described clearly and completely below in conjunction with the drawings in the present invention. In addition, the forms of the various structures described in the following embodiments are merely examples. The present invention is not limited to the various structures described in the following embodiments. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] The present invention designs a deep-water image enhancement method based on multi-color space coupling, which includes the following steps:

[0052] S1. Perform color distortion correction on deep-water images in the RGB color space: determine the optimal channel, the moderately attenuated channel, and the heavily attenuated channel based on the pixel average value; perform asymmetric histogram cropping pre-correction on the optimal channel; design a loss function and use iterative compensation to make the attenuated channel approach the optimal channel in terms of channel structure and pixel distribution;

[0053] S2. Clustering and enhancing the luminance component in the Lab color space: Convert the color-distorted image to the Lab color space; cluster the luminance component L into several pixel blocks using the unsupervised pixel clustering model (PCM); perform local contrast enhancement and guided filtering-based noise reduction on each pixel block;

[0054] S3. Naturalness optimization in Lab color space: Use adaptive exponential function to perform nonlinear stretching on chromaticity components a and b to improve color naturalness.

[0055] The specific process of S1 includes:

[0056] S11. Determine the optimal channel: Calculate the average pixel values ​​of the three RGB channels and sort them from the largest to the smallest average values ​​to determine the optimal channel. , medium attenuation channel , heavy attenuation channel ; The pixel average calculation formula is:

[0057] ;

[0058] in, is the height of the input deep-water image; is the width of the input deep-water image; Indicates channel Pixel value of Indicates the red, green, or blue channel; is the summation operator; and are the row and column indices of the pixel respectively;

[0059] S12, optimal channel pre-correction: using the histogram mode as the dividing point, cropping a fixed proportion of pixels starting from the beginning on the left side of the mode to obtain , cropping a fixed ratio of pixels from the tail on the right side of the mode to obtain ; The optimal channel Smaller than The pixel value is set to , greater than The pixel value is set to , to remove extreme pixel values;

[0060] S13, attenuation channel compensation: Design loss function and use iterative compensation formula to compensate for moderate attenuation channel and heavy attenuation channels Compensation is performed, and the iteration stopping condition is that the loss function value is less than 0.01 or the number of iterations reaches 200; the loss function and compensation formula are:

[0061] ;

[0062] in, The optimal channel The average pixel value of Attenuation channel or The average pixel value of is the reference channel similarity of natural images; express and similarity; Indicates a moderately attenuated channel or a heavily attenuated channel;

[0063] ;

[0064] in, is the pixel value of the attenuation channel after compensation; is the original attenuation channel pixel value; Used for global compensation control; Used for local compensation control; is the optimal channel pixel value after pre-correction.

[0065] The specific process of S2 includes:

[0066] S21, brightness component normalization: linearly map the brightness component L from the interval [0,100] to [0,1];

[0067] S22. Pixel Clustering Model PCM Processing: The luminance component L is clustered and segmented into N pixel blocks using the unsupervised pixel clustering model PCM. PCM includes an input layer, a feature extraction layer, and an output layer. The feature extraction layer consists of two encoder-decoder structures, each encoder or decoder consisting of a 1x1 convolutional layer, a batch normalization layer, and an activation function linear structure. The maximum number of cluster categories N = 50, and the middle channel dimension mid = 2048. PCM outputs the pixel clustering probability space and through Function determines pixel category space :

[0068] ;

[0069] in, Represents clustering operation; is the nth pixel block; N = 50 is the maximum number of cluster categories;

[0070] ;

[0071] in, Dimension Pixel clustering probability space; The function returns the category index corresponding to the maximum probability;

[0072] S23, local brightness enhancement: for each pixel block Perform contrast enhancement based on box filtering and noise reduction based on guided filtering:

[0073] ;

[0074] in, is the enhanced pixel value; It is a box filter operation; Pixel block Pixel value of is the local window size of the box filter; Increase weight for high frequency components; and is an arithmetic operator; is the multiplication operator;

[0075] ;

[0076] in, is the pixel value after noise reduction; To guide the filtering operation; is the local window size of the guided filtering; is the regularization parameter.

[0077] The nonlinear stretching formula for chromaticity components a and b in S3 is:

[0078] ;

[0079] in, is the enhanced chroma pixel value; is the original pixel value of chroma component a or b; is a natural constant; Used to control the intensity of chroma stretching.

[0080] In Example 1, the deep-water image enhancement method based on multi-color space coupling of the present invention is specifically described in combination with a specific implementation case.

[0081] Reference Figures 1-8In this embodiment, images collected from a certain deep underwater space are processed. The quality of underwater images is affected by both the inherent optical properties of water and the physical process of light propagation, resulting in obvious color distortion. Light waves of different wavelengths attenuate differently underwater. Red light with a longer wavelength is most easily absorbed by water, while blue-green light with a shorter wavelength has stronger penetrating power. However, in deep waters, blue and green light will also partially attenuate, resulting in deep-water images generally appearing blue-green. Figure 1 Several sets of typical deep-water images are shown based on the UIEB dataset.

[0082] In the RGB color space, the serious color distortion problem in deep-water images is essentially due to the different degrees of changes in the pixel histogram distribution of its three color channels. In order to reduce the impact of color distortion, it is necessary to compensate each channel to restore it to the standard color distribution under natural conditions. However, most traditional color distortion correction methods directly use the optimal channel (i.e., the channel with the least attenuation) to compensate other channels with greater attenuation, and use a fixed compensation intensity indiscriminately. This approach has the following limitations: (1) In actual situations, the optimal channel of deep-water images also has slight attenuation. Directly using the optimal channel without pre-correction is likely to further amplify the error. (2) The underwater imaging mechanism is complex. Compensating different pixels in different attenuation channels to the same degree may cause problems such as under-compensation or over-compensation.

[0083] For deep-water images in the RGB color space, the amount of detail loss in its three channels varies due to different degrees of attenuation. Figure 2 The three channels of deep-water images are displayed. The color channel marked with a red box is considered to be the channel with the best detail preservation, that is, the channel with the least attenuation, and is generally considered the optimal channel. By testing 100 randomly selected deep-water images from the UIEB dataset, it is found that the optimal channel is equivalent to the channel with the largest pixel average value. Therefore, the pixel average value can be used to quantitatively evaluate the attenuation degree of each channel in the underwater image. The calculation of the pixel average value is shown in formula (1).

[0084]

[0085] Among them, H and W are the height and width of the input deep water image, respectively. The average pixel value of each channel is calculated and the optimal channel is defined from large to small. , medium attenuation channel and heavy attenuation channels .

[0086] Before using the optimal channel to compensate the attenuated channel, the optimal channel needs to be pre-corrected. Deep-water images are affected by the ocean environment, and each channel is mixed with some extreme pixel values. Therefore, by removing the extreme pixel values ​​in the optimal channel, the optimal channel can be corrected. However, the traditional method of removing extreme pixel values ​​is to symmetrically crop a fixed proportion of pixels at the beginning and end of the histogram. Its effectiveness depends on the symmetry assumption of the histogram, and the symmetry of the histogram of deep-water images is generally destroyed. Therefore, the traditional method is not effective for processing deep-water images and may even aggravate the deviation of the channel. To this end, the present invention improves the traditional method as follows: taking the histogram mode as the dividing point, a fixed proportion of pixels starting from the beginning on the left side of the mode are taken as the demarcation point. , a fixed ratio of pixels starting from the tail on the right side of the mode is used as , and will be less than The pixel value is set to , greater than The pixel value is set to , to remove channel extreme values ​​and achieve the optimal channel Finally, histogram stretching is used to map the optimal channel interval to [0, 255]. Figure 3 A comparative diagram of the traditional method for removing extreme pixel values ​​and the improved method of the present invention is shown, which more intuitively demonstrates the difference between the two methods when processing asymmetric histograms.

[0087] In the optimal channel After pre-calibration, the attenuation channel can be adjusted based on the optimal channel. and After observing a large number of natural image color channels, we found that the three channels of high-quality natural images often have high similarity. By calculating the similarity of the channels of 500 natural images, we can find the average value of the channel similarity between the best channel of the natural image and the remaining two channels. The value is 92%, which is significantly higher than the similarity between the optimal channel and the attenuation channel in deep water images. Based on this finding, the present invention designs the following loss function to guide the attenuation channel to be similar to the optimal channel.

[0088]

[0089] in, and Represent the optimal channel With attenuation channel and The average pixel value. Used to measure the optimal channel With attenuation channel and similarity, is the target channel similarity, which is 0.92 here. The smaller the Loss value, the more similar the attenuated channel is to the optimal channel, that is, the color distribution of the deep-water image is closer to the natural situation.

[0090] For the attenuation channel, the present invention uses the following formula for compensation.

[0091]

[0092] Where, The difference between pixel average values ​​is used to achieve overall control of the attenuation channel approaching the optimal channel.

[0093] It is used to locally control the compensation of each pixel in the attenuation channel, ensuring that more compensation is provided to pixels with heavier attenuation and less compensation is provided to pixels with lighter attenuation, avoiding under-compensation and over-compensation problems.

[0094] Formula (2) and formula (3) respectively give the loss function and compensation formula of the attenuation channel. The compensation formula needs to be iterated multiple times until the loss function is less than the threshold, so that the attenuation channel is similar to the optimal channel in terms of channel structure and pixel distribution. For the compensated attenuation channel, the histogram stretching is used to map its interval to Through the loss function and multiple iterations, adaptive compensation of the attenuation channel is achieved, solving the problems of under-compensation and over-compensation in traditional compensation methods mentioned in limitation (2). Figure 4 This diagram shows the results of the proposed deep-water image color distortion correction technology for the RGB color space. It can be seen that compared to the original underwater image, the restored underwater image has a richer color distribution and better visual effects.

[0095] Design of a PCM model for deep-water image pixel clustering based on brightness features: The RGB color space has inherent advantages in addressing color distortion, as color distortion can be reflected in the histogram distribution of its three channels. However, in the RGB color space, brightness and chromaticity information are coupled together in the pixel values, making it significantly disadvantageous in processing brightness and chromaticity components independently. To address this, we convert the underwater image processed in step 1 into the Lab color space to separate and process the brightness component L and the chromaticity components a and b.

[0096] By enhancing the contrast of the luminance component L, the global brightness and texture of the deep-water image can be effectively improved. The present invention uses a custom pixel clustering model PCM to cluster pixels with similar luminance characteristics in the luminance component L of the deep-water image into pixel blocks, so as to segment the L channel into N non-overlapping pixel blocks, as shown in Equation (4).

[0097]

[0098] in Pixel clustering operations implemented for clustering models, Any pixel block should have similar brightness and texture features. To address this, the present invention designs a pixel clustering model (PCM).

[0099] The network architecture of PCM is as follows Figure 5 As shown in the figure, its framework can be divided into three parts: input layer, feature extraction layer and output layer. The feature extraction layer consists of two sets of encoder-decoder structures to realize two dimensionality increase and dimensionality reduction operations.

[0100] Input layer: The PCM model uses the brightness component L of the underwater image as the input image. Before input, L needs to be normalized. The purpose is to convert the data in the brightness component L into a unified data distribution space, reduce the discreteness of the data in L, so that the subsequent feature extraction layer can better extract the brightness features of L and enhance the PCM model's ability to fit the mapping relationship between brightness features and pixel clustering probability distribution. The specific process is to convert the data in L from Interval linear transformation to Within the range.

[0101] The pixel clustering model PCM designed in this paper is an unsupervised model, that is, PCM can perform inference without training. This results in the PCM model being unable to predict how many categories the pixels in the input brightness component L should be classified into before inference. For this reason, an indicator N of the maximum number of categories is introduced in the input layer. In each iteration, PCM clusters the input brightness component L into n categories, where n is If N is set too large, PCM will overcluster. If N is set too small, PCM will not be able to correctly cluster the pixels in L, reducing the accuracy of edge extraction of each clustered pixel block. Therefore, it is crucial to set N appropriately.

[0102] Feature extraction layer: The feature extraction layer of the pixel clustering model PCM is composed of two sets of encoder-decoder architectures. Among them, since the encoding-decoding process is essentially a convolution operation, the encoder / decoder designed by the present invention is composed of a 1x1 convolution layer, a batch normalization layer and an activation function linear composition. For any encoder / decoder, the brightness feature of the input feature map is first extracted by the 1x1 convolution layer, and then the batch normalization operation is performed through the batch normalization layer to restore the distribution space required for the brightness feature. Finally, the nonlinear transformation is introduced through the activation function to enhance the pixel clustering model PCM's ability to fit the mapping relationship between the brightness feature and the pixel probability distribution space.

[0103] The feature extraction layer diagram of the pixel clustering model PCM is as follows Figure 6 As shown in the figure, in the two encoder-decoder architectures of the PCM model, multi-layered extraction of the brightness features of the deep-water image's brightness component L is achieved by continuously adjusting the input and output parameters of the convolutional layers. For the first encoder-decoder architecture, encoder a receives the brightness component and maps it to a higher-dimensional feature space, denoted as the intermediate channel mid. This intermediate channel mid is designed to perform a dimensionality-upgrade operation on the brightness features of each pixel in L to capture deeper brightness information. Decoder a uses the high-dimensional features output by encoder a as input feature maps. Its output channels are equal to the maximum number of pixel cluster categories N. It maps the input feature maps to the pixel cluster probability space, fitting the mapping relationship between the brightness features and the pixel cluster probability space.

[0104] For the second encoder-decoder architecture, encoder b performs a further dimensionality increase on the pixel cluster probability space, with N input channels and mid output channels, thereby extracting deeper luminance features. Unlike the first dimensionality increase, this dimensionality increase is performed on the pixel cluster probability space, aiming to mine more abstract and complex luminance features. Decoder b performs another dimensionality reduction operation, mapping the high-dimensional features output by encoder b back to the pixel cluster probability space. This further refines and integrates the luminance features, thereby more accurately fitting the mapping relationship between luminance features and the pixel cluster probability space.

[0105] The pixel clustering model (PCM) proposed in this paper performs two dimensionality-increase and dimensionality-reduction operations, but these two operations differ significantly. The first dimensionality-increase operation focuses on extracting deep luminance features from the single-channel input image of the luminance component L, while the second dimensionality-increase operation focuses on mining deeper luminance information in the pixel clustering probability space. This multi-level feature processing approach helps improve the accuracy of the PCM model in clustering pixels of the luminance component L of underwater images.

[0106] Output layer: In the pixel clustering model PCM, two sets of encoder-decoder structures are used to extract the features of the brightness component L. The output result of the feature extraction is a The three-dimensional matrix is ​​denoted as , represents the pixel clustering probability space of L, that is, the probability that each pixel belongs to each category, as shown in formula (5).

[0107]

[0108] Taking formula (5) as an example, the maximum number of categories of pixel clustering is Taking the first row of the matrix as an example, the 1st to 3rd columns represent the probability that the first pixel is located in the 1st to 3rd pixel blocks. Since the matrix form of the brightness component L is converted into a one-dimensional vector form according to the row before the feature extraction output, the coordinates in the matrix are The value represents the coordinate in L The probability that the pixel belongs to pixel block 1 is 0.3, and the coordinates are The value represents the coordinate in L The probability that the pixel belongs to pixel block 2 is 0.3, and the coordinates are The value represents the coordinate in L The probability that the pixel belongs to pixel block 3 is 0.4.

[0109] In the pixel clustering probability space In the , the pixel block to which each pixel belongs is the category corresponding to the maximum value of the pixel clustering probability space, which is recorded as the pixel category space Taking the matrix given in formula (5) as an example, for the luminance component L, the coordinates are The pixel belongs to pixel block 3 because its probability is the highest, which is 0.5. So the pixel belongs to pixel block 3. Similarly, the coordinates in the brightness component L are The pixel belongs to pixel block 1, and the coordinates in the brightness component L are The pixel belongs to pixel block 2. The process of obtaining the pixel category space can be expressed as formula (6), which is recorded as the argmax function.

[0110]

[0111] Figure 7 A schematic diagram of the PCM clustering results based on the brightness channel L is shown. It can be seen that pixels with similar brightness and texture features are clustered into pixel blocks, which is helpful for the subsequent local brightness enhancement operation of each pixel block.

[0112] After refined clustering by the pixel clustering model PCM, the brightness component L is divided into several blocks. By contrast enhancement of each pixel block separately, the overall contrast of the brightness component L is improved from multiple dimensions, ultimately improving the brightness layout and texture of the underwater image. As the nth pixel block of the brightness component L, it can reflect the local brightness of the image. It is composed of low-frequency components and high-frequency components. By reducing the low-frequency components and increasing the high-frequency components, the local brightness block can be effectively improved. The contrast of the global brightness component L is enhanced.

[0113] For local luminance blocks , box filtering is used to extract The low-frequency components of Subtract the low frequency component to obtain the high frequency component, and amplify the high frequency component to achieve The local contrast enhancement is shown in (7).

[0114]

[0115] in, is the local brightness block Contains pixel values, Used to control the local window size of the box filter, For box filter operation, Used to control the degree of enhancement of high-frequency components.

[0116] In the field of image processing, edge details and noise in an image are considered high-frequency information. When enhancing high-frequency components, noise information is often also amplified. Therefore, it is necessary to enhance the local brightness block after the enhancement. Perform noise reduction to reduce the impact of the amplified noise. In the noise reduction algorithm, guided filtering can effectively distinguish between edges and noise because of its edge-preserving property, thereby avoiding the situation where edges that are also high-frequency information are mistakenly deleted. The local brightness block after guided filtering noise reduction is shown in formula (8):

[0117]

[0118] in, Used to control the local window size of the guided filter, is the regularization parameter, This is a guided filtering operation. After removing the amplified noise through guided filtering, the final enhanced local brightness block can be obtained. , the local luminance blocks are spliced ​​to obtain the final enhanced luminance component L.

[0119] After the brightness component contrast enhancement based on the pixel clustering model PCM, the global brightness of the underwater image has been improved to a certain extent. However, its naturalness needs to be further improved. By adjusting the a and b components in the Lab color space, the naturalness of the underwater image can be further improved and the image information can be enriched. The range of the chromaticity components a and b is The present invention introduces an adaptive exponential function to adjust the chrominance components a and b, thereby improving the naturalness of underwater images. The formula of the adaptive exponential function is shown in (9):

[0120]

[0121] Among them, x is the pixel value of the chromaticity component a or b. In the Lab color space, when a and b are 0, the color channel presents a true neutral grayscale value. The adaptive exponential function performs symmetrical enhancement with the neutral gray axis as the zero point, avoiding the phenomenon of artificial color block accumulation caused by the color cast correction algorithm, making the color transition of subjects such as corals and fish smoother and more natural. In addition, the nonlinear mapping brought by the adaptive exponential function simultaneously enhances the color difference distinction of the edge of the details, while enhancing the color level of the low-saturation area, it effectively suppresses the chromaticity diffusion effect of the highlight area, so that the blurred texture structure and boundary contours in the underwater scene can be clearly presented, overcoming the detail blurring problem caused by linear methods such as histogram equalization, and significantly improving the visual recognizability of complex underwater scenes. The image of the adaptive exponential function is shown in Figure 2. Figure 8 shown.

[0122] After the above steps, the inherent color distortion, low image brightness and low naturalness of deep-water images have been effectively improved. Figure 9 This figure shows the results of the proposed deep-water image enhancement technique. It can be seen that through multi-dimensional enhancement in both RGB and Lab color spaces, the color distortion of the original deep-water image is effectively resolved, and both image brightness and texture detail are effectively enhanced. Although some differences still exist with the reference image, a significant improvement has been achieved compared to the original deep-water image.

[0123] Example 2. This example selects the UIEB dataset as a test dataset, which contains 890 underwater images with different degradation problems such as color distortion, brightness attenuation, and low naturalness, and corresponding reference restoration images. For deep-water images, the color distortion problem needs to be solved first. According to the process of step 1 in the technical solution, it is necessary to first select the optimal channel and two attenuation channels based on the pixel average value. For the optimal channel, pre-correction is performed by an improved method of removing extreme pixel values, wherein the fixed ratio set in the pre-correction starts from 0.1% and increases to 2% with a step size of 0.1%. The optimal fixed ratio suitable for cropping is selected through manual observation. After the optimal channel is compensated, the compensation formula and loss function are used for continuous iteration. When the loss function is less than 0.01 or the iteration exceeds 200 times, the iteration is stopped to achieve compensation for the attenuation channel. Finally, the three repaired channels are histogram stretched to achieve the repair of the color distortion problem of the deep-water image.

[0124] Taking into account that the Lab color space is conducive to separating the brightness and chromaticity components, after solving the color distortion problem of deep-water images based on the RGB color space, the present invention is based on the Lab color space to solve the problems of brightness attenuation and low naturalness of deep-water images. In order to comprehensively improve the brightness of deep-water images from multiple dimensions, the present invention adopts the pixel clustering model PCM in step 2 of the technical solution to achieve the cutting of the brightness component L and local brightness enhancement. PCM consists of two sets of encoder-decoder architectures, and its core parameters are N and mid. In order to take into account both clustering effect and hardware performance, N is set to 50 and mid is set to 2048. For other parameters, SGD is selected as the optimizer, the number of iterations is set to 1000, and the learning rate adjustment mechanism adopts exponential decay.

[0125] Based on the pixel clustering model PCM, the brightness component L is cut into several local brightness blocks. For each local brightness block, by enhancing its high-frequency component, the contrast of the local brightness block can be effectively improved, thereby enhancing the global contrast of the brightness component L, and achieving global brightness enhancement and texture improvement of the deep-water image. According to the process of step 3 in the technical solution, the present invention extracts the low-frequency component of the local brightness block through box filtering, obtains the high-frequency component by subtracting the low-frequency component, and multiplies the high-frequency component. However, the enhancement of the high-frequency component will amplify the influence of noise to a certain extent. Therefore, a guided filter with edge-preserving characteristics is introduced at the same time to remove the influence of noise, and finally enhance the contrast of the local brightness block to achieve global brightness enhancement of the deep-water image. Finally, in order to further enhance the naturalness of the deep-water image, the present invention proposes an adaptive exponential function to achieve nonlinear stretching of the chroma channels a and b.

[0126] Through the above steps, the inherent color distortion, brightness attenuation, and low naturalness of deep-water images have been effectively improved. To comprehensively evaluate the advantages of the proposed deep-water image enhancement technology based on multi-color space coupling in the field of image enhancement, the proposed method was compared with other classical methods. The test data consisted of 50 underwater images randomly selected from the UIEB dataset. The underwater image enhancement methods compared included CLAHE, DCP, GC, ICM, UCM, UDCP, and ULAP. The comparison methods involved included image enhancement methods based on traditional algorithm engineering and physical models, all of which are representative. Figure 10 The following figure shows the test results of different methods on the UIEB dataset. It is clear that most of the compared methods failed to effectively improve underwater images in various scenarios, indicating a lack of universal applicability. Furthermore, some methods, such as the UCM method, overcompensated, resulting in red artifacts in the restored underwater images. Some methods, such as the GC method, blurred texture details during the enhancement process, and some methods, such as the UDCP method, reduced the brightness of underwater images. In comparison, the method proposed in this paper is more versatile and effectively addresses various issues with underwater images.

[0127] To further validate the performance of the proposed method, we quantitatively evaluated the enhancement results of various methods using reference metrics, including peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), as well as non-reference metrics, including information entropy, underwater image quality evaluation (UIQM), and underwater color image quality evaluation (UCIQE). To further demonstrate the versatility of the enhancement method and reduce the impact of extreme data, we randomly sampled 100 images from the UIEB dataset for testing and took the average as the final result. The test results are shown in Table 1.

[0128] Table 1 Comparison results of objective indicators of various methods on UIEB dataset

[0129] Evaluation indicators CLAHE DCP GC ICM UCM UDCP ULAP The present invention PSNR↑ 17.272 11.657 13.795 13.979 15.238 10.747 13.151 <![CDATA[ 17.567 ]]> SSIM↑ 0.802 0.500 0.654 0.680 0.723 0.457 0.555 <![CDATA[ 0.839 ]]> Entropy↑ 7.019 6.392 6.359 6.989 <![CDATA[ 7.409 ]]> 6.304 6.779 7.290 UIQM↑ 0.572 0.096 0.330 0.158 0.384 0.014 0.291 <![CDATA[ 1.113 ]]> UCIQE↑ 0.545 0.500 0.478 0.540 0.525 0.525 0.553 <![CDATA[ 0.564 ]]>

[0130] Table 1 shows the objective indicator comparison results of various underwater image enhancement methods on the UIEB test dataset. Among them, PSNR and SSIM are reference evaluation indicators, which need to be compared with the reference restoration image given in the UIEB dataset to obtain evaluation scores. PSNR scores by comparing the pixel errors and color differences between the restoration image and the reference image, while SSIM scores by comparing the restoration image and the reference image in three aspects: brightness, contrast and overall structure. It can be seen from the data in the table that the method proposed in the present invention has achieved the highest scores in both PSNR and SSIM, which means that the image restored by the method proposed in the present invention is close to the reference restoration image in terms of color distribution, brightness distribution, contrast distribution and other features, which fully confirms the effectiveness of the strategy given by the method of the present invention for solving problems such as color distortion, brightness attenuation and low naturalness.

[0131] In addition, Entropy, UIQM and UCIQE, as non-reference evaluation indicators, can directly use mathematical formulas to evaluate the quality of the restored underwater images, which also means that the non-reference evaluation indicators can only evaluate some features of the restored images. Among them, Entropy is used to measure the amount of information contained in the image, UIQM is used to measure the contrast, color and clarity of the image, and UCIQE is used to measure the chroma, saturation and contrast of the image. According to the data in the table, it can be seen that the method proposed in the present invention achieved the highest score in the two indicators UIQM and UCIQE, and achieved the second highest score in Entropy, which confirms that the method of the present invention has performed good restoration of underwater images in terms of contrast, color, saturation and other features. However, according to Figure 9 The comparison results show that the UCM method, which has the highest Entropy score, overcompensates during the restoration process, resulting in red artifacts in the restored image. Subjectively, the restored image is far inferior to the image restored by the proposed method. This demonstrates that the quality of the restored image cannot be directly evaluated based on a single reference evaluation metric. A comprehensive evaluation of various evaluation metrics and subjective assessments is required to comprehensively assess the quality of the restored image. Therefore, a comprehensive evaluation of multiple aspects shows that the deepwater image enhancement technology proposed in this invention, based on multi-color space coupling, can effectively address the inherent problems of deepwater images and exhibits excellent restoration results and versatility.

[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deep-water image enhancement method based on multi-color space coupling, characterized in that: The following steps are involved: S1. Perform color distortion correction of deep-water images in RGB color space: determine the optimal channel, moderate attenuation channel, and severe attenuation channel based on the pixel average value; Perform asymmetric histogram cropping pre-correction on the optimal channel; Design the loss function and make the attenuation channel approach the optimal channel in terms of channel structure and pixel distribution through iterative compensation; S2. Clustering and enhancing the luminance component in the Lab color space: Convert the color-distortion-corrected image to the Lab color space; cluster the luminance component L into several pixel blocks using the unsupervised pixel clustering model PCM; perform local contrast enhancement and guided filtering-based noise reduction on each pixel block; S3. Naturalness optimization in Lab color space: Use adaptive exponential function to perform nonlinear stretching on chromaticity components a and b to improve color naturalness.

2. The method according to claim 1, characterized in that The specific process of S1 includes: S11. Determine the optimal channel: Calculate the average pixel values ​​of the three RGB channels and sort them from the largest to the smallest average values ​​to determine the optimal channel. , medium attenuation channel , heavy attenuation channel ; The pixel average calculation formula is: ； in, is the height of the input deep-water image; is the width of the input deep-water image; Indicates channel Pixel value of Indicates the red, green, or blue channel; is the summation operator; and are the row and column indices of the pixel respectively; S12, optimal channel pre-correction: using the histogram mode as the dividing point, cropping a fixed proportion of pixels starting from the beginning on the left side of the mode to obtain , cropping a fixed ratio of pixels from the tail on the right side of the mode to obtain ; The optimal channel Smaller than The pixel value is set to , greater than The pixel value is set to , to remove extreme pixel values; S13, attenuation channel compensation: Design loss function and use iterative compensation formula to compensate for moderate attenuation channel and heavy attenuation channels Compensation is performed, and the iteration stopping condition is that the loss function value is less than 0.01 or the number of iterations reaches 200 times; the loss function and compensation formula are: ； in, The optimal channel The average value of pixels; Attenuation channel or The average value of pixels; is the reference channel similarity of natural images; express and similarity; Indicates a moderately attenuated channel or a heavily attenuated channel; ； in, is the pixel value of the attenuation channel after compensation; is the original attenuation channel pixel value; Used for global compensation control; Used for local compensation control; is the optimal channel pixel value after pre-correction.

3. The method according to claim 1, characterized in that The specific process of S2 includes: S21, brightness component normalization: linearly map the brightness component L from the interval [0,100] to [0,1]; S22. Pixel Clustering Model PCM Processing: The luminance component L is clustered and segmented into N pixel blocks using the unsupervised pixel clustering model PCM. PCM includes an input layer, a feature extraction layer, and an output layer. The feature extraction layer consists of two encoder-decoder structures, each encoder or decoder consisting of a 1x1 convolutional layer, a batch normalization layer, and an activation function linear structure. The maximum number of cluster categories N = 50, and the middle channel dimension mid = 2048. PCM outputs the pixel clustering probability space and through Function determines pixel category space : ； in, Represents clustering operation; is the nth pixel block; N = 50 is the maximum number of cluster categories; ； in, Dimension Pixel clustering probability space; The function returns the category index corresponding to the maximum probability; S23, local brightness enhancement: for each pixel block Perform contrast enhancement based on box filtering and noise reduction based on guided filtering: ； in, is the enhanced pixel value; It is a box filter operation; Pixel block Pixel value of is the local window size of the box filter; Increase weight for high frequency components; and is an arithmetic operator; is the multiplication operator; ； in, is the pixel value after noise reduction; To guide the filtering operation; is the local window size of the guided filtering; is the regularization parameter.

4. The method according to claim 1, characterized in that The nonlinear stretching formula for the chromaticity components a and b in S3 is: ； in, is the enhanced chroma pixel value; is the original pixel value of chroma component a or b; is a natural constant; Used to control the intensity of chroma stretching.

Images