Underwater degraded image enhancement method and system based on image fusion

Abstract

The invention belongs to the technical field of image processing, and particularly discloses an underwater degraded image enhancement method and system based on image fusion. The method comprises the steps of performing color compensation on other color channels by using a color channel with the maximum average pixel value based on RGB channel pixel values of an underwater image, and performing processing by using a gray world white balance method to obtain a color correction image; processing an L channel of the color correction image by using a contrast-limited adaptive histogram equalization algorithm improved based on a particle swarm optimization algorithm, and then combining the L channel with the A and B channels to generate a first fusion image; performing bilateral filtering processing on the color correction image to obtain a second fusion image; and extracting image weights of the first fusion image and the second fusion image, fusing the first fusion image and the second fusion image through a multi-scale fusion algorithm, and outputting an enhanced image. According to the invention, the color performance of the image is optimized, and the global brightness enhancement and the local detail improvement are coordinated and unified.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and more specifically, relates to a method and system for enhancing underwater degraded images based on image fusion. Background Technology

[0002] The integrity of a ship's underwater structure is fundamental to its survivability. However, assessing damage to the underwater hull structure, propellers, and rudder is a significant challenge, often requiring close-up examination and photography using underwater optical cameras. However, different wavelengths of light are scattered and absorbed differently by the water in the underwater environment, resulting in underwater images exhibiting a uniform blue-green hue and losing true color (i.e., color cast). Furthermore, the presence of dissolved substances and suspended particles in the water causes scattering and refraction of light, leading to increased noise, blurred details, and low contrast in underwater images, thus reducing overall image quality.

[0003] Acquiring high-quality images essentially means reversing the aforementioned degradation process and restoring the image's color, contrast, and details. Among existing image enhancement techniques, fusion-based enhancement techniques (such as the Fusion algorithm) can achieve superior enhancement effects, but they still suffer from limited color improvement in some scenarios. The Underwater Dark Channel Prior (UDCP) method is insufficient in correcting color distortion and tends to result in an overall darker image. Homomorphic filtering (HP) can significantly improve image contrast and brightness, but it introduces red color cast. The Generalized Dark Channel Prior (GDCP) method has limited effectiveness in color and contrast restoration, often resulting in images that are either too bright or too dark. The deep learning-based FUnIE (Fast Underwater Image Enhancement) method can effectively suppress blue scattering, but its processing effect on blue-green or green scattering is poor, and the resulting image is prone to yellow color cast. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this application aims to provide an underwater image enhancement method and system based on image fusion, which addresses the limitations of existing image enhancement technologies in improving contrast and brightness and the existence of color shift issues.

[0005] The first aspect of this application relates to a method for enhancing underwater degraded images based on image fusion, comprising the following steps: Step S1: Based on the average pixel value of the RGB color channels of the underwater image, determine the color of the underwater image, use the color channel with the largest average pixel value to perform color compensation on the other color channels, and then perform grayscale world white balance processing on the color-compensated image to obtain a color-corrected image. Step S2: Convert the color-corrected image from the RGB color space to the LAB color space. Process the L channel using a contrast-limited adaptive histogram equalization algorithm based on particle swarm optimization. Then merge the L channel with the A and B channels to generate the first fused image. Perform bilateral filtering on the color-corrected image to obtain the second fused image. Step S3: Extract image weights from the first fused image and the second fused image, and fuse the first fused image and the second fused image using a multi-scale fusion algorithm to output an enhanced image.

[0006] In some implementations, the color compensation method in step S1 is specifically as follows: Comparison of underwater images , , If the average pixel values ​​of the three channels are the same, the underwater image is determined to have no color cast and no color compensation is performed; the color channel with the largest average pixel value is used to compensate the other color channels. The compensation formula is: ; in, The pixel value of the color channel to be compensated; The color channel pixel value with the largest average pixel value; The maximum average pixel value; The average pixel value of the color channel to be compensated.

[0007] In some implementations, a contrast-limited adaptive histogram equalization algorithm based on particle swarm optimization is used. The processing method for the L channel specifically includes the following steps: Step A. Initialize a group of particles based on the particle swarm optimization algorithm, and randomly construct the position of each particle using the local region block size and contrast limit threshold in the contrast-limited adaptive histogram equalization algorithm; Step B. Apply a contrast-limited adaptive histogram equalization algorithm to the L channel using the local region block size corresponding to the position of each particle and the contrast limit threshold; Step C. Calculate the contrast measurement index of the L channel after processing, as the fitness value of the particle; Step D. Update the individual optimal position of each particle and the global optimal position of the entire population, and update the velocity and position of all particles; Step E. Repeat steps B to D until the maximum number of iterations is reached or convergence is achieved. Output the combination of local region block size and contrast limit threshold corresponding to the global optimal position, as well as the processed L channel. Combine the L channel with the A and B channels to generate the first fused image.

[0008] In some implementations, grayscale white balance processing is performed on the color-compensated image, specifically including the following steps: Calculate color-compensated images , , The average pixel value of the three channels is then calculated, and the average value of the red, green and blue channels is then calculated. Then divide the average of the average pixel values ​​by the average of each color channel to obtain the weight of each color channel; Multiply the pixel values ​​of each channel by their weights to obtain the color-corrected pixel values ​​of each color channel; The color-corrected pixel values ​​of each color channel are merged to obtain the color-corrected image.

[0009] In some implementations, step S3 specifically includes the following steps: Step S3.1: Extract the Laplacian contrast weight, local contrast weight, saliency weight, and exposure weight from the first fused image and the second fused image, respectively; Step S3.2: Normalize the Laplacian contrast weight, local contrast weight, saliency weight, and exposure weight to obtain a weight map; Step S3.3: Construct the corresponding Laplacian pyramid for the first fused image and the second fused image, and construct the corresponding Gaussian pyramid for the weight map; Step S3.4: Based on the first fused image and the second fused image, perform the fusion operation between the fused image and the weight map at each pyramid level, and finally obtain the fusion result through pyramid reconstruction and output the enhanced image.

[0010] The second aspect of this application relates to an underwater degraded image enhancement system based on image fusion, comprising: The color correction module is used to determine the color of underwater images, use the color channel with the largest average pixel value to compensate for the color of other color channels, and then perform grayscale world white balance processing on the color-compensated image to obtain a color-corrected image. The fused image acquisition module is used to convert the color-corrected image from the RGB color space to the LAB color space, process the L channel using a contrast-limited adaptive histogram equalization algorithm based on particle swarm optimization, and then merge it with the A and B channels to generate the first fused image; the color-corrected image is then subjected to bilateral filtering to obtain the second fused image. The enhanced image acquisition module is used to extract image weights from the first fused image and the second fused image, fuse the first fused image and the second fused image through a multi-scale fusion algorithm, and output the enhanced image.

[0011] In some implementations, the color correction module includes a color compensation unit for comparing underwater images. , , If the average pixel values ​​of the three channels are the same, the underwater image is determined to have no color cast and no color compensation is performed; the color channel with the largest average pixel value is used to compensate the other color channels. The compensation formula is: ; in, The pixel value of the color channel to be compensated; The color channel pixel value with the largest average pixel value; The maximum average pixel value; The average pixel value of the color channel to be compensated.

[0012] In some implementations, the fused image acquisition module includes a first brightness adjustment unit and a second brightness adjustment unit; The first brightness adjustment unit is used to convert the color correction image from the RGB color space to the LAB color space. After processing the L channel with a limited contrast adaptive histogram equalization algorithm based on particle swarm optimization algorithm, it is merged with the A channel and the B channel to generate the first fused image. The second brightness adjustment unit is used to perform bilateral filtering on the color-corrected image to obtain the second fused image; The first brightness adjustment unit includes: an initialization particle subunit, a brightness adjustment subunit, a fitness calculation subunit, a particle update subunit, and a channel merging subunit; Initialize particle subunits to initialize a group of particles based on the particle swarm optimization algorithm. The position of each particle is randomly constructed using the local region block size and contrast limit threshold in the contrast-limited adaptive histogram equalization algorithm. The brightness adjustment subunit is used to perform a contrast-limited adaptive histogram equalization algorithm on the L channel using the local region block size corresponding to the position of each particle and the contrast limit threshold. The fitness calculation subunit is used to calculate the contrast measurement index of the L channel after processing, which is used as the fitness value of the particle. The particle update subunit is used to update the individual optimal position of each particle and the global optimal position of the entire population, and to update the velocity and position of all particles. The channel merging subunit is used to merge the L channel corresponding to the global optimal position with the A and B channels to generate the first fused image.

[0013] In some implementations, the color correction module includes a color correction unit, which includes: a pixel average value calculation subunit, a channel weight acquisition subunit, a multiplier, and a merging subunit; The pixel average calculation subunit is used to calculate the color-compensated image. , , The average pixel value of the three channels is then calculated, and the average value of the red, green and blue channels is then calculated. The channel weight acquisition subunit is used to divide the average of the average pixel values ​​by the average of each color channel to obtain the weight of each color channel; The multiplier is used to multiply the pixel value of each channel by the corresponding weight to obtain the pixel value of each color channel after color correction; The merging subunit is used to merge the pixel values ​​of each color channel after color correction to obtain a color-corrected image.

[0014] In some implementations, the enhanced image acquisition module includes an image weight acquisition unit, a normalization processing unit, a pyramid construction unit, and a fusion unit; The image weight acquisition unit is used to extract Laplacian contrast weight, local contrast weight, saliency weight and exposure weight from the first fused image and the second fused image, respectively; The normalization processing unit is used to normalize the Laplacian contrast weight, local contrast weight, saliency weight, and exposure weight to obtain a weight map; The pyramid building unit is used to build a corresponding Laplacian pyramid for the first fused image and the second fused image, and to build a corresponding Gaussian pyramid for the weight map; The fusion unit is used to perform fusion operations on the fused image and the weight map at each pyramid level based on the first fused image and the second fused image, and finally obtain the fusion result through pyramid reconstruction and output the enhanced image.

[0015] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0017] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0018] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0019] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: This application provides an underwater image enhancement method based on image fusion. The proposed improved CLAHE algorithm based on particle swarm optimization can automatically optimize the combination of block size and contrast limit threshold, so that CLAHE obtains the optimal solution for image contrast enhancement, achieving global contrast improvement. In addition, the image is processed by a bilateral filter, which effectively improves the local contrast of image brightness. The resulting images from the two processing methods are fused at multiple scales, which combines global brightness adjustment with local detail enhancement, achieving a coordinated unity of global and local information.

[0020] This application provides an underwater image enhancement method based on image fusion. It proposes to use the color channel with the largest average pixel value in the RGB color space to compensate other color channels. The color channel compensation formula can effectively solve the color shift problem in underwater images caused by the scattering and absorption of different colored light by water bodies. Based on this, the grayscale world white balance algorithm effectively removes red artifacts and further optimizes the color performance of the image. Attached Figure Description

[0021] Figure 1 This is a schematic flowchart of the underwater degraded image enhancement method provided in the embodiments of this application.

[0022] Figure 2 This is a flowchart illustrating the multi-scale fusion technology method provided in the embodiments of this application.

[0023] Figure 3(a) is a schematic diagram of the original images in the UCCS dataset provided in the embodiments of this application. The three images are blue, blue-green, and green backgrounds, respectively.

[0024] Figure 3(b) is a schematic diagram of the image after processing the original image of Figure 3(a) by UDCP provided in the embodiment of this application.

[0025] Figure 3(c) is a schematic diagram of the image after processing the original image of Figure 3(a) by the image enhancement method of this application according to the embodiment of this application.

[0026] Figure 4(a) is a schematic diagram of the original images in the UIEB dataset provided in the embodiment of this application. The three images have yellow-green, blue, and green backgrounds, respectively.

[0027] Figure 4(b) is a schematic diagram of the image after processing the original image of Figure 4(a) by UDCP according to the embodiment of this application.

[0028] Figure 4(c) is a schematic diagram of the image after processing the original image of Figure 4(a) by the image enhancement method of this application according to the embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0031] In this application, the terms “first” and “second” are used to distinguish different objects, rather than to describe a specific order of objects.

[0032] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0033] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more.

[0034] This application provides an underwater image enhancement method based on image fusion. First, the average pixel value of the RGB channels of the underwater image is calculated, and color judgment is performed to determine the largest color channel, which is then used as the reference channel. A novel color compensation method is applied to compensate the other two channels. Then, a gray-scale world white balance algorithm is used to process the compensated underwater image to obtain a color-corrected image. Simultaneously, to enhance image contrast, the color-corrected image is applied to an improved contrast-limited adaptive histogram equalization algorithm based on particle swarm optimization and a bilateral filter to obtain a first fused image and a second fused image. Weights are extracted from the first and second fused images, and the extracted weights are then fused with the two fused images at multiple scales to obtain the output image. Specifically, as follows... Figure 1 As shown.

[0035] The embodiments of this application are described below with reference to the accompanying drawings.

[0036] This application provides a method for enhancing underwater degraded images based on image fusion, specifically including the following steps: Step 1: Underwater Image Color Correction Because light of different wavelengths attenuates at different rates in water, red light has the longest wavelength and attenuates the fastest, while shorter wavelengths of green and blue light can reach relatively deep depths. Therefore, underwater images mostly appear green, blue-green, or blue. In addition, the scattering effect of suspended particles in the water can also cause problems such as image blurring and decreased contrast. Traditional white balance algorithms (such as gray-world white balance) can effectively remove blue tones, but they fail when processing underwater images with unbalanced color channel weights due to inaccurate weight calculations. When the weight of either the red or blue channel, or both, is too low, the algorithm will overcompensate, especially causing severe color distortion and artifacts in the red channel. To address this, this application first performs color judgment before gray-world white balance processing and introduces a new color compensation mechanism to correct this defect, ultimately obtaining a more accurate color reproduction result image.

[0037] Step one is as follows: Step 1.1: Underwater Image Color Compensation This application provides a color compensation method, which first calculates the color compensation of underwater images. of , , Channel average pixel value , , The formula for calculating the average pixel value is as follows: (1); in, and Indicates image size. This represents the pixel value of a certain channel; after calculating the average pixel value of each of the three channels, the three average values ​​are compared to obtain the maximum average pixel value. And there are several possible scenarios: (1) If the color is normal, then the image is considered to have no color cast and no color compensation is needed.

[0038] (2) = If the average pixel value of the red channel is the largest, then the pixel values ​​of the blue and green channels are compensated by the red channel. Underwater images The green channel compensation formula is as follows: (2); Underwater images The formula for compensation in the blue channel is as follows: (3).

[0039] (3) = If the average pixel value of the green channel is the largest, then the pixel values ​​of the red and blue channels are compensated by the green channel. Underwater images The red channel compensation formula is as follows: (4); Underwater images The formula for compensation in the blue channel is as follows: (5).

[0040] (4) = If the average pixel value of the blue channel is the largest, then the pixel values ​​of the red and green channels are compensated by the blue channel. Underwater images The red channel compensation formula is as follows: (6); Underwater images The green channel compensation formula is as follows: (7); The compensated color channels are combined to obtain the color-corrected image. .

[0041] Step 1.2: Grayscale World White Balance Processing Color-compensated images Gray-scale white balance processing is performed to further correct the image colors; firstly, the color compensation image is calculated. of , , Three-channel average pixel value , , Then calculate the average of the average pixel values ​​of the red, green, and blue channels. ,Will The weight of each channel is obtained by dividing the value by the mean of each color channel. Finally, the adjusted value is obtained by multiplying the pixel value of each channel by the weight, as shown in the following formula: (8); The adjusted color channels are merged to obtain the color-corrected image. .

[0042] Step 2: Underwater image contrast enhancement The contrast enhancement processing in this application uses the white balance processing result of the underwater image, i.e., the color-corrected image. The input image is converted from the RGB color space to the LAB color space. The L, A, and B channel values ​​are extracted. An improved contrast-limited adaptive histogram equalization algorithm (CLAHE) based on particle swarm optimization (PSO) is applied to the L channel. This CLAHE is then merged with the original A and B channels to obtain the first fused image, achieving a global contrast enhancement. The color-corrected image is then... The second fused image is obtained after bilateral filtering, which improves the local contrast of the image brightness.

[0043] Step 2.1: Improved CLAHE based on PSO Traditional CLAHE requires manual setting of local region tile size and contrast limit threshold, making it difficult to find the global optimum. This application proposes an improved CLAHE, which uses a particle swarm optimization algorithm with contrast measurement index (UIConM) as the objective function to automatically optimize the combination of tile size and contrast limit threshold parameters.

[0044] The specific technical process is as follows: Step 2.1.1: PSO initializes a group of particles, with each particle's position being random. , and These represent the block size and contrast limit threshold corresponding to the particle, respectively.

[0045] Step 2.1.2: For each particle, perform CLAHE processing on the input image using the parameters corresponding to its position.

[0046] Step 2.1.3: Calculate the UIConM of the processed image as the fitness value of the particle; Step 2.1.4: Update the individual optimal position of each particle. and the global optimal position of the entire population .

[0047] Step 2.1.5: Update the velocity and position of all particles according to the following formula: (9); (10); in, t Indicates the current iteration number. This represents the velocity of the particle in the search space. This indicates the position of the particle in the search space. It is inertial weight. and It is a learning factor. and These are two random numbers, ranging from 0 to 1. This indicates the best position found by the particle in the current iteration. This indicates the best position found by the population in the current iteration.

[0048] Step 2.1.6: Repeat steps 2.1.2 to 2.1.5 until the maximum number of iterations is reached or convergence is achieved.

[0049] Step 2.1.7: Output the global optimum The corresponding parameter combination and the processed image, which is the first fused image.

[0050] Step 2.2: Bilateral Filter A bilateral filter is an edge-preserving smoothing filter; it performs weighted averaging not only in the spatial domain but also in the color or luminance domain. The output image after processing an image through a bilateral filter is: (11); (12); (13); in, The input image to the filter is, in this application, a color-corrected image. , The coordinates of the center pixel to be processed. In The size of the center window is ( The coordinates of other pixels within the range. Coordinates are pixel values, Gaussian function in the spatial domain, with standard deviation of , Range: Gaussian function, Standard Deviation: , This is the normalization factor.

[0051] Step 3: Fusion Weights are extracted from the first and second fused images after brightness and contrast enhancement. The extracted weights are Laplacian contrast weights. ), local contrast weight ( Significance weight () ) and exposure weight ( The extracted weights are normalized, and finally, the images are fused using a multi-scale fusion algorithm to obtain the output image. The specific technical process is as follows: Figure 2 As shown.

[0052] Step 3.1: Weight Calculation The weights used are Laplacian contrast weights ( Local contrast weight () Significance weight () ) and exposure weight ( Below is the method for calculating the weights.

[0053] Step 3.1.1 Laplacian contrast weights ( ) The Laplacian filter can highlight high-frequency components (such as edges and noise) in an image while suppressing low-frequency components (such as background and slowly changing textures). In other words, it enhances areas with abrupt changes in grayscale and weakens areas with slow changes in grayscale, thus enhancing details. The expression for the Laplacian operator is as follows: (14); This application first calculates the grayscale of the image and inputs it into a Laplacian filter, then takes the absolute value of the filtered output to obtain the Laplacian contrast weight.

[0054] Step 3.1.2 Local contrast weighting ( ) Local contrast weight ( The purpose of this method is to enhance local contrast. In image processing, contrast is dynamically adjusted by analyzing brightness differences in local areas of an image to enhance the visual effect of specific regions. The principle is based on pixel neighborhood information driving weight calculations, thereby adjusting local contrast. The following is the formula for calculating local contrast weights: (15); in, Represents the input brightness channel; This indicates the low-pass filtering result; in this application, the gray level of the input image is taken, and the gray level of the input image and its filtering result are substituted into formula (15) to obtain the local contrast weight by taking the second norm.

[0055] Step 3.1.3 Significance weight ( The saliency weights aim to restore or enhance objects or regions that have become indistinct due to environmental conditions (such as underwater light attenuation, scattering, and low contrast). The input image is converted from the RGB color space to the LAB color space, and the L, A, and B channel values ​​are extracted to calculate the saliency weights. (16); in, , , These represent the pixel values ​​of the L, A, and B channels of the image, respectively. , , This represents the average pixel value.

[0056] Step 3.1.4 Exposure weight ( ) (17); in, As an evaluation index of image pixel exposure level, the formula is as follows: Representing an image midpoint( The pixel value at ) The standard deviation is taken; the gray level of the input image is taken, and the gray level of the input image is substituted into formula (17) to obtain the exposure weight.

[0057] Step 3.2: Multi-scale fusion Calculate the four weights for the two images (the first fused image obtained through CLAHE and the second fused image obtained through guided filtering) and normalize them as follows: (18); in, and These are the normalized weights for the two images, and the images are finally fused using a multi-scale fusion algorithm.

[0058] The multi-scale image fusion method used in this application specifically includes the following steps: Step 3.2.1: Construct the corresponding Laplacian pyramids for the first fused image and the second fused image; Step 3.2.2: Construct the corresponding Gaussian pyramid for the weighted graph; Step 3.2.3: Perform image and weight fusion operations at each pyramid level, and finally obtain the fused result through pyramid reconstruction, as shown in the following formula: (19); in, This indicates the reconstructed image obtained through fusion; Indicates the number of pyramid levels; This represents the Gaussian pyramid used to construct the weighted graph; This represents the Laplacian pyramid used to construct the fused image; To merge images, Represents pixels, This represents the normalized weight.

[0059] This application provides an underwater degraded image enhancement method and system based on image fusion, which can effectively correct the color cast of underwater images, improve contrast, and effectively remove red artifacts, thereby significantly improving the image clarity and visual fidelity. (1) The color channel compensation formula proposed in this application can effectively solve the problem of color shift in underwater images caused by the scattering and absorption of different colors of light by water. Based on this, the grayscale world white balance algorithm effectively removes red artifacts and further optimizes the color performance of the image.

[0060] (2) The improved CLAHE based on particle swarm optimization algorithm proposed in this application can automatically optimize the combination of block size and contrast limit threshold, so that CLAHE can obtain the optimal solution for improving the contrast of the image and achieve global contrast improvement of the image. In addition, the image is processed by a bilateral filter, which effectively improves the local contrast of the image brightness. The resulting images obtained by the two processing methods are fused by multi-scale, which combines global brightness adjustment with local detail enhancement, and achieves the coordination and unity of overall and local information.

[0061] Example 1 To verify the performance of this application in processing underwater images, image samples were processed on the open-source datasets UCCS and UIEB using both the application method and the traditional physical model-based image enhancement method (UDCP), respectively, resulting in enhanced images (Figures 3(a)~3(c) and 4(a)~4(c)). The Underwater Image Quality Evaluation Metric (UIQM) was then calculated. A higher UIQM value indicates better image quality. The UIQM evaluation results are shown in Tables 1 and 2. Based on the image processing results, it can be seen that the image enhancement effect of this application is significantly better than that of the UDCP algorithm.

[0062] Table 1

[0063] Table 2

[0064] Example 2 This application uses a Windows 64-bit operating system and is developed based on the Python programming language. The device performance is an Intel® Core™ i5-13420H CPU @ 2.1GHz and 24GB RAM.

[0065] The underwater degradation image enhancement method provided in this application specifically includes the following steps: Step S1: Import the original image (RGB color space); This application processes multiple images from the open-source dataset UIEB, taking one of the degraded images as an example.

[0066] Step S2: Underwater image color determination Calculate the underwater image according to formula (1) of , , Channel average pixel value , , Then compare the three average pixel values ​​to find the channel corresponding to the maximum average pixel value.

[0067] Step S3: Underwater image color compensation According to the reference channel in step S2, the color compensation formulas (2) to (7) proposed in this invention are used to perform color compensation on the other two channels. The compensated R, G, and B channels are merged to obtain the result image. Compared with the color distortion (green color shift) in the original underwater degraded image, the color compensation formula proposed in this application effectively corrects the color shift.

[0068] Step S4: Grayscale World White Balance Processing The color compensation image in step S3 is processed for grayscale world white balance according to formula (8), and the adjusted color channels are merged to obtain the color corrected image. After color compensation, white balance processing can effectively remove the red artifacts introduced by traditional white balance processing.

[0069] Step S5: Process the color-corrected image obtained in step S4. The input image is converted from the RGB color space to the LAB color space, and the L, A, and B channel values ​​are extracted. The improved CLAHE proposed in this paper processes the L channel. The particle swarm optimization algorithm automatically finds the optimal combination of contrast limit threshold and local region block size, and merges the corresponding L channel processing result with the original A and B channels to convert it into the RGB color space to obtain the first fused image. The global contrast of the image is significantly improved.

[0070] Step S6: Process the color-corrected image obtained in step S4. The input image is processed using bilateral filtering, and the filtering radius is set according to formulas (11) to (13). Spatial domain standard deviation Range and standard deviation The resulting image, namely the second fused image, is obtained.

[0071] Step S7: Calculate the four weights of the first fused image and the second fused image obtained in steps S5 and S6, respectively; Step S7.1: First, calculate the grayscale of the image and input it into the Laplacian filter. Set the Laplacian convolution kernel to... The absolute value of the filter output is used to obtain the Laplace weight; Step S7.2: Extract the grayscale value of the input image and apply a binomial kernel to it. Perform low-pass filtering, setting the high-frequency cutoff value to... According to formula (15), the gray level of the input image and its filtering result are substituted into the above formula to obtain the L2 norm and thus the local contrast weight. Step S7.3: Convert the input image from RGB color space to LAB color space, extract the L, A, and B channel values, and calculate the significance weight according to formula (16); Step S7.4: Take the grayscale of the input image and substitute it into formula (17) to obtain the exposure weight and standard deviation. Set it to 0.25.

[0072] Step S8: Normalize the two fused images obtained in step S7 using formula (18) with 4 weights each, and then reconstruct the image according to the multi-scale image fusion method and formula (19) to obtain the enhanced image. In this application, the number of pyramid layers is set to 5. The underwater degraded image enhancement method based on image fusion provided in this application is used to process the original underwater image to obtain the final image. Compared with the original underwater degraded image, the final image corrects the color cast, improves the contrast, and significantly enhances the image clarity and visual fidelity.

[0073] Example 3 The underwater degraded image enhancement system based on image fusion provided in this application is described below. The underwater degraded image enhancement system based on image fusion described below can be referred to in correspondence with the underwater degraded image enhancement method based on image fusion described above.

[0074] This application provides an underwater degraded image enhancement system based on image fusion, comprising: The color correction module is used to determine the color of the underwater image based on the average pixel value of the RGB color channels, use the color channel with the largest average pixel value to compensate the other color channels, and then perform grayscale world white balance processing on the color-compensated image to obtain the color-corrected image. The fused image acquisition module is used to convert the color-corrected image from the RGB color space to the LAB color space, process the L channel using a contrast-limited adaptive histogram equalization algorithm based on particle swarm optimization, and then merge it with the A and B channels to generate the first fused image; the color-corrected image is then subjected to bilateral filtering to obtain the second fused image. The enhanced image acquisition module is used to extract image weights from the first fused image and the second fused image, fuse the first fused image and the second fused image through a multi-scale fusion algorithm, and output the enhanced image.

[0075] In some implementations, the color correction module includes a color compensation unit for comparing underwater images. , , If the average pixel values ​​of the three channels are the same, the underwater image is determined to have no color cast and no color compensation is performed; the color channel with the largest average pixel value is used to compensate the other color channels. The compensation formula is: ; in, The pixel value of the color channel to be compensated; The color channel pixel value with the largest average pixel value; The maximum average pixel value; The average pixel value of the color channel to be compensated.

[0076] In some implementations, the fused image acquisition module includes a first brightness adjustment unit and a second brightness adjustment unit; The first brightness adjustment unit is used to convert the color correction image from the RGB color space to the LAB color space. After processing the L channel with a limited contrast adaptive histogram equalization algorithm based on particle swarm optimization algorithm, it is merged with the A channel and the B channel to generate the first fused image. The second brightness adjustment unit is used to perform bilateral filtering on the color-corrected image to obtain the second fused image; The first brightness adjustment unit includes: an initialization particle subunit, a brightness adjustment subunit, a fitness calculation subunit, a particle update subunit, and a channel merging subunit; Initialize particle subunits to initialize a group of particles based on the particle swarm optimization algorithm. The position of each particle is randomly constructed using the local region block size and contrast limit threshold in the contrast-limited adaptive histogram equalization algorithm. The brightness adjustment subunit is used to perform a contrast-limited adaptive histogram equalization algorithm on the L channel using the local region block size corresponding to the position of each particle and the contrast limit threshold. The fitness calculation subunit is used to calculate the contrast measurement index of the L channel after processing, which is used as the fitness value of the particle. The particle update subunit is used to update the individual optimal position of each particle and the global optimal position of the entire population, and to update the velocity and position of all particles. The channel merging subunit is used to merge the L channel corresponding to the global optimal position with the A and B channels to generate the first fused image.

[0077] In some implementations, the color correction module includes a color correction unit, which includes: a pixel average value calculation subunit, a channel weight acquisition subunit, a multiplier, and a merging subunit; The pixel average calculation subunit is used to calculate the color-compensated image. , , The average pixel value of the three channels is then calculated, and the average value of the red, green and blue channels is then calculated. The channel weight acquisition subunit is used to divide the average of the average pixel values ​​by the average of each color channel to obtain the weight of each color channel; The multiplier is used to multiply the pixel value of each channel by the corresponding weight to obtain the pixel value of each color channel after color correction; The merging subunit is used to merge the pixel values ​​of each color channel after color correction to obtain a color-corrected image.

[0078] In some implementations, the enhanced image acquisition module includes an image weight acquisition unit, a normalization processing unit, a pyramid construction unit, and a fusion unit; The image weight acquisition unit is used to extract Laplacian contrast weight, local contrast weight, saliency weight and exposure weight from the first fused image and the second fused image, respectively; The normalization processing unit is used to normalize the Laplacian contrast weight, local contrast weight, saliency weight, and exposure weight to obtain a weight map; The pyramid building unit is used to build a corresponding Laplacian pyramid for the first fused image and the second fused image, and to build a corresponding Gaussian pyramid for the weight map; The fusion unit is used to perform fusion operations on the fused image and the weight map at each pyramid level based on the first fused image and the second fused image, and finally obtain the fusion result through pyramid reconstruction and output the enhanced image.

[0079] It is understood that the detailed functional implementation of each of the above units / modules can be found in the description in the aforementioned method embodiments, and will not be repeated here.

[0080] It should be understood that the above-described device is used to execute the methods in the above embodiments. The corresponding program modules in the system are similar in implementation principle and technical effect to those described in the above methods. The working process of the system can be referred to the corresponding process in the above methods, and will not be repeated here.

[0081] Based on the methods in the above embodiments, this application provides an electronic device that may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor may invoke logical instructions stored in the memory to execute the methods in the above embodiments.

[0082] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0083] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0084] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0085] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0086] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0087] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0088] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0089] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An underwater degraded image enhancement method based on image fusion, characterized in that, The method comprises the following steps: Step S1: judging the color of the underwater image based on the average pixel value of the RGB color channel, compensating the other color channels using the color channel with the maximum average pixel value, performing gray world white balance processing on the color-compensated image, and obtaining a color-corrected image; Step S2: converting the color-corrected image from the RGB color space to the LAB color space, processing the L channel using a limited contrast adaptive histogram equalization algorithm improved based on a particle swarm optimization algorithm, and merging the A channel and the B channel to generate a first fused image; performing bilateral filtering processing on the color-corrected image to obtain a second fused image; Step S3: extracting image weights from the first fused image and the second fused image, fusing the first fused image and the second fused image through a multi-scale fusion algorithm, and outputting an enhanced image.

2. The underwater degraded image enhancement method of claim 1, wherein, The color compensation method in step S1 specifically comprises the following steps: Comparing underwater images , , Channel average pixel value, if the average pixel value of three channels is the same, it is determined that the underwater image has no color deviation, and no color compensation is performed; Compensating the other color channels using the color channel with the maximum average pixel value; The compensation formula is: ; wherein, is the color channel pixel value to be compensated; is the color channel pixel value with the largest average pixel value; is the largest average pixel value; is the color channel average pixel value to be compensated.

3. The underwater degradation image enhancement method according to claim 1 or 2, characterized in that, The processing method of the L channel using the limited contrast adaptive histogram equalization algorithm improved based on the particle swarm optimization algorithm specifically comprises the following steps: Step A. Initialize a group of particles based on the particle swarm optimization algorithm, and randomly construct the position of each particle using the local region block size and the contrast limit threshold in the limited contrast adaptive histogram equalization algorithm; Step B. Process the L channel using the local region block size and the contrast limit threshold corresponding to the position of each particle through the limited contrast adaptive histogram equalization algorithm; Step C. Calculate the contrast measurement index of the processed L channel as the fitness value of the particle; Step D. Update the individual optimal position of each particle and the global optimal position of the entire population, and update the speed and position of all particles; Step E. Repeat steps B to D until the maximum number of iterations is reached or convergence is achieved, and output the combination of the local region block size and the contrast limit threshold corresponding to the global optimal position and the processed L channel, and merge the A channel and the B channel to generate a first fused image.

4. The underwater degraded image enhancement method of claim 2, wherein, The gray world white balance processing on the color-compensated image specifically comprises the following steps: calculating a color compensation image , , three-channel average pixel value, and then calculating an average value of the red, green and blue three-channel average pixel values; Divide the average value of the average pixel value by the average value of each color channel to obtain the weight of each color channel; Multiply the pixel value of each channel by the corresponding weight to obtain the color-corrected pixel value of each color channel; Merge the color-corrected pixel values of each color channel to obtain a color-corrected image.

5. The underwater degraded image enhancement method of claim 1, wherein, Step S3 specifically comprises the following steps: Step S3.1: extracting Laplacian contrast weights, local contrast weights, saliency weights, and exposure weights from the first fused image and the second fused image, respectively; Step S3.2: normalizing the Laplacian contrast weights, the local contrast weights, the saliency weights, and the exposure weights to obtain a weight map; Step S3.3: constructing a corresponding Laplacian pyramid for the first fused image and the second fused image, and constructing a corresponding Gaussian pyramid for the weight map; Step S3.4: based on the first fusion image and the second fusion image, a fusion operation of the fusion image and the weight map is respectively performed on each pyramid level, and finally a fusion result is obtained through pyramid reconstruction, and an enhanced image is output.

6. An underwater degraded image enhancement system based on image fusion, characterized in that, Comprise: The color correction module is used for judging the color of the underwater image based on the average pixel value of the RGB color channel of the underwater image, using the color channel with the maximum average pixel value to compensate the other color channels, and then performing gray world white balance processing on the color compensated image to obtain a color corrected image; The fusion image acquisition module is used for converting the color corrected image from the RGB color space to the LAB color space, processing the L channel using the limited contrast adaptive histogram equalization algorithm improved based on the particle swarm optimization algorithm, and merging the A channel and the B channel to generate a first fusion image; the color corrected image is processed by bilateral filtering to obtain a second fusion image; The enhanced image acquisition module is used for extracting image weights from the first fusion image and the second fusion image, fusing the first fusion image and the second fusion image through a multi-scale fusion algorithm, and outputting an enhanced image.

7. The underwater retrographic image enhancement system of claim 6, wherein, The color correction module comprises a color compensation unit for comparing the average pixel value of the three channels of the underwater image , , If the average pixel values of the three channels are the same, it is determined that the underwater image has no color deviation, and no color compensation is performed. The color channel with the maximum average pixel value is used to compensate the other color channels; The compensation formula is: ; wherein is the color channel pixel value to be compensated; is the color channel pixel value with the largest average pixel value; is the largest average pixel value; is the color channel average pixel value to be compensated.

8. The underwater degenerative image enhancement system of claim 6 or 7, wherein, The fusion image acquisition module comprises a first brightness adjustment unit and a second brightness adjustment unit; The first brightness adjustment unit is used for converting the color corrected image from the RGB color space to the LAB color space, processing the L channel using the limited contrast adaptive histogram equalization algorithm improved based on the particle swarm optimization algorithm, and merging the A channel and the B channel to generate a first fusion image; The second brightness adjustment unit is used for processing the color corrected image by bilateral filtering to obtain a second fusion image; The first brightness adjustment unit comprises an initialization particle subunit, a brightness adjustment subunit, a fitness calculation subunit, a particle update subunit and a channel merging subunit; The initialization particle subunit is used for initializing a group of particles based on the particle swarm optimization algorithm, and constructing the position of each particle using the local region block size and the contrast limit threshold in the limited contrast adaptive histogram equalization algorithm randomly; The brightness adjustment subunit is used for processing the L channel using the local region block size and the contrast limit threshold corresponding to the position of each particle by the limited contrast adaptive histogram equalization algorithm; The fitness calculation subunit is used for calculating the contrast measurement index of the processed L channel as the fitness value of the particle; The particle update subunit is used for updating the individual optimal position of each particle and the global optimal position of the whole population, and updating the speed and position of all particles; The channel merging subunit is used for merging the L channel corresponding to the global optimal position with the A channel and the B channel to generate a first fusion image.

9. The underwater retrographic image enhancement system of claim 7, wherein, The color correction module comprises a color correction unit, and the color correction unit comprises a pixel average value calculation subunit, a channel weight acquisition subunit, a multiplier and a merging subunit; The pixel average value calculation subunit is configured to calculate an average value of the color compensation image 、 、 three-channel average pixel values, and then calculate an average value of the red, green, and blue three-channel average pixel values. The channel weight acquisition subunit is used for dividing the average value of the average pixel value by the average value of each color channel to obtain the weight of each color channel; The multiplier is used for multiplying the pixel value of each channel by the corresponding weight to obtain the pixel value of each color channel after color correction; The merging subunit is used for merging the color-corrected pixel values of each color channel to obtain a color-corrected image.

10. The underwater retrographic image enhancement system of claim 6, wherein, The enhanced image acquisition module comprises an image weight acquisition unit, a normalization processing unit, a pyramid construction unit and a fusion unit. The image weight acquisition unit is used for extracting Laplacian contrast weight, local contrast weight, saliency weight and exposure weight from the first fusion image and the second fusion image respectively. The normalization processing unit is used for normalizing the Laplacian contrast weight, the local contrast weight, the saliency weight and the exposure weight to obtain a weight map. The pyramid construction unit is used for constructing a corresponding Laplacian pyramid for the first fusion image and the second fusion image, and constructing a corresponding Gaussian pyramid for the weight map. The fusion unit is used for performing a fusion operation of the fusion image and the weight map at each pyramid level based on the first fusion image and the second fusion image, and finally obtaining a fusion result through pyramid reconstruction to output an enhanced image.

Images