Underwater robot vision enhancement method based on foreground and background principal component fusion

Abstract

The application provides an underwater robot vision enhancement method based on foreground and background principal component fusion, first acquires a color distortion underwater image, adopts a color balance strategy to perform color balance on a and b channels of a Lab color space, and performs contrast enhancement on an L channel to obtain a color correction image; then performs contrast enhancement on a foreground image after color correction, and performs defogging processing on a background image; a principal component analysis fusion strategy is introduced, foreground and background components of images with different information are fused to obtain a clear underwater image. The method retains texture details and other information of an original image, enhances contrast of the image, and solves a color distortion problem of an underwater image.

Description

Technical Field

[0001] This invention relates to the field of underwater image processing technology, and more specifically, to a method for enhancing the vision of underwater robots based on principal component fusion of foreground and background. Background Technology

[0002] Underwater image enhancement technology plays a vital role in underwater archaeology, marine exploration, and underwater engineering. However, due to the complex underwater imaging environment, underwater images suffer from severe quality degradation, which limits their visibility and information acquisition capabilities. Therefore, researching effective underwater image sharpening methods is a crucial problem that urgently needs to be solved in the field of underwater vision.

[0003] In recent years, researchers have proposed many underwater image enhancement methods to address the aforementioned problems. These methods can be broadly categorized into image restoration-based methods, image enhancement-based methods, and deep learning-based methods. However, image restoration-based methods require a large number of prior assumptions, leading to difficulties in model optimization. Meanwhile, image enhancement-based methods typically neglect underwater image imaging models, resulting in potential issues such as local color distortion and noise amplification in the enhanced image. Furthermore, deep learning-based methods require extensive underwater image training for the model, but acquiring realistic underwater images is extremely difficult, thus limiting research in this area.

[0004] Furthermore, the combination of color compensation and color balance has gradually become the mainstream method for color correction of underwater images. Although these methods have good performance in color correction, their effectiveness and robustness still need to be further improved for severely degraded underwater images.

[0005] For example, the invention patent with application number CN201911319794.7 and invention title: "An Underwater Image Enhancement Method Based on Selective Color Compensation and Three-Interval Equalization" proposes that, based on a color-distorted underwater image, the image is decomposed into three channels: R, G, and B. The color mean S3 of each of the R, G, and B channels is calculated. The calculated mean values ​​are compared. When G is greater than B, the G channel image is selected as the compensation benchmark, and the R and B channels are compensated accordingly. When B is greater than G, the B channel image is selected as the compensation benchmark, and the R and G channels are compensated accordingly. This yields a color-corrected image. The R, G, and B channels are then decomposed into three sub-intervals using the lower quantization method, and different equalization operations are applied to each sub-interval. The equalized R, G, and B channels are then fused to obtain the final enhanced underwater image. This method addresses the problem from the perspective of attenuation compensation; however, excessive contrast enhancement may lead to the loss of image details.

[0006] In general, traditional color transfer methods rely on referenced land images for color correction, but this approach has limitations in terms of applying color distortion to diverse underwater images.

[0007] In order to solve the above problems, people have been seeking an ideal technological solution. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by providing an underwater robot vision enhancement method based on foreground and background principal component fusion. This method eliminates the need to rely on land reference images to correct color casts, preserves the texture details and other information of the original image while enhancing image contrast, and solves the problem of color distortion in underwater images.

[0009] To achieve the above objectives, the technical solution adopted by this invention is: an underwater robot vision enhancement method based on foreground and background principal component fusion, comprising the following steps:

[0010] Step 1) Obtain the color-distorted underwater image, perform box filtering on the L channel, and calculate the mean and variance of local image patches using the following formulas:

[0011]

[0012]

[0013]

[0014] in, It is the sum of pixels in a local block of an image. The value is set to 25. These are local block pixel values. Represents the local block mean. It is the original input image. This represents the image after the box filter has been applied.

[0015] Step 2) Based on the processing of the bright channel in Step 1), perform pixel equalization processing on channels a and b.

[0016] The calculation formulas for pixel processing in channels a and b are expressed as follows:

[0017]

[0018]

[0019]

[0020]

[0021] in, and These are the pixel averages of channels a and b, respectively. and These are the length and width of the input image. The input a-channel image, The input b-channel image.

[0022] Step 3) Based on the color-corrected image obtained in Step 2), the calculation formula for separating the image into foreground and background images is as follows:

[0023]

[0024]

[0025]

[0026] in, It is the critical value for image separation. =0.5, It is a color-corrected image. , It is the foreground image extracted pixel by pixel from the color-corrected image. It is the background image extracted pixel by pixel from the color-corrected image.

[0027] Step 301) For the image separation in step 3), for the color-corrected foreground image, traverse the RGB channels of the color-corrected image, and replace the pixel values ​​in the image channels that are less than the first threshold with the first threshold, and replace the pixel values ​​that are greater than the second threshold with the second threshold, to obtain the contrast-enhanced image.

[0028] The formula for calculating the foreground image contrast enhancement is as follows:

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] in, It is the number of pixels in the image. and It refers to the length and width of the image. It's the percentile. and It represents the position of the image pixel corresponding to the percentile. It is the threshold corresponding to the percentile. The array represents the number of pixels in the image whose values ​​are below a threshold. Update to threshold After experimental verification . Percentiles The corresponding threshold is used to determine if the pixel value in the image is greater than the threshold. Update to threshold , . It is an image with enhanced contrast. It is after passing through a threshold and less than the threshold Updated image, It is the minimum value of the processed image. It is the maximum value of the processed image. .

[0035] Step 302) For the image separation in Step 3), the background image is divided into four regions. The pixel value and channel pixel average of each region are calculated, and then the standard deviation is calculated based on the pixel average. Based on the known channel pixel average and standard deviation, the region score is calculated, and finally, the region with the highest score is selected to obtain the atmospheric light.

[0036] The formula for calculating the atmospheric light value is as follows:

[0037]

[0038]

[0039]

[0040] in, , , and It is a four-part image of the background. and It is the length and width of the background image. This is a background image. It is the average pixel value of the RGB channels of the region image. It is the standard deviation of the corresponding channel. It is the pixel score value of the RGB channels of the region image. The area with the highest rating.

[0041] Step 303) Based on the image scene depth local similarity assumption, the image is divided into 15×15 equal-sized regions. The transport mapping is estimated for each region, and the transport mappings are merged together to obtain the transport mapping of the entire image.

[0042] The calculation formula for the transmission graph is expressed as follows:

[0043]

[0044]

[0045]

[0046] in, This is the image after dehazing. Atmospheric light value, , Input background image patch, This is the initial transmission diagram. It is the sum of information loss during the dehazing process. Pixels with values ​​greater than 255 or less than 0 in the dehazed image block are cropped out. The value of the cost function is used to evaluate the dehazed image quality corresponding to the currently updated transport graph in the iterative update. It is a parameter that adjusts the trade-off between information loss and mean deviation. It is the total number of pixels in the image block. Used to measure the contrast of an image. Used to measure the brightness of an image.

[0047] Step 4) Calculate the fusion weights by analyzing the eigenvalues ​​and eigenvectors of the covariance matrices of the enhanced and dehazed images, and then linearly combine the enhanced and dehazed images according to the weights to obtain the fused image.

[0048] The calculation formula for the fused image is expressed as follows:

[0049]

[0050]

[0051] in, It is a foreground contrast enhanced image. It is a background image with dehazing. It is the covariance matrix. and It is the average pixel value of the image. It is the final enhanced image. and It is a normalized weight. It is a foreground contrast enhanced image. It is a background image with the haze removed.

[0052] This invention has outstanding substantive features and significant progress compared to the prior art. Specifically, this invention uses a color equalization strategy to perform color equalization on the a and b channels of the Lab color space for the original color-distorted underwater image, while simultaneously enhancing the contrast of the L channel to obtain a color-corrected image; then, the contrast of the color-corrected foreground image is enhanced, while the background image is dehazed using a dark channel prior; finally, a principal component analysis fusion strategy is introduced to obtain a clear underwater image by fusing the foreground and background components of images with different information.

[0053] This method eliminates the reliance on land reference images to correct color casts, preserves the texture details of the original image while enhancing image contrast, and solves the problem of color distortion in underwater images. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating an underwater robot vision enhancement method based on foreground and background principal component fusion according to the present invention.

[0055] Figure 2 This is an enhancement result and image histogram for the underwater UCCS dataset, as presented by the present invention and other methods.

[0056] Figure 3 This invention and other methods provide enhanced results and image histograms for the underwater UIQS dataset.

[0057] Figure 4 This is an enhancement result and image histogram for the underwater UIEB dataset, as presented by the present invention and other methods. Detailed Implementation

[0058] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0059] like Figure 1 As shown, Figure 1 As shown, an underwater robot vision enhancement method based on foreground and background principal component fusion includes the following steps:

[0060] Step 1) Obtain the color-distorted underwater image, perform box filtering on the L channel, and calculate the mean and variance of local image patches using the following formulas:

[0061]

[0062]

[0063]

[0064] in, It is the sum of pixels in a local block of an image. The value is set to 25. These are local block pixel values. Represents the local block mean. This is the original input image. This represents the image after the box filter has been applied.

[0065] Step 2) Based on the processing of the bright channel in Step 1), perform pixel equalization processing on channels a and b.

[0066] The calculation formulas for pixel processing in channels a and b are expressed as follows:

[0067]

[0068]

[0069]

[0070]

[0071] in, and These are the pixel averages of channels a and b, respectively. and These are the length and width of the input image. The input a-channel image, The input b-channel image.

[0072] Step 3) Based on the color-corrected image obtained in Step 2), the calculation formula for separating the image into foreground and background images is as follows:

[0073]

[0074]

[0075]

[0076] in, It is the critical value for image separation. =0.5, It is a color-corrected image. , It is the foreground image extracted pixel by pixel from the color-corrected image. It is the background image extracted pixel by pixel from the color-corrected image.

[0077] Step 301) For the image separation in step 3), for the color-corrected foreground image, traverse the RGB channels of the color-corrected image, and replace the pixel values ​​in the image channels that are less than the first threshold with the first threshold, and replace the pixel values ​​that are greater than the second threshold with the second threshold, to obtain the contrast-enhanced image.

[0078] The formula for calculating the foreground image contrast enhancement is as follows:

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] in, It is the number of pixels in the image. and It refers to the length and width of the image. It's the percentile. and It represents the position of the image pixel corresponding to the percentile. It is the threshold corresponding to the percentile. The array represents the number of pixels in the image whose values ​​are below a threshold. Update to threshold After experimental verification . Percentiles The corresponding threshold is used to determine if the pixel value in the image is greater than the threshold. Update to threshold , . It is an image with enhanced contrast. It is after passing a threshold and less than the threshold Updated image, It is the minimum value of the processed image. It is the maximum value of the processed image. .

[0085] Step 302) For the image separation in Step 3), the background image is divided into four regions. The pixel value and channel pixel average of each region are calculated, and then the standard deviation is calculated based on the pixel average. Based on the known channel pixel average and standard deviation, the region score is calculated, and finally, the region with the highest score is selected to obtain the atmospheric light.

[0086] The formula for calculating the atmospheric light value is as follows:

[0087]

[0088]

[0089]

[0090] in, , , and It is a four-part image of the background. and It is the length and width of the background image. This is a background image. It is the average pixel value of the RGB channels of the region image. It is the standard deviation of the corresponding channel. It is the pixel score value of the RGB channels of the region image. The area with the highest rating.

[0091] Step 303) Based on the image scene depth local similarity assumption, the image is divided into 15×15 equal-sized regions. The transport mapping is estimated for each region, and the transport mappings are merged together to obtain the transport mapping of the entire image.

[0092] The calculation formula for the transmission graph is expressed as follows:

[0093]

[0094]

[0095]

[0096] in, This is the image after dehazing. Atmospheric light value, , Input background image patch, This is the initial transmission diagram. It is the sum of information loss during the dehazing process. Pixels with values ​​greater than 255 or less than 0 in the dehazed image block are cropped out. The value of the cost function is used to evaluate the dehazed image quality corresponding to the currently updated transport graph in the iterative update. It is a parameter that adjusts the trade-off between information loss and mean deviation. It is the total number of pixels in the image block. Used to measure the contrast of an image. Used to measure the brightness of an image.

[0097] Step 4) Calculate the fusion weights by analyzing the eigenvalues ​​and eigenvectors of the covariance matrices of the enhanced and dehazed images, and then linearly combine the enhanced and dehazed images according to the weights to obtain the fused image.

[0098] The calculation formula for the fused image is expressed as follows:

[0099]

[0100]

[0101] in, It is a foreground contrast enhanced image. It is a dehazed background image. It is the covariance matrix. and It is the average pixel value of the image. It is the final enhanced image. and It is a normalized weight. It is a foreground contrast enhanced image. It is a background image with the haze removed.

[0102] The performance of this method is verified using the UCCS, UIQS, and UIEB datasets. The UCCS

[60] dataset contains three subsets: blue, green, and blue-green. Finally, the proposed method is used for ablation studies and extension studies to demonstrate its generalization ability. Subjective and objective comparisons are also made with WCBF and CCCE; deep learning methods: FGAN, PUIE-Net, UIEC-Net, Water-Net, and SGUIE-Net.

[0103] like Figure 2 As shown, the color correction performance of 10 different underwater image enhancement methods on UCCS

[60] was compared. IBLA

[48] , GDCP

[12] and GIFM

[15] introduced purple, dark red and green artifacts when restoring the image, and could not restore low-quality underwater images well. WCBF

[57] and CCCE

[30] effectively eliminated color difference, but these two methods had local color distortion and local overexposure problems when enhancing the image. The enhancement result of FGAN

[58] was yellow overall, while PUIE-Net

[59] and UIEC-Net

[35] corrected the color cast well and enhanced the image texture details, but the overall enhancement result was light blue. Water-Net

[32] corrected the color difference when enhancing the image, but introduced a slight blue color and local darkness in some areas. SGUIE-Net

[37] corrected the color cast of the image, but introduced a lot of purple artifacts. Figure 2The enhancement results and histogram distribution show that the method of the present invention has good performance in eliminating image color cast, improving image contrast and saturation, and enhancing image texture details.

[0104] like Figure 3 As shown, ten underwater image enhancement methods and the method of this invention are compared on the UIQS

[60] dataset to compare the visible performance of different methods in underwater image enhancement. Figure 3 As shown, IBLA

[48] and GIFM

[15] cannot restore the dataset, and GDCP

[12] and WCBF

[57] introduce purple artifacts when restoring the image. In contrast, CCCE

[30] can enhance the image better, but the enhancement result has local overexposure and detail blurring problems. FGAN

[58] introduces yellow artifacts when enhancing the image, and the overall enhancement result is yellowish. PUIE-Net

[59] and UIEC-Net

[35] have good performance in color correction when enhancing the image, but there is noise amplification in the local image. Water-Net

[32] has good texture details and color when enhancing the image, but the enhanced image has local color distortion problems. SGUIE-Net

[36] is bluish-purple when enhancing the dataset. In contrast, the method of the present invention corrects the image color cast better and improves the image contrast and texture details. In addition, from the perspective of the three-color histogram distribution, the image enhanced by the method of the present invention has a more uniform RGB distribution and a wider dynamic range.

[0105] like Figure 4 As shown, this invention compares ten underwater image enhancement methods with the proposed method on the UIEB

[32] dataset to compare the generalization performance of different methods in underwater image enhancement. Figure 4 As shown, IBLA

[48] , GDCP

[12] , and GIFM

[15] cannot restore this dataset. WCBF

[57] , PUIE-Net

[59] , and UIEC-Net

[35] fully consider the degradation principle of underwater images, improve image contrast and texture details, but the enhanced images have local color distortion. CCCE

[30] corrects the color cast of the image, but introduces red artifacts and blurs local details. The enhanced images of FGAN

[58] and SGUIE-Net

[37] introduce yellow and purple artifacts, and the local areas of the image are overexposed. In contrast, the image enhanced by this method has better visual clarity, improves image contrast, and enhances texture details.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A visual enhancement method for underwater robots based on foreground and background principal component fusion, characterized in that: Includes the following steps: Step 1) To obtain an underwater image with color distortion, first, a box filter is applied to the L channel to calculate the mean and variance of local blocks in the low-quality image. Step 2) Based on the processing of the L channel in Step 1), perform pixel equalization processing on the a and b channels; The calculation formula for pixel equalization processing of channels a and b is expressed as follows: in, and These are the pixel averages of channels a and b, respectively. and These are the length and width of the input image. The input a-channel image, The input b-channel image; Step 3) Based on the color-corrected image obtained in Step 2, separate the image into a foreground image and a background image. The calculation formula is as follows: in, It is the critical value for image separation. , It is a color-corrected image. , It is the foreground image extracted pixel by pixel from the color-corrected image. It is the background image extracted pixel by pixel from the color-corrected image; Step 301) For the color-corrected foreground image, traverse the RGB channels of the color-corrected image, and replace the pixel values ​​in the image channels that are less than the first threshold with the first threshold and the pixel values ​​that are greater than the second threshold with the second threshold according to the set threshold, so as to obtain the contrast-enhanced image. The formula for calculating the contrast enhancement of the foreground image in color correction is: in, It is the number of pixels in the image. and It refers to the length and width of the image. These are percentiles, where i and j are the positions of the image pixels corresponding to those percentiles. It is the threshold corresponding to the percentile. The array represents the number of pixels in the image whose values ​​are below a threshold. Update to threshold , Percentiles The corresponding threshold is used to determine if the pixel value in the image is greater than the threshold. Update to threshold ; It is an image with enhanced contrast. It is after passing a threshold and less than the threshold Updated image, It is the minimum value of the processed image. It is the maximum value of the processed image. ; Step 302) Divide the background image into four regions, calculate the pixel value and channel pixel average of each region, and then calculate the standard deviation based on the pixel average. Calculate the region score based on the known channel pixel average and standard deviation, and finally select the region with the highest score to obtain atmospheric light. The formula for calculating atmospheric light value is: in, , , and It is a four-part image of the background. and It is the length and width of the background image. Background image; It is the average pixel value of the RGB channels of the region image. It is the standard deviation of the corresponding channel. It is the pixel score value of the RGB channels of the region image. It is the area with the highest rating; Step 303) Based on the local similarity assumption of image scene depth, the image is divided into 15×15 equal-sized regions. The transport mapping is estimated for each region, and the transport mappings are merged together to obtain the transport mapping of the entire image. Step 4) Calculate the fusion weights by analyzing the eigenvalues ​​and eigenvectors of the covariance matrices of the enhanced and dehazed images, and then linearly combine the enhanced and dehazed images according to the weights to obtain the fused image; The formula for calculating the linear combination of the enhanced and dehazed images is as follows: in, It is a foreground contrast enhanced image. It is a dehazed background image. It is the covariance matrix. and It is the average pixel value of the image; It is the final enhanced image. and It is a normalized weight. It is a foreground contrast enhanced image. It is a background image with the haze removed.

2. The underwater robot vision enhancement method based on foreground and background principal component fusion according to claim 1, characterized in that: In step 1), the formulas for performing box filtering on the L channel and calculating the mean and variance of local image blocks are as follows: in, It is the sum of pixels in a local block of an image. The value is set to 25. These are local block pixel values. Represents the local block mean. It is the original input image. This represents the image after the box filter has been applied.

3. The underwater robot vision enhancement method based on foreground and background principal component fusion according to claim 1 or 2, characterized in that: In step 303), by assuming local similarity in scene depth, transport mapping is estimated for different regions. By merging the transport mappings, the formula for calculating the transport map of the image is as follows: in, This is the image after dehazing. Atmospheric light value, , Input background image patch, This is the initial transmission diagram. It is the sum of information loss during the dehazing process. Pixels with a value greater than 255 or less than 0 in the dehazed image block are cropped out. The value of the cost function is used to evaluate the dehazed image quality corresponding to the currently updated transport graph in the iterative update. It is a parameter that adjusts the trade-off between information loss and mean deviation. It is the total number of pixels in the image block; Used to measure the contrast of an image; Used to measure the brightness of an image.

Images