A method for collecting clear underwater images based on optimal band image fusion

Abstract

The present invention discloses a method for collecting clear underwater images based on optimal band image fusion, comprising the following steps: S1: collecting a hyperspectral image of a target to be observed in an application scenario, combining the spectral characteristics of the water body and the target, and selecting the optimal band for underwater observation using a band selection method based on multi-criteria decision-making; S2: constructing an RGB-optimal band underwater binocular imaging system, simultaneously acquiring underwater color images and optimal band images, and registering the acquired image pairs; S3: using a multimodal image fusion method based on visual saliency to fuse the complementary information of the image pairs acquired in S2 to obtain high-quality underwater images. The method utilizes the information in the optimal band image to enhance the color image, is an objective image quality enhancement method, and avoids information destruction caused by increasing contrast or misestimation of parameters in traditional methods.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image processing, and in particular to a method for collecting clear underwater images based on optimal band image fusion. Background Art

[0002] Underwater images usually suffer from image quality degradation due to light scattering and absorption, which reduces the accuracy of underwater target detection and recognition and further hinders the development of vision-guided autonomous underwater vehicles (AUVs). Therefore, obtaining clear underwater images with rich details and prominent targets is an urgent problem to be solved in the current field of marine engineering and underwater robots. Traditional underwater image sharpening methods include enhancement and restoration. Enhancement methods such as white balance, color correction, and histogram equalization can make images clear to a certain extent, but they are rarely effective for underwater images with complex physical properties and they usually do not consider the causes of degradation. Image restoration methods usually estimate the interference factors affecting the image through certain prior knowledge or assumptions to eliminate the degradation effects. However, post-processing operations or inaccurate parameter estimation can cause damage to image pixels. Therefore, it is still a challenge to obtain clear underwater images in different scenes or under different distortion conditions. Summary of the invention

[0003] In view of the problems existing in the prior art, the present invention discloses a method for collecting clear underwater images based on optimal band image fusion, which specifically includes the following steps:

[0004] Collect hyperspectral images of the target to be observed, and select the best band for underwater observation based on the spectral characteristics of the water body and the observed target and the band selection method based on multi-criteria decision-making;

[0005] The underwater color image and the best band image are captured by a binocular imaging system, and the acquired image pairs are registered to obtain a registered image pair;

[0006] A multimodal image fusion method based on visual saliency is used to fuse the complementary information of the registered image pairs to obtain high-quality underwater images.

[0007] Four different evaluation indicators are established using limited hyperspectral image data as prior information. The standardized score of each band image under four different evaluation indicators is obtained from four aspects: information entropy, clarity, target saliency measurement and target separability measurement. Each standardized score is adaptively weighted using the distribution characteristics of the prior information, and a decision matrix is ​​constructed. The decision matrix is ​​used to comprehensively evaluate each band, and the band with the highest score is selected as the best band.

[0008] The color image in the registered image pair is converted into YCbCr color space and the brightness channel is extracted. The brightness channel and the best band image are decomposed using a Gaussian filter to obtain the detail layer and the base layer. The detail layer is fused using the maximum absolute value fusion rule. The base layer is subjected to pixel-level saliency value calculation and histogram equalization processing to obtain the saliency weight, and the base layer is weighted fused. Finally, the fused base layer and the fused detail layer are combined and the color space is reconstructed to obtain an enhanced underwater image.

[0009] The enhanced underwater image is obtained specifically in the following manner:

[0010] Convert the acquired color image of the image pair into YCbCr space;

[0011] For the brightness channel Y of the color image and the best band image I OP Perform Gaussian filtering to obtain the base layer Y of the color image and the best band image B and

[0012] Y B =Gaussian(Y,σ s )

[0013]

[0014] By subtracting the base layer from the source image, we get the color image and the detail layer Y of the best band image. D and

[0015] Y D =YY 1

[0016]

[0017] Calculate the base layer Y of the color image and the best band image B and The visual saliency map V 1 and V 2 : Let V(p) be the saliency intensity of pixel p, N be the total number of pixels, I(p) be the intensity of pixel p, and the visual saliency map be calculated as follows:

[0018]

[0019] According to the visual saliency map V 1 and V 2 Compute base layer fusion weights for source image pairs and

[0020]

[0021]

[0022] Calculate the base layer F of the fused image B :

[0023]

[0024] According to the maximum absolute value fusion regulation, calculate the detail layer Y D and The corresponding fusion weight and

[0025]

[0026]

[0027] According to the detail layer fusion weight and Get the fused detail layer F D :

[0028]

[0029] The fused base layer F B and fused detail layer F D Add and combine with color information Cb, Cr to reconstruct the fused image into RGB color space to obtain the fused enhanced image F:

[0030] F=YCbCr2RGB((F B +F D ), Cb, Cr).

[0031] Due to the adoption of the above technical scheme, the present invention provides a method for collecting clear underwater images based on optimal band image fusion, which generates clear underwater images with richer details and more prominent targets by fusing complementary information of different modal images. Unlike traditional image post-processing methods based on enhancement or restoration, this method enhances image quality at the imaging stage and is a real-time image sharpening method. In addition, this method uses the information in the optimal band image to enhance the color image, which is an objective image quality enhancement method and avoids the information destruction caused by increasing contrast or misestimation of parameters in traditional methods. The present invention provides a new idea for solving the problem of underwater image quality degradation and has a strong reference significance for the design of underwater imaging systems. In addition, the present invention provides a basis for underwater robots to realize accurate target recognition, target capture and other tasks, and the problems it solves are of great practical significance for accelerating the construction of marine ranches. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 The present invention is a flow chart of a method for collecting clear underwater images based on optimal band image fusion.

[0034] Figure 2 Schematic diagram of the process of generating high-quality underwater images in the method of the present invention. DETAILED DESCRIPTION

[0035] In order to make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0036] like Figure 1 The method for collecting clear underwater images based on optimal band image fusion shown in the figure specifically comprises the following steps:

[0037] S1: In the application scenario, the hyperspectral image of the target to be observed is collected, and the spectral characteristics of the water body and the target are combined to select the best band for underwater observation using a band selection method based on multi-criteria decision-making;

[0038] S2: Build an RGB-optimal band underwater binocular imaging system to simultaneously acquire underwater color images and optimal band images, and register the acquired image pairs to obtain registered image pairs;

[0039] S3: A multimodal image fusion method based on visual saliency is used to fuse the complementary information of the image pairs obtained in S2 to obtain high-quality underwater images.

[0040] The specific method used in S1 is as follows:

[0041] S11: Collect the hyperspectral image of the target to be observed in the application scenario and perform reflectance correction;

[0042] S12: Normalize data by band to between 0 and 255;

[0043] S13: For each single-band image with wavelength λ, according to the density function of its corresponding reflectivity value i Calculate its information entropy H(λ):

[0044]

[0045] S14: Use the Sobel operator to calculate the gradient value G(x, y, λ) of each pixel (x, y, λ), and use the Tenengrad gradient function to calculate the clarity T(λ) of each band image:

[0046]

[0047] S15: Calculate the target saliency measure S(λ) of each band image according to the reflectivity intensity I(x, y, λ) and saliency intensity V(x, y, λ) of each pixel in the target pixel set Ω′:

[0048]

[0049] S16: For each single-band image with a wavelength of λ, the mean reflectivity μ of the target pixel is included. o (λ) and standard deviation σ o (λ), and the mean reflectance μ of the background pixels b (λ) and standard deviation σ b (λ), calculate its target separability measure D(λ):

[0050]

[0051] S17: Normalize H(λ), T(λ), S(λ), and D(λ) obtained in S13-S16 to between 0 and 1, and obtain the standardized scores f of the L band images under four different evaluation indicators 1 (λ), f 2 (λ), f 3 (λ), f 4 (λ), and construct the decision matrix M:

[0052]

[0053] S18: According to the decision matrix M in S17, the positive ideal solution PIS and the negative ideal solution NIS are obtained:

[0054]

[0055]

[0056] S19: According to the standard deviation σ of each column in the decision matrix M i , calculate the weights w of the four evaluation indicators i :

[0057]

[0058] S20: According to the decision matrix M and indicator weight w i, calculate the distance between the score of each band and the positive and negative ideal solutions and

[0059]

[0060]

[0061] S21: According to S20 and Calculate the score Score(λ) of each band, and select the band with the highest score as the best observation band. The calculation method of Score(λ) is:

[0062]

[0063] The specific method used in S2 is as follows:

[0064] S21: Build an RGB-optimal band underwater binocular imaging system, the RGB-optimal band underwater binocular imaging system includes: an RGB camera and a full-color camera, wherein a liquid crystal tunable filter is arranged at the front end of the camera; other components include Nvidia Jetson TX2, STM32 controller and battery module; all the above components are fixed in a waterproof housing; an LED supplementary light source is fixed on the housing and on both sides of the RGB camera and the full-color camera respectively;

[0065] S22: placing the calibration plate in the field of view of the RGB-optimal band underwater binocular imaging system described in S21, moving it left and right to obtain multiple sets of calibration images, and obtaining calibration parameters of the binocular imaging system.

[0066] S23: carrying the calibrated imaging system on the underwater robot to simultaneously obtain underwater color images and optimal band images;

[0067] S24: According to the calibration parameters calculated in S22, the underwater RGB-optimal band image pair obtained in S23 is registered.

[0068] The specific method used in S3 is as follows:

[0069] S31: for the image pair obtained in S2, convert the color image into YCbCr space;

[0070] S32: Brightness channel Y of color image and optimal band image I OP Perform Gaussian filtering to obtain the base layer Y of the color image and the best band image B and

[0071] Y B =Gaussian(Y,σs )

[0072]

[0073] S33: Obtain the detail layer Y of the color image and the best band image by subtracting the base layer obtained in S31 from the source image D and

[0074] Y D =YY 1

[0075]

[0076] S34: Calculate the base layer Y of the color image and the best band image B and The visual saliency map V 1 and V 2 : Let V(p) be the saliency intensity of pixel p, N be the total number of pixels, I(p) be the intensity of pixel p, and the visual saliency map be calculated as follows:

[0077]

[0078] S35: Based on the visual saliency map V obtained in S34 1 and V 2 , calculate the base layer fusion weights of the source image pair and

[0079]

[0080]

[0081] S36: Calculate the base layer F of the fused image B :

[0082]

[0083] S37: Calculate the detail layer Y according to the maximum absolute value fusion regulation D and The corresponding fusion weight and

[0084]

[0085] S38: Detail layer fusion weights calculated in S37 and Get the fused detail layer F D :

[0086]

[0087] S39: The fused base layer F in S36 B and the fused detail layer F in S38 D Add them together and combine them with the color information Cb, Cr in S1 to reconstruct the fused image into RGB color space to obtain the fused enhanced image F:

[0088] F=YCbCr2RGB((F B +F D ), Cb, Cr)

[0089] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for collecting clear underwater images based on optimal band image fusion. Features include: Collect hyperspectral images of the target to be observed, and select the best band for underwater observation based on the spectral characteristics of the water body and the observed target and the band selection method based on multi-criteria decision-making; The underwater color image and the best band image are captured by a binocular imaging system, and the acquired image pairs are registered to obtain a registered image pair; A multimodal image fusion method based on visual saliency is used to fuse the complementary information of the registered image pair to obtain high-quality underwater images; specifically, the color image in the registered image pair is converted into YCbCr color space and the brightness channel is extracted, the brightness channel and the best band image are decomposed using a Gaussian filter to obtain a detail layer and a base layer, the detail layer is fused using the maximum absolute value fusion rule, the base layer is subjected to pixel-level saliency value calculation and histogram equalization processing to obtain the saliency weight, and the base layer is weightedly fused, and finally the enhanced underwater image is obtained by combining the fused base layer and the fused detail layer and reconstructing the color space.

2. The method according to claim 1, Features: Four different evaluation indicators are established using limited hyperspectral image data as prior information. The standardized score of each band image under four different evaluation indicators is obtained from four aspects: information entropy, clarity, target saliency measurement and target separability measurement. Each standardized score is adaptively weighted using the distribution characteristics of the prior information, and a decision matrix is ​​constructed. The decision matrix is ​​used to comprehensively evaluate each band, and the band with the highest score is selected as the best band.

3. The method according to claim 1, Features: The enhanced underwater image is obtained specifically in the following manner: Convert the acquired color image of the image pair into YCbCr space; For the brightness channel Y of the color image and the best band image I OP Perform Gaussian filtering to obtain the base layer Y of the color image and the best band image B and Y B =Gaussian(Y,σ s ) By subtracting the base layer from the source image, we get the color image and the detail layer Y of the best band image. D and AND D =YY 1 Calculate the base layer Y of the color image and the best band image B and The visual saliency map V 1 and V 2 : Let V(p) be the saliency intensity of pixel p, N be the total number of pixels, I(p) be the intensity of pixel p, and the visual saliency map be calculated as follows: According to the visual saliency map V 1 and V 2 Compute base layer fusion weights for source image pairs and Calculate the base layer F of the fused image B : According to the maximum absolute value fusion regulation, calculate the detail layer Y D and The corresponding fusion weight and According to the detail layer fusion weight and Get the fused detail layer F D : The fused base layer F B and fused detail layer F D Add them together and combine them with the color information Cb, Cr to reconstruct the fused image into the RGB color space to obtain the fused enhanced image F: F=YCbCr2RGB((F B +F D ),Cb,Cr)。

Images