A multi-cue underwater image restoration method combined with light field

Abstract

The application discloses a kind of multi-cue underwater image restoration methods of combining light field, comprising:1, respectively acquire the m different view angles of underwater turbid polarized light field image under polarization angle j;2, based on underwater turbid polarized light field image, calculate multi-cue transmissivity t f With light field clue combination polarization clue;3, respectively carry out full focus to the underwater turbid polarized light field image of polarization angle j under m view angles, obtain full focus polarization image L j ;4, based on full focus polarization image L j , obtain final background light B f With judging index combination;5, multi-cue transmissivity t f With final background light B f Substitute underwater scattering model and obtain underwater restoration image.The application can effectively solve the image blurring problem caused by underwater particle scattering, and improve the quality of transmissivity and background light estimation, while suppressing background light interference, improve the color restoration degree of restoration image.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of underwater image restoration, and particularly relates to a restoration method for underwater images. BACKGROUND

[0002] China has nearly 3 million square kilometers of marine territory. In order to rationally develop and utilize marine resources and explore the marine environment, underwater detection imaging technology emerges as the times require. Light in water will be scattered and absorbed by particles, which will cause the photographed image to be seriously disturbed, resulting in problems such as unclear image, poor contrast and color distortion, and cannot meet the needs of human underwater research and application. Therefore, it is of great significance and value to carry out underwater image restoration research and improve the clarity and color fidelity of underwater image targets.

[0003] At present, underwater image restoration methods mainly include image enhancement-based algorithms and underwater imaging physical model-based algorithms. The image enhancement-based algorithm mainly uses some basic algorithms in image processing such as histogram equalization, homomorphic filtering and wavelet transform to perform underwater image restoration operation. For example, an automatic color equalization algorithm introduces a brightness function to realize the core idea of Retinex theory. Some scholars use Hear wavelet decomposition to enhance image edges, which can enhance the visual effect of the image. The underwater imaging physical model-based algorithm mainly analyzes the degradation process of underwater images and establishes an underwater image degradation model, inverts the degradation model, and compensates for the distortion in the degradation process to obtain an underwater restored image. For example, an underwater dark channel prior algorithm is used to obtain the transmittance of the image and then perform image restoration. Li et al. restore underwater images by a method of blue-green channel defogging and red channel correction. Gao et al. are inspired by the dark channel algorithm to propose a prior bright channel algorithm.

[0004] The above technical methods have the following problems in specific implementation:

[0005] The image enhancement-based algorithm is based on the characteristics of underwater images themselves and not underwater physical conditions, so the restored image may have problems such as noise amplification and color distortion. In the underwater imaging physical model-based algorithm, although most prior algorithms can restore the image to a certain extent, they do not consider the polarization characteristics of light scattering in the medium, and the processing result is not ideal for severely degraded images. Moreover, most algorithms restore the image on the basis of a single image, which can improve the low contrast and color distortion of the underwater image to a certain extent, but lacks the direction and angle information of the light field of the single image, and the estimation of the background light is not accurate, resulting in problems such as poor background light suppression, high texture noise and low clarity. SUMMARY

[0006] To address the shortcomings of existing technologies, this invention proposes a multi-cue underwater image restoration method that combines light field to effectively solve the image blurring problem caused by underwater particle scattering and improve the quality of transmittance and background light estimation. This method can improve the color reproduction of the restored image while suppressing background light interference.

[0007] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0008] The present invention provides a multi-crucial underwater image restoration method combining light fields, characterized by the following steps:

[0009] Step 1: Obtain underwater turbidity polarized light field images from m different directional viewpoints at polarization angles of 0°, 45°, 90°, and 135° respectively;

[0010] Step 2: Based on the underwater turbid polarized light field image, calculate the multi-cable transmittance t at the center viewpoint by combining polarization cues and light field cues. f (x);

[0011] Step 2.1: Calculate the total polarization intensity image I at the i-th viewpoint using equation (1). i Thus, the set of multi-view polarization total light intensity images A = {I} is obtained. i |i=1,2,3,···m}:

[0012]

[0013] In equation (1), This represents the polarization image with a polarization angle of 0° at the i-th viewpoint. This represents a polarization image with a polarization angle of 90° at the i-th viewpoint;

[0014] Step 2.2: Based on the multi-view polarization total light intensity image set A={I i |i=1,2,3,···,m}, obtain its polar plane image by fixing one angular direction and spatial direction of A;

[0015] Step 2.3: Based on the polar plane image, obtain the transmittance t containing the light field cues using equation (2). l ;

[0016]

[0017] In equation (2), β is the underwater medium coefficient, f represents the focal length, and d represents the parallax value obtained through the polar plane image;

[0018] Step 2.4: Using the first Using one viewpoint as the central viewpoint, the polarization line transmittance t at the central viewpoint is calculated using equation (3). p:

[0019]

[0020] In formula (3), p represents the polarized image with a polarization degree of 0° at the i-th view angle, i = 1, 2, …, m, w, e are the length and width of the selected background region, b represents the polarized image with a polarization degree of 90° at the i-th view angle, i = 1, 2, …, m, w, e are the length and width of the selected background region, b represents the polarized image with a polarization degree of 0° at the i-th view angle, i = 1, 2, …, m, w, e are the length and width of the selected background region, b represents the polarized image with a polarization degree of 90° at the i-th view angle, i = 1, 2, …, m, w, e are the length and width of the selected background region, b max represents the brightest value of the background region, b min represents the darkest value of the background region, P represents the polarization degree:

[0021] Step 2.5: Calculate the multi-cue transmittance t f at the central view angle by using formula (4):

[0022]

[0023] In formula (4), p n represents the value of the polarized total light intensity map at the central view angle after turbidity normalization, p represents the parameter affecting the curve slope and central coordinates, and is obtained by formula (5):

[0024]

[0025] In formula (5), g l is the image with the lowest turbidity in the m underwater turbid polarized light field images of different directional view angles, and is taken as the clear underwater true value image, g p is the image with the highest turbidity in the m underwater turbid polarized light field images of different directional view angles, and is taken as the completely turbid underwater image, p nl is the peak signal-to-noise ratio of the normalized clear underwater true value image g′ l , p np is the peak signal-to-noise ratio of the normalized completely turbid underwater image g′ p ;

[0026] Step 3: Perform a full-focus operation on the underwater turbid polarized light field images with polarization angles of 0°, 45°, 90° and 135° at m view angles to obtain full-focus polarization images L0, L 45 , L 90 , L 135 at four polarization angles fused with m directional view angle information.

[0027] Step 4: Based on the full-focus polarization images L0, L 45 , L 90 , L 135 , and combined with the judgment index, the final background light B f;

[0028] Step 4.1: the full-focus polarization image L j is divided into N image blocks of the same size wherein, represents the L j th image block, j = 0°, 45°, 90°, 135°;

[0029] The judgment index of the L th image block is calculated by using formula (6) The image block corresponding to the maximum judgment index is obtained

[0030]

[0031] In formula (6), represents the mean value of the L th image block, and σ(x, y) represents the standard deviation of the L th image block, is an adjustment factor;

[0032] Step 4.2: the background light B j in the full-focus polarization image L j of the polarization angle j is calculated by using formula (7)

[0033]

[0034] In formula (7), mean() represents a mean value function, and j represents a polarization angle.

[0035] Step 4.3: the final background light B f is calculated by using formula (8):

[0036] B f = max(B j ), j = 0°, 45°, 90°, 135° (8)

[0037] Step 5: the underwater restoration image R is obtained by using the underwater scattering physical model shown in formula (9).

[0038]

[0039] The electronic device comprises a memory and a processor, and the memory is used to store a program supporting the processor to execute the multi-thread underwater image restoration method, and the processor is configured to execute the program stored in the memory.

[0040] The computer readable storage medium stores a computer program, and the computer program is run by a processor to execute the steps of the multi-thread underwater image restoration method.

[0041] Compared with the prior art, the underwater image restoration method has the advantages that:

[0042] 1、The present application combines light field imaging technology and polarization technology to improve the calculation quality of transmittance and background light, overcome the information distortion and loss of underwater single-dimensional images, effectively suppress the interference of background light, and effectively improve the color fidelity and clarity of the restored underwater image.

[0043] 2、The present application uses light field imaging technology to calculate the scene depth using light field epipolar images to obtain transmittance containing light field threads, which can effectively improve the texture details and color fidelity of the restored image.

[0044] 3、The present application uses Stokes vector to calculate transmittance containing polarization threads using polarization images, which can effectively suppress background light and improve the clarity of the restored image during image restoration.

[0045] 4、The present application fuses light field and polarization threads to obtain multi-thread transmittance according to the turbidity of underwater images, which can effectively combine the advantages of light field and polarization to improve the quality of transmittance estimation.

[0046] 5、The present application estimates background light through light field full-focus operation, which can effectively overcome the information distortion and loss of underwater single-dimensional images, suppress the interference of background light, and improve the quality of background light estimation. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The present application is an experimental scene diagram;

[0048] Figure 2 It is an epipolar image diagram;

[0049] Figure 3 It is an epipolar slope diagram. DETAILED DESCRIPTION

[0050] In this embodiment, a multi-thread underwater image restoration method combining light field is applied to a scene as shown in the figure, which includes a potential displacement platform 1, a polarization camera 2, and a target object 3. Figure 1 The underwater image restoration method is performed according to the following steps:

[0051] Step 1: Obtain m underwater turbid polarization light field images of different direction views at polarization angles of 0°, 45°, 90° and 135°, respectively;

[0052] Step 1.1: In an underwater turbid scene, use a moving electric displacement platform and a polarization camera to capture m underwater turbid polarized light field images from different directional perspectives. The electric displacement platform is used to acquire images from m different directional perspectives, and the polarization camera is used to acquire images from different polarization angles.

[0053] Step 1.2: In underwater scenes with different turbidity, repeat the shooting steps in Step 1.1 to obtain underwater turbidity polarized light field images under different turbidity.

[0054] Step 2: Based on the underwater turbid polarized light field image, calculate the multi-cable transmittance t at the center viewpoint by combining polarization cues and light field cues. f (x);

[0055] Step 2.1: Calculate the total polarization intensity image I at the i-th viewpoint using equation (1). i Thus, the set of multi-view polarization total light intensity images A = {I} is obtained. i |i=1,2,3,···m}:

[0056]

[0057] In equation (1), This represents the polarization image with a polarization angle of 0° at the i-th viewpoint. This represents a polarization image with a polarization angle of 90° at the i-th viewpoint;

[0058] Step 2.2: Based on the multi-view polarization total light intensity image set A={I i |i=1,2,3,···,m}, such as Figure 2 As shown, (s,t) represents spatial information, and (u,v) represents angular information. The polar plane image of A is obtained by fixing one angular direction and one spatial direction. Figure 2 The lower and right sides are schematic diagrams of the polar plane;

[0059] Step 2.3: Based on the polar plane image, such as Figure 3 As shown, the parallax d is estimated by detecting the slope of the oblique lines in the polar plane image. The slope of the corresponding polar plane image is different for target points at different depths. The transmittance t containing the light field cues is obtained by equation (2). l ;

[0060]

[0061] In equation (2), β is the underwater medium coefficient, f represents the focal length, and d represents the parallax value obtained through the polar plane image;

[0062] Step 2.4: Using the first The polarization degree of the central view angle image is calculated by the Stokes vector, the brightest value and the darkest value of the background region of the polarization image are obtained by selecting the background region of the polarization image, and finally the polarization clue transmittance t of the central view angle is calculated by formula (3) p :

[0063]

[0064] In formula (3), represents the polarization image with a polarization degree of 0° at the first view angle, represents the polarization image with a polarization degree of 90° at the first view angle, w and e are the length and width of the selected background region, b max represents the brightest value of the background region, b min represents the darkest value of the background region, and P represents the polarization degree:

[0065] Step 2.5: Calculate the multi-clue transmittance t of the central view angle by formula (4) f :

[0066]

[0067] In formula (4), p n represents the value of the polarization total light intensity image of the central view angle after turbidity normalization, represents a parameter affecting the curve slope and central coordinates, and is obtained by formula (5):

[0068]

[0069] In formula (5), g l is the image with the lowest turbidity in the m underwater turbid polarized light field images of different direction view angles, and is taken as the clear underwater true value image, g p is the image with the highest turbidity in the m underwater turbid polarized light field images of different direction view angles, and is taken as the completely turbid underwater image, p nl is the peak signal-to-noise ratio of the normalized clear underwater true value image g l ′, p np is the peak signal-to-noise ratio of the normalized completely turbid underwater image g′ p ;

[0070] Step 3: Perform a full focus operation on the underwater turbid polarized light field images with polarization angles of 0°, 45°, 90° and 135° at m view angles to obtain four full focus polarization images L0, L 45 , L 90 , L 135 with the fusion of m direction view angle information;

[0071] Step 3.1: Refocus the underwater turbid polarized light field images with polarization angles of 0°, 45°, 90° and 135° under m views to generate focus stack images under four polarization angles;

[0072] Step 3.2: Calculate the gradient value of each focus stack image under each polarization angle based on the focus stack images under four polarization angles, realize multi-focus image fusion according to the maximum gradient value index of each pixel coordinate, and obtain the scene full-focus polarization images L0, L 45 , L 90 , L 135 ;

[0073] Step 4: Obtain the final background light B 45 , B 90 , B 135 based on the full-focus polarization images L0, L f ;

[0074] Step 4.1: Divide the full-focus polarization images L j , j = 0°, 45°, 90°, 135° into N image blocks with the same size respectively Use formula (6) to construct the judgment index Q, calculate the Q value of each image block to obtain the image block corresponding to the maximum Q value of the full-focus polarization image L j ;

[0075]

[0076] In formula (6), μ(x, y) represents the average value of the image block, σ(x, y) represents the standard deviation, is an adjustment factor;

[0077] Step 4.2: Calculate the background light B0, B 45 , B 90 , B 135 in the full-focus polarization images L0, L 45 , L 90 , L 135 ;

[0078]

[0079] In formula (7), mean() represents the mean function, B j represents the background light in the full-focus polarization image L j , j represents the polarization angle, represents the image block with the maximum Q value in the full-focus polarization image L j ;

[0080] Step 4.3: Calculate the final background light B using formula (8) f :

[0081] B f = max(B0, B 45 , B 90 , B 135 ) (8)

[0082] Step 5: Substitute the multi-thread transmittance t f and the final background light B f into the underwater scattering physical model formula (9) to obtain the underwater restored image R.

[0083]

[0084] In formula (9), R represents the underwater restored image, represents the polarization total light intensity image of the central view angle, t f represents the multi-thread transmittance, and B f represents the final background light.

[0085] In this embodiment, an electronic device includes a memory for storing a program supporting a processor to execute the above multi-thread underwater image restoration method, and the processor is configured to execute the program stored in the memory.

[0086] In this embodiment, a computer readable storage medium stores a computer program on the computer readable storage medium, and the computer program is executed by a processor to perform the steps of the above multi-thread underwater image restoration method.

Claims

1. A multi-cue underwater image restoration method that combines light field, characterized in that, is performed as follows: Step 1: acquire m underwater turbid polarized light field images of different direction views at polarization angles of 0°, 45°, 90° and 135° respectively; Step 2: Compute the multi - cue transmittance t at the center view based on the underwater turbid polarized light field image, combining the polarized cue and the light field cue f (x); Step 2.1: Calculate the polarized total light intensity image Ii at the ith view angle by formula (1) i Thus, the multi-view polarized total light intensity image set A = {I i |i = 1, 2, 3, ··· m} is obtained: In formula (1), denotes a polarized image with a polarization angle of 0° at the i-th viewing angle, denotes a polarized image with a polarization angle of 90° at the i-th viewing angle; Step 2.2: Based on the multi-view polarized total light intensity image set A = {I i | i = 1, 2, 3, ···, m}, the polar image of A is obtained by fixing one angle direction and spatial direction of A; Step 2.3: Based on the epipolar images, the transmittance t containing the light field cues is obtained by equation (2) l ; In formula (2), β is a coefficient of underwater medium, f represents a focal length, and d represents a parallax value obtained through an epipolar image; Step 2.4: With the first view as the center view, the polarization cue transmittance t : of the center view is calculated using equation (3) p : In formula (3), denotes the polarized image with a polarization degree of 0° at the viewing angle of the (i+1)th view, denotes the polarized image with a polarization degree of 90° at the viewing angle of the (i+1)th view, w, e are the length and width of the selected background region, b max denotes the brightest value of the background region, b min denotes the darkest value of the background region, P denotes the polarization degree: Step 2.5: Calculate the multi-clue transmittance t in the center view using formula (4) f : In formula (4), p n denotes the value of the haze-normalized total intensity plot for the central viewing angle, denotes a parameter that influences the slope of the curve and the center coordinate and is obtained from formula (5): In formula (5), g l is the image with the lowest turbidity among the m underwater turbid polarized light field images of different directional views, and is taken as the clear underwater true value image, g p is the image with the highest turbidity among the m underwater turbid polarized light field images of different directional views, and is taken as the completely turbid underwater image, p nl is the normalized clear underwater true value image g l ′, and p np is the peak signal-to-noise ratio of the normalized completely turbid underwater image g′ p ; Step 3: Perform all-focus operation on the underwater turbid polarized light field images with polarization angles of 0°, 45°, 90°, and 135° under m views to obtain all-focus polarization images L0, L 45 , L 90 , L 135 ; Step 4: Based on the full-focus polarization image L0, L 45 , L 90 , L 135 , combined with the judgment index to obtain the final background light B f ; Step 4.1: Full focus polarized image L j divided into N image blocks of the same size wherein, denotes L j the nth image block, j = 0°, 45°, 90°, 135°; The judging index of the nth image block is calculated by using formula (6) The judging index of the nth image block is calculated by using formula (6) The judging index of the nth image block is calculated by using formula (6) ​ In formula (6), denotes the average value of the nth image block , and σ(x, y) denotes the standard deviation of the nth image block , is an adjustment factor; Step 4.2: Compute the full-focus polarization image L at polarization angle j using formula (7) j background light B in the image L j ; In formula (7), mean() represents a mean value function, and j represents a polarization angle; Step 4.3: Calculate the final background light B using formula (8) f : B f = max(B j ), j = 0°, 45°, 90°, 135° (8) Step 5: obtain an underwater recovered image R by using an underwater scattering physical model shown in formula (9); 2. An electronic device comprising a memory and a processor, characterized in that The memory is configured to store a program supporting the processor to execute the multi-clue underwater image recovery method in claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, performs the steps of the multi-clue underwater image recovery method in claim 1.

Images