An underwater image restoration method based on multi-frame images under artificial illumination

Abstract

The application provides an underwater image restoration method based on multiple frames of images under artificial light, and belongs to the technical field of image processing, and comprises the following steps: (1) a systematic method for estimating background light is proposed to estimate the background light in the region; (2) a transmittance solving method based on multiple frames of sequence images is proposed to obtain the transmittance of the corresponding channel of the underwater image; and finally, the background light and the transmittance in the contents (1) and (2) are used to realize image restoration through an underwater image restoration formula. The application can more accurately restore the definition of the image in a turbid water body environment under artificial light, and the method has higher adaptability to the turbid water body.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method for underwater image restoration based on multiple frames under artificial lighting. Background Technology

[0002] When underwater robots perform tasks such as close-range underwater target detection and identification, the unique underwater imaging environment presents challenges. Water selectively absorbs light, and suspended particles in the water cause severe scattering, leading to significant degradation of underwater images based on optical vision. This results in blurry images, low contrast, color distortion, and excessive noise. Therefore, restoring these severely degraded underwater visual images to enable accurate and rapid target identification, while maintaining the autonomy and real-time performance required for underwater robot operations, is a crucial problem that urgently needs to be solved.

[0003] Underwater image restoration primarily involves physical modeling of the underwater image degradation mechanism, reversing the degradation process to obtain a clear underwater image. Existing typical underwater image restoration methods improve and optimize fog dehazing methods based on dark channel priors and apply them to underwater environments. This method is called underwater dark channel prior. However, due to the lack of red channel information, the restored image exhibits color cast. Galdran et al. proposed a red channel prior method to address the rapid attenuation of red light in underwater environments, improving the dark channel prior method using the inverse of the red channel. However, this method does not provide ideal contrast enhancement for the restored image. To address the shortcomings of applying dark channel priors in underwater environments, Chiang et al. assumed that the normalized residual energy ratio in the underwater environment was known and proposed a method based on wavelength compensation and image dehazing, effectively enhancing the contrast of underwater images. However, most existing underwater restoration methods are only applicable to underwater images under natural lighting conditions and in relatively clear water environments. However, in actual underwater robot operations, such as identifying and detecting targets on the seabed, the water flow generated by the robot's propulsion often stirs up seabed sediment, making the images captured by the underwater robot's optical vision system unclear due to water turbidity. Existing underwater image restoration methods are unsuitable for this environment. Therefore, it is meaningful to research an underwater image restoration method based on multiple frames of images suitable for artificial lighting conditions, particularly in turbid water environments. Summary of the Invention

[0004] The purpose of this invention is to provide an underwater image restoration method based on multiple frames under artificial lighting. This patent can solve the problem of unsatisfactory image processing results caused by inaccurate estimation of parameters such as background light and transmittance when processing images in turbid water environments under artificial lighting, thereby improving the underwater image restoration effect.

[0005] The objective of this invention is achieved as follows: The steps are as follows:

[0006] Step (1): Input the original underwater image. For turbid water bodies under artificial lighting, use the image segmentation method based on the YCbCr color space and the quadtree method to estimate the background light B of the underwater image. λ λ is the wavelength, λ∈{R,G,B};

[0007] Step (2): Combining the process of the underwater robot approaching the target from a distance, acquire multiple frames of images. Utilize the light intensity attenuation relationship between corresponding points in the multiple frames to first solve for the total attenuation coefficient c. λ The image depth d(x) is obtained by using a depth estimation method based on image saturation; the total attenuation coefficient c of each channel is calculated accordingly. λ Given the depth of field d(x) of the underwater image, calculate the transmittance t of the corresponding channel. λ (x);

[0008] Step (3): Combining Step (2) and Step (3), the underwater image background light estimation result and transmittance estimation result are given. The underwater image is restored according to the underwater image restoration formula, and the final clear underwater image is output.

[0009] The present invention also includes the following structural features:

[0010] 1. Step (1) specifically includes:

[0011] This invention proposes a systematic method for estimating background light. First, based on an underwater illumination image segmentation method using the YCbCr color space, to avoid high similarity between the target object and the background at grayscale levels during image grayscale conversion, the image is first converted to the YCbCr color space. Then, information from the Cb and Cr component channels is fused, and threshold segmentation is used to obtain the foreground and background. Next, a quadtree block method is used to quickly locate the background light region within the segmented background. Finally, in the finally located image block region, pixel values ​​are sorted from largest to smallest, and the top 10% of pixel values ​​are removed to eliminate interference. The average of the remaining pixel values ​​is then used to obtain the final background light B. λ .

[0012] 2. Step (2) specifically includes:

[0013] This invention proposes a method for calculating transmittance based on a multi-frame sequence of images. Light attenuates exponentially with increasing distance *d* when propagating in water, and transmittance is typically expressed as *t*. λ Transmittance (x) reflects the degree of attenuation and is an important parameter for compensating for the energy attenuation of light at different wavelengths in underwater image restoration. The expression for transmittance is: In the formula c λ Let be the total attenuation coefficient, and d(x) be the distance between point x on the target and the underwater camera. Therefore, in order to obtain an accurate transmittance of an underwater image, c must be calculated separately. λ and d(x).

[0014] First, by observing the underwater robot's approach to the target from a distance, multiple frames of images are acquired. Then, the total attenuation coefficient c is calculated using the light intensity attenuation relationship between corresponding points in these multiple frames. λ According to the underwater imaging model, the optical loop attenuation process at point x on the target is as follows:

[0015]

[0016] In the formula:

[0017] E λ,0 (x) represents the light intensity at point x on the target object;

[0018] E λ,d (x) represents the light intensity received by the camera at a distance d from the target object;

[0019] x represents the coordinates of a point in the scene, x = (x', y');

[0020] c λ This is the total attenuation coefficient;

[0021] d is the distance between point x on the target and the underwater camera.

[0022] The above formula can be transformed into:

[0023]

[0024] Based on the laws of light propagation in water, images at different distances *d* can all correspond to the attenuation formula mentioned above. Considering the process of an underwater robot approaching a target from a distance, and acquiring two frames of images, assuming the depths of field for the two frames are *d1* and *d2* respectively, then:

[0025]

[0026] In the formula: It is expressed as the light intensity received by the underwater camera at a distance d1 from the target object; B represents the light intensity received by the underwater camera at a distance d2 from the target object; λ 1 B λ 2 E represents the background light corresponding to images with depths of field d1 and d2, respectively; λ,1 (x1) and E λ,2 (x2) represents the light intensity at point x on the target object in the images with depths d1 and d2, respectively.

[0027] Take the logarithm of both sides of the above expression:

[0028]

[0029] The difference between the two equations is:

[0030]

[0031] E in the above formula λ,1 (x1) and E λ,2 (x2) is the light intensity reflected from the same point x on the target object in the images at distances d1 and d2. Assume the light intensity emitted by the artificial light source is E. λ,0 Then we have:

[0032]

[0033] In the formula: ρ λ It represents the reflectivity of the target object's surface.

[0034] Therefore, it can be simplified to:

[0035]

[0036] The above formula can be obtained by analyzing the light intensity attenuation pattern between two frames. Therefore, we can select a corresponding point on the sphere in the two-frame sequence, with depths of field d1 and d2 and light intensity E. λ,1 (x1) and E λ,2 (x2) (where the light intensity is the pixel value of the corresponding point) are known, so we can use these points to calculate the total light attenuation coefficient c. λ .

[0037] Secondly, the underwater image depth d(x) is obtained through a depth estimation method based on image saturation. Saturation generally refers to the purity of image colors. When an underwater target is illuminated by an artificial light source, the foreground white light in the underwater scene captured by the underwater camera increases, and the saturation decreases. As the distance increases, the effect of the artificial light source weakens, and the image saturation increases. Therefore, this invention uses image saturation to estimate the underwater image depth d(x).

[0038] The total attenuation factor c can be obtained through the above steps.λ And depth of field d(x), using transmittance Thus, the transmittance t of the underwater image was obtained. λ (x).

[0039] 3. Step (3) specifically includes:

[0040] Based on the Jaffe-McGlamery underwater optical model, combined with Figure 1 Underwater imaging model under artificial lighting:

[0041] I λ (x)=J λ (x)t λ (x)+B λ (1-t λ (x))

[0042] In the formula: I λ (x) represents the original image captured by the underwater camera; J λ (x) represents an ideal, clear underwater image.

[0043] Using the background light B obtained in steps (1) and (2) λ and transmittance t λ (x), for the input original image I λ (x) will output the restored clear underwater image J λ (x):

[0044]

[0045] Compared with existing technologies, the advantages of this invention are as follows: Existing underwater image restoration methods mainly target underwater environments under natural lighting. Research on underwater image restoration under artificial lighting is scarce. Due to the presence of artificial light sources, the brightness of the foreground is higher than that of the background, which is completely opposite to the properties of natural light. Furthermore, most methods rely on assumptions such as "the normalized residual energy ratio is known" and "the attenuation factor can be obtained empirically." However, the complex and variable operating environment of underwater robots makes it difficult to meet these assumptions. This invention, however, addresses practical needs and the working environment of most deep-sea underwater robots. On one hand, it designs an image segmentation method based on the YCbCr color space and a quadtree method for background light estimation, effectively avoiding interference from white objects, suspended objects reflecting light, and overexposed areas of artificial light sources. On the other hand, it designs a method based on multiple frame sequences of images, using the attenuation formula of light propagating in a medium to solve for the total light intensity attenuation coefficient. Simultaneously, it uses depth-of-field estimation based on saturation to obtain the depth of field d(x) of the underwater image. The total attenuation coefficient c obtained through the above methods is then calculated. λAnd the depth of field d(x), that is, the transmittance t of the corresponding channel of the underwater image. λ (x). This invention can more accurately restore the clarity of images in turbid water environments under artificial lighting, and the method has greater adaptability to turbid water bodies. Attached Figure Description

[0046] Figure 1 This is an underwater imaging model under artificial lighting in this invention patent.

[0047] Figure 2 This is the underwater imaging optical loop model of this invention patent.

[0048] Figure 3 These are underwater degradation images used in the experiments of this invention patent.

[0049] Figure 4 This is a comparison diagram of the restoration effects of the present invention and existing underwater image restoration algorithms. Detailed Implementation

[0050] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0051] Combination Figures 1 to 4 The specific implementation steps of the underwater image restoration method based on multiple frames under artificial lighting according to the present invention are as follows:

[0052] Step (1): Input the original underwater image. For turbid water bodies under artificial lighting, use the image segmentation method based on YCbCr color space and the quadtree method to estimate the background light of the underwater image.

[0053] For a single underwater image, the background light value is a constant, determined by the ambient light in the background area. However, due to the influence of artificial light sources, the brightness of the background area is lower than that of the foreground area. Therefore, classic methods easily misjudge bright spots on the target object as background light. Simultaneously, because the water is turbid and contains more suspended particles, when artificial light is present, these particles reflect light, creating numerous points with high brightness values. These points significantly interfere with the estimation of background light. Therefore, to accurately estimate the background light of an image, the characteristics of the background light in the working environment must first be clearly defined. Based on the above analysis, the background light under artificial light should have the following characteristics: the background light is the brightest point of light in the water (not an object) located at infinity; that is, the background light is not in the target area of ​​the foreground and is not reflected light from suspended objects.

[0054] Based on the above analysis, in order to ensure that the background light is not in the target area, we first consider separating the foreground and background areas, and then use the quadtree method to estimate the background light in the segmented background area.

[0055] To separate the foreground and background, image segmentation techniques are initially considered. However, traditional gray-level thresholding methods suffer from significant scattering due to suspended particles in turbid water, leading to similar gray-level appearances between the target and background in underwater images. This results in segmentation results with overlapping targets and backgrounds, and numerous interference areas. To address this issue, this invention proposes an underwater illumination image segmentation method based on the YCbCr color space, reducing noise interference in the segmentation results.

[0056] Since the original input underwater image is generally stored in RGB color space, we need to convert its color space first. Y is the brightness component of the color, while CB and CR are the concentration offset components of blue and red. The conversion between YCbCr color space and RGB color space is shown in formula (1), where the value range of Y component is [16,235], and the value range of Cb and Cr components is [16,240].

[0057]

[0058] This invention selects to fuse the information of Cb and Cr components in the YCbCr color space of underwater images, and then uses the classic Otsu thresholding method to segment the target object from the background.

[0059] First, the Cb and Cr component information of the underwater image is quantized from [16, 240] to [0, 255] to form a pseudo grayscale image corresponding to the Cb and Cr components. The size of the Cb and Cr component channel map is set to M×N, and the total number of gray levels in the corresponding pseudo grayscale image is L, n i Let p be the number of pixels at gray level i, and p be the probability of its occurrence. i A threshold t is selected to classify all pixels in the image into two categories: c1 (0≤c1≤t) and c2 (t≤c2≤L) (i.e., background and target). To ensure that the segmented target regions contain complete target objects, the two sets obtained from the Cb and Cr component classifications are merged. During the operation, the pixel gray values ​​in the foreground region of the classification set remain unchanged; only their coordinate positions are merged to obtain the final foreground and background segmentation results. This completes the classification of foreground targets and background in the underwater image, preparing for the next step of background light estimation.

[0060] Then, the quadtree method is used to determine the region where the background light is located in the background area. When dividing the image into blocks, the image blocks are first labeled clockwise as i = 1, 2, 3, 4, and the mean μ of each image block is calculated. i and standard deviation σ i And calculate Q for each block. i =μ i +σ i Compare Q i Between the sizes, select Q. i Repeat the above block division operation for the block with the smallest value until the size of the image block reaches the given threshold. This image block is the area where the background light is located.

[0061] Finally, the background light B was estimated. λ Within the final locked block area, pixel values ​​are sorted from largest to smallest. The top 10% of pixel values ​​are removed to eliminate interference, and the average of the remaining pixel values ​​is used to obtain the final background light B. λ The background light estimation method in this invention patent effectively avoids interference from pixels of white objects and overexposed areas of artificial light sources in the image, thus speeding up the background light estimation process.

[0062] Step (2): Combining the process of the underwater robot approaching the target from a distance, acquire multiple frames of images. Utilize the light intensity attenuation relationship between corresponding points in the multiple frames to first solve for the total attenuation coefficient c. λ The image depth d(x) is obtained through a depth estimation method based on image saturation. Finally, the total attenuation coefficient c of each channel is calculated. λ Given the depth of field d(x) of the underwater image, calculate the transmittance t of the corresponding channel. λ (x).

[0063] When light travels through water, its intensity decreases exponentially with increasing distance *d*, and this is generally expressed by the transmittance *t*. λ (x) reflects the degree of attenuation, and it is an important parameter for compensating for the energy attenuation of light at different wavelengths in underwater image restoration. Transmittance t λ The expression for (x) is:

[0064]

[0065] In the formula: x represents the coordinates of a point in the scene, x = (x', y'); d(x) represents the distance between a point x in the scene and the underwater camera, i.e., the depth of field. λ This represents the total attenuation coefficient of light with wavelength λ.

[0066] Therefore, in order to obtain an accurate transmittance of an underwater image, the total attenuation coefficient c must first be obtained. λ And depth of field d(x).

[0067] Total attenuation coefficient c λ Besides being affected by the wavelength of light, it is also influenced by seawater salinity and phytoplankton concentration. Currently, most studies provide an attenuation coefficient c based on the type of marine environment. λ Based on empirical values, this invention proposes to solve the total attenuation coefficient c based on the light source energy attenuation law between multiple frames of images. λ In underwater imaging models, the principles of optical path and loop are similar; this invention only selects the optical loop portion for study. Combined with... Figure 1 and Figure 2 According to the underwater imaging model, the optical loop attenuation process at point x on the target is as follows:

[0068]

[0069] In the formula: E λ,0 (x) represents the light intensity at point x on the target object; E λ,d (x) represents the light intensity received by the camera at a distance d from the target object;

[0070] Formula (3) can be varied as follows:

[0071]

[0072] Based on the laws of light propagation in water, images at different distances *d* can all correspond to the aforementioned attenuation formula. Considering the process of an underwater robot approaching a target from a distance, two frames are acquired. Assuming the depths of field of the two frames are *d1* and *d2*, respectively, and combining... Figure 2 have:

[0073]

[0074] In the formula: It is expressed as the light intensity received by the underwater camera at a distance d1 from the target object; This is expressed as the light intensity received by the underwater camera at a distance d2 from the target object; E represents the background light corresponding to images with depths of field d1 and d2, respectively; λ,1 (x1) and E λ,2 (x2) represents the light intensity at point x on the target object in the images with depths d1 and d2, respectively.

[0075] Take the logarithm of both sides of the above formula (5):

[0076]

[0077] The difference between the two equations is:

[0078]

[0079] E in equation (7) above λ,1 (x1) and E λ,2 (x2) is the light intensity reflected from the same point x on the target object in images d1 and d2. Assume the light intensity emitted by the artificial light source is E. λ,0 Then we have:

[0080]

[0081] In the formula: ρ λ It represents the reflectivity of the target object's surface.

[0082] Therefore, combining formula (8), formula (7) can be simplified to:

[0083]

[0084] The above formula can be obtained by analyzing the light intensity attenuation pattern between two frames. Therefore, we can select the coordinates of a corresponding point on the sphere in the two frames. For this point, the depth of field d1 and d2, and the light intensity E are given. λ,1 (x1) and E λ,2 (x2) (light intensity, i.e., the pixel value of the corresponding point) are all known, background light B λ 1 and B λ 2 As can be seen in step (1), there is only one unknown c in formula (9). λ Therefore, the total light attenuation coefficient c can be calculated using such a point. λ .

[0085] Total attenuation coefficient c λ It can be decomposed into absorption coefficient α λ With scattering coefficient b λ The sum: c λ =a λ +b λ Furthermore, during underwater imaging, the background light B λ Proportional to the scattering coefficient b λ Inversely proportional to the total attenuation coefficient c λ Then we have:

[0086]

[0087] Gould et al., through extensive experiments, discovered that the scattering coefficient in water generally exhibits an approximately linear relationship with wavelength, i.e.

[0088] b λ =(-0.00113λ+1.62517)b λ＝555 (11)

[0089] In the formula, b λ＝555 This represents the scattering coefficient corresponding to light with wavelength λ = 555.

[0090] Combining formulas (10) and (11), we obtain:

[0091]

[0092] In the formula: c r c g c b This represents the total attenuation coefficient of the corresponding R, G, and B channels; b r b g b b This represents the scattering coefficients of the corresponding R, G, and B channels; B r B g B b This represents the background light of the corresponding R, G, and B channels; λ r , λ g , λ b This indicates the wavelength of the corresponding R, G, and B colors of light.

[0093] In summary, theoretically, it is only necessary to determine the total attenuation coefficient of one of the three channels (R, G, and B) to obtain the background light B. λ The total attenuation coefficient c of the other two channels is obtained from the wavelength λ. λ .

[0094] The total attenuation coefficient c was obtained. λ Subsequently, this invention obtains the underwater image depth d(x) using a depth estimation method based on image saturation. Under artificial light, foreground pixels in underwater images are brighter than background pixels. When the foreground has bright pixels and the background has darker pixels, MIP-based depth estimation may fail because the depths D of the foreground and background are different. mip The values ​​are similar, so an accurate depth map cannot be generated at this time. When an underwater target is illuminated by an artificial light source, the foreground white light in the underwater scene captured by the underwater camera increases, and the saturation decreases; as the distance increases, the effect of the artificial light source weakens, and the image saturation increases. Therefore, image saturation can be used to estimate the depth of field of an underwater image. This invention uses image saturation to estimate the depth of field.

[0095] Image local saturation S(x):

[0096]

[0097] In the formula, R(x), G(x), and B(x) represent the values ​​at corresponding points x in the three channels R, G, and B, respectively.

[0098] The image saturation values ​​mentioned above only reflect the relative distance between the nearest and farthest points in an underwater scene and cannot be directly substituted into the transmittance expression to calculate transmittance. To convert the relative distances between points in an underwater image into absolute distances, it is necessary to first determine the actual distance of the nearest point in the image. In actual underwater robot operations, the distance d0 between the underwater robot and the target object can be measured using its own Doppler log (DVL).

[0099] Therefore, the actual depth of field d(x) is:

[0100] d(x)=D×S(x)+d0 (14)

[0101] In the formula: D is the conversion coefficient. Through extensive experimental verification, D = 9 in this invention patent.

[0102] The total attenuation coefficient c obtained through the above steps λ And depth of field d(x), based on transmittance The underwater image transmittance t can then be obtained. λ (x).

[0103] Step (3): Combine Step (1) and Step (1) to obtain the underwater image background light estimation result and transmittance estimation result, restore the underwater image according to the underwater image restoration formula, and output the final clear underwater image.

[0104] Combination Figure 1 According to the Jaffe-McGlamery underwater optical model, the underwater imaging model under artificial illumination is as follows:

[0105] I λ (x)=J λ (x)t λ (x)+B λ (1-t λ (x)) (15)

[0106] In the formula: I λ (x) is a blurred underwater image captured by an underwater camera; J λ (x) represents the ideal, clear image obtained after restoration; t λ (x) represents the transmittance of the underwater image.

[0107] Using the background light B obtained in steps (1) and (2) λ and transmittance t λ (x), for the input original image I λ (x), according to formula (15), the restored clear underwater image J is output. λ (x):

[0108]

[0109] (5) Application Cases

[0110] To verify the effectiveness of the underwater image restoration method based on multiple frames under artificial lighting proposed in this invention, the algorithm proposed in this invention was experimentally compared with the UDCP algorithm by Drews et al., the red channel prior algorithm by Galdran et al., the MIP algorithm by Bianco et al., the underwater image restoration algorithm based on image blur and light absorption by Peng et al., and the WCID algorithm by Chiang et al. The original degraded underwater images used in the experiments are attached. Figure 3 As shown in the attached figure, the underwater image restoration results are compared. Figure 4 As shown. Wherein:

[0111] Figure 4 (a) Using the algorithm proposed in this patent to... Figure 3 The result of the processing;

[0112] Figure 4 (b) Correspondingly, the UDCP algorithm proposed by Drews et al. is used for... Figure 3 The result of the processing;

[0113] Figure 4 (c) Correspondingly, the red channel prior algorithm proposed by Galdran et al. is used for... Figure 3 The result of the processing;

[0114] Figure 4 (d) Corresponding to the MIP algorithm proposed by Bianco et al. Figure 3 The result of the processing;

[0115] Figure 4 (e) Correspondingly, the algorithm proposed by Peng et al. is used to... Figure 3 The result of the processing;

[0116] Figure 4 (f) Corresponding to the WCID algorithm proposed by Chiang et al. Figure 3 The result of the processing.

[0117] Combined with appendix Figure 4The experimental results show that the UDCP algorithm by Drews et al. produces a noticeable halo on the target object in the processed image, with excessive brightness enhancement, and the algorithm's incorrect compensation for the red channel causes the image to be reddish; the red channel prior algorithm by Galdran et al. produces an image with a reddish overall background, and the blue sphere in the target object almost blends into the background after restoration; the MIP algorithm by Bianco et al. produces an image with severe color distortion; the underwater image restoration algorithm by Peng et al. based on image blur and light absorption incorrectly enhances the dark background of the original underwater image to blue, and makes the overall color of the processed image darker; the WCID algorithm by Chiang et al. produces an image with better target object color, but the background is grayish and target object details are lost. In contrast, the algorithm proposed in this invention effectively improves image clarity, makes target object details more obvious, and the restored image colors are more natural compared to other algorithms.

[0118] To more objectively evaluate the image quality of the algorithm's experimental results, this invention patent selects two commonly used underwater image quality evaluation metrics, UCIQE (Underwater Color Image Quality Evaluation) and UIQM (Underwater Image Quality Measurement), as the quality evaluation metrics for underwater image restoration. UCIQE reflects the linear quantitative evaluation result of the relationship between color cast, blur, and contrast after underwater image restoration; a higher value indicates a better image processing effect. UIQM mainly uses a linear combination of the color measurement metric (UICM), sharpness measurement metric (UISM), and contrast measurement metric (UIConM) as the evaluation basis; a higher value indicates better color balance, sharpness, and contrast in the processed image.

[0119] The quantitative analysis results of image restoration by different algorithms are shown in Table 1. By comparing and analyzing the evaluation indicators of each algorithm, it can be found that the algorithm proposed in this invention is superior to other underwater image restoration algorithms in both the objective evaluation indicators of UCIQE and UIQM. Therefore, for turbid water environments with artificial light sources, the algorithm proposed in this invention can achieve underwater image restoration better than other underwater image restoration algorithms.

[0120] Table 1 Comparison of objective evaluation metrics among different algorithms

[0121]

[0122] In summary, this invention provides a method for underwater image restoration based on multiple frames of images under artificial lighting. It belongs to the field of image processing technology. This method takes the operation of an unmanned and untethered underwater robot based on underwater visual images as the background, and targets the turbid water environment under artificial lighting. It combines the process of the underwater robot approaching the target object from a distance to a distance to collect multiple frames of images, and performs underwater image restoration based on the multiple frames of images. The main contents include: (1) Proposing a systematic method for estimating background light. First, the underwater lighting image segmentation method based on the YCbCr color model is used to segment the image to obtain the foreground and background. Then, the quadtree method is used to quickly lock the background light area in the obtained background area, and finally the background light is estimated in the area; (2) Proposing a method for solving the transmittance based on multiple frames of images. First, the attenuation law of light propagation in the medium is used to solve the total light intensity attenuation coefficient c based on the light intensity attenuation relationship between corresponding points in the multiple frames of images. λ Then, the depth of field d(x) of the underwater image is obtained using saturation-based depth estimation, and the total attenuation coefficient c is obtained through the above method. λ By combining the depth of field d(x), the transmittance of the corresponding channel in the underwater image can be obtained. Finally, the background light and transmittance from contents (1) and (2) are used to achieve image restoration through the underwater image restoration formula. This invention can more accurately restore the clarity of images in turbid water environments under artificial lighting, and the method has higher adaptability to turbid water bodies.

Claims

1. A method for underwater image restoration based on multiple frames under artificial lighting, characterized in that, The steps are as follows: Step (1): Input the original underwater image. For turbid water bodies under artificial lighting, use the image segmentation method based on the YCbCr color space and the quadtree method to estimate the background light B of the underwater image. λ λ is the wavelength, λ∈{R,G,B}; Step (2): Combining the process of the underwater robot approaching the target from a distance, acquire multiple frames of images. Utilize the light intensity attenuation relationship between corresponding points in the multiple frames to first solve for the total attenuation coefficient c. λ The image depth d(x) is obtained by using a depth estimation method based on image saturation; the total attenuation coefficient c of each channel is calculated accordingly. λ Given the depth of field d(x) of the underwater image, calculate the transmittance t of the corresponding channel. λ (x); Step (3): Combining Step (1) and Step (2), the underwater image background light estimation result and transmittance estimation result are given. The underwater image is restored according to the underwater image restoration formula, and the final clear underwater image is output.

2. The underwater image restoration method based on multiple frames under artificial lighting as described in claim 1, characterized in that, The specific steps (1) are as follows: First, based on the underwater illumination image segmentation method of YCbCr color space, by avoiding the high similarity between the target object and the background in grayscale when the image is grayscaled, the image is converted to YCbCr color space. Then, the information in the Cb and Cr component channels is fused and threshold segmentation is used to obtain the foreground and background. Then, in the segmented background, the quadtree block method is used to quickly lock the area where the background light of the image is located. Finally, in the finally locked image block area, the pixel values ​​are sorted from large to small, the interference of the first 10% of the pixel values ​​is removed, and the average value of the remaining pixel values ​​is taken to obtain the final background light B. λ .

3. The underwater image restoration method based on multiple frames under artificial lighting as described in claim 1, characterized in that, Step (2) specifically involves the following: When light propagates in a water medium, its intensity decreases exponentially with increasing distance d, generally expressed as transmittance t. λ (x) reflects the degree of attenuation, and the transmittance expression is: In the formula: c λ Let d(x) be the total attenuation coefficient, and d(x) be the distance between point x on the target and the underwater camera. First, by observing the underwater robot's approach to the target from a distance, multiple frames of images are acquired. Then, the total attenuation coefficient c is calculated using the light intensity attenuation relationship between corresponding points in these multiple frames. λ According to the underwater imaging model, the optical loop attenuation process at point x on the target is as follows: In the formula: E λ,0 (x) represents the light intensity at point x on the target object; E λ,d (x) represents the light intensity received by the camera at a distance d from the target object; x represents the coordinates of a point in the scene, x = (x', y'); c λ This is the total attenuation coefficient; d is the distance between point x on the target and the underwater camera; The above equation can be transformed into: According to the laws of light propagation in water, images at different distances *d* can all correspond to the attenuation formula mentioned above. Considering the process of an underwater robot approaching a target from a distance, and acquiring two frames of images, assuming the depths of field for the two frames are *d1* and *d2* respectively, then: In the formula: It is expressed as the light intensity received by the underwater camera at a distance d1 from the target object; B represents the light intensity received by the underwater camera at a distance d2 from the target object; λ 1 B λ 2 These are the background lights corresponding to the images with depths of field d1 and d2, respectively; E λ,1 (x1) and E λ,2 (x2) represents the light intensity at point x on the target object in the images with depths d1 and d2, respectively; Take the logarithm of both sides of the above expression: The difference between the two equations is: In the above formula: E λ,1 (x1) and E λ,2 (x2) is the light intensity reflected from the same point x on the target object in the images at distances d1 and d2. Assume the light intensity emitted by the artificial light source is E. λ,0 ,have: In the formula: ρ λ Indicates the reflectivity of the target object's surface; Therefore, it can be simplified to: The above formula can be obtained by analyzing the light intensity attenuation pattern between two frames. Therefore, we select a corresponding point on the sphere in the two frame sequence images, with its depth of field d1 and d2 and light intensity E. λ,1 (x1) and E λ,2 (x2) is known; the total light attenuation coefficient c can be calculated using this point. λ ; The depth of field d(x) of the underwater image is obtained by a depth estimation method based on image saturation. Saturation generally refers to the purity of the image color. When the underwater target is illuminated by an artificial light source, the foreground white light in the underwater scene captured by the underwater camera increases and the saturation decreases. As the distance increases, the effect of artificial light sources weakens, and the image saturation increases; the image saturation is used to estimate the depth of field d(x) of the underwater image; The total attenuation factor c is obtained through the above steps. λ And depth of field d(x), using transmittance Thus, the transmittance t of the underwater image is obtained. λ (x).

4. The underwater image restoration method based on multiple frames under artificial lighting as described in claim 1, characterized in that, Step (3) specifically refers to: Based on the Jaffe-McGlamery underwater optical model, the underwater imaging model under artificial illumination is as follows: I λ (x)=J λ (x)t λ (x)+B λ (1-t λ (x)) In the formula: I λ (x) represents the original image captured by the underwater camera; J λ (x) represents an ideal, clear underwater image; Using the background light B obtained in steps (1) and (2) λ and transmittance t λ (x), for the input original image I λ (x) will output the restored clear underwater image J λ (x):

Images