An underwater image enhancement method based on eight-neighborhood constraint polarization imaging

Abstract

This invention provides an underwater image enhancement method based on eight-neighborhood constraint polarization imaging, comprising: acquiring four polarization images, denoted as I0(x,y), I... 45 (x,y), I 90 (x,y), I 135 (x,y); Calculate the polarization angle and degree of polarization for each pixel in the image based on the Stokes vector; Select the polarization angle value with the highest frequency, obtain the corresponding pixel position, and perform eight-neighbor constraint; After passing the eight-neighbor constraint, average the selected polarization degree values ​​and use it as the background light polarization degree value p. B The method calculates the background light intensity and the background light intensity of the polarized portion; it subtracts from the Stokes vector M to calculate the background light intensity at infinity; based on the calculated background light intensity at infinity, it calculates the transmittance; and it uses an anchor brightness adaptive algorithm to enhance the image, resulting in an enhanced image. This invention provides stable enhancement effects, is applicable to water media with varying turbidity, and effectively enhances image contrast and visibility.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to a polarization imaging underwater image enhancement method based on eight-neighborhood constraints. Background Technology

[0002] In underwater media with high turbidity, cameras often struggle to obtain sufficiently clear underwater images. This is because suspended particles in the water scatter and reflect light during its propagation, and the resulting backscattering severely affects camera imaging, causing a decrease in image contrast.

[0003] Currently, underwater image enhancement methods can be divided into three categories: The first category directly modifies the image at the pixel level. Its advantage is fast processing speed, but because it doesn't consider underwater imaging models during enhancement, its applicability is limited, and it often cannot be adjusted for water media with varying turbidity, leading to varying degrees of insufficient or excessive enhancement. The second category is based on physical methods, which consider underwater imaging physics models during image restoration. It establishes a signal attenuation model based on the characteristics of light propagation in water and restores the image using parameters calculated from this model. Its advantages include high specificity for different underwater scenes, effectively solving problems such as low underwater imaging contrast and poor visibility, and strong real-time enhancement feasibility. The third category is deep learning-based underwater image enhancement, which trains on existing underwater image datasets to extract different underwater features and enhances underwater images in a targeted manner. Its advantage is good restoration results, but because this method has high equipment requirements and relies on existing datasets for training, and current underwater image datasets do not comprehensively cover all scenarios, this method cannot effectively enhance underwater images in certain situations. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an underwater image enhancement method based on eight-neighborhood constraints for polarization imaging. This invention captures four polarization images at different angles using a rotating polarizing lens and processes the images using Stokes vectors. During this process, this invention proposes an eight-neighborhood constraint to reduce the interference of quantum noise generated by the polarization camera on polarization degree estimation, significantly improving the accuracy of parameter estimation and the final enhancement effect. In the post-processing stage, this invention proposes a brightness-adaptive post-processing method to solve the problem of overexposure in background light areas caused by low transmittance values. This invention provides stable enhancement effects, is applicable to water media with varying turbidity, and effectively enhances image contrast and visibility.

[0005] The technical means employed in this invention are as follows:

[0006] An underwater image enhancement method based on eight-neighborhood constraints for polarization imaging includes:

[0007] Acquire four polarization images, denoted as I0(x,y), I... 45 (x,y), I 90 (x,y), I 135 (x,y);

[0008] Calculate the polarization angle and degree of polarization for each pixel in the image based on the Stokes vector;

[0009] Select the polarization angle value that appears most frequently, obtain the corresponding pixel position, and perform eight-neighbor constraint;

[0010] After applying the eight-neighbor constraint, the average of the filtered polarization degree values ​​is taken as the background light polarization degree value p. B ;

[0011] Calculate the background light intensity and the background light intensity of the polarized portion;

[0012] Calculate the background light intensity at infinity by subtracting it from the Stokes vector M;

[0013] Transparency is calculated based on the calculated background light intensity at infinity.

[0014] An anchor brightness adaptive algorithm is used to enhance the image, resulting in an enhanced image.

[0015] Furthermore, the polarizing mirror angles of the four polarized images are 0°, 45°, 90° and 135°, respectively, with the horizontal direction as the 0° direction, and the polarizing mirror directions of the other images rotated relative to the 0° direction.

[0016] Further, the step of calculating the polarization angle and degree of polarization corresponding to each pixel of the image based on the Stokes vector includes:

[0017] The corresponding Stokes vector is calculated using the following formula:

[0018] M(x,y)=I0(x,y)+I 90 (x,y)

[0019] Q(x,y)=I0(x,y)-I 90 (x,y)

[0020] U(x,y)=I 45 (x,y)-I 135 (x,y)

[0021] Based on the Stokes vector, the polarization angle and degree of polarization corresponding to each pixel of the image are calculated using the following formula:

[0022]

[0023]

[0024] Where θ(x,y) is the polarization angle value and p(x,y) is the degree of polarization value.

[0025] Furthermore, the specific rules for the eight-neighborhood constraint are as follows:

[0026] Select the region with the highest frequency of polarization angle values, sort the polarization degree values ​​of the region in descending order, take out the first one-thousandth and record the position of the first one-thousandth of pixels;

[0027] The eight-neighbor area is expanded from the center position of the previous one-thousandth of a pixel position;

[0028] Set two factors ω max ,ω min Determine whether the polarization degree of the pixels in this region satisfies the following formula:

[0029]

[0030] The number of pixels that do not meet the requirements in the eight-neighborhood is counted. If more than four pixels in the eight-neighborhood do not meet the requirements, the central pixel is considered to be interference from camera noise and is removed from the top 0.1%.

[0031] Further, the calculation of the background light intensity and the background light intensity of the polarized portion includes:

[0032] The formula for calculating background light intensity is as follows:

[0033]

[0034] Among them, B p (x,y) represents the background light intensity of the polarized portion, p B This indicates the polarization degree of the background light.

[0035] The formula for calculating the background light intensity of the polarized portion is as follows:

[0036] B px (m,n)=B p (m,n)cos 2 [θ B (m,n)]

[0037]

[0038] Among them, B px (x,y) represents a component of the background light polarization on the x-axis.

[0039] Furthermore, the calculation of the background light intensity at infinity by subtracting from the Stokes vector M includes:

[0040] In the Stokes vector M(x,y), the intensity value of the background light at infinity tends to the intensity of the corresponding position in the background light map B(x,y). Based on this principle, we can directly set a matrix A(x,y) of the same dimension to be equal to the absolute value of the difference between the background light map B(x,y) and the Stokes vector B(x,y).

[0041] Let ε equal 0.005, find the pixels in A(x,y) that are less than ε, and take the average of the pixel values ​​in A(x,y) that are less than ε, as the background light intensity value B at infinity. ∞ .

[0042] Furthermore, the calculation of transmittance based on the calculated background light intensity at infinity includes:

[0043] The perspective plot t(x,y) is calculated using the following formula:

[0044]

[0045] The formula for calculating the restored image is as follows:

[0046] I(x,y)=J(x,y)·t(x,y)+(1-t(x,y))·B ∞

[0047]

[0048] Where x, y represent the coordinates of the underwater image, I(x, y) is the underwater image acquired by the camera, J(x, y) represents the light directly reflected from the target object, t(x, y) is the transmittance map, and B... ∞ It is the background light intensity value at infinity.

[0049] Furthermore, the process of enhancing the image using an anchor brightness adaptive algorithm to obtain an enhanced image includes:

[0050] Assume that the Stokes vector M(x,y) is under normal exposure and has reached the optimal exposure before image restoration;

[0051] Based on the above assumptions, the average intensity of the Stokes vector M(x,y) is taken as the brightness anchor point α. anchor Similarly, the mean intensity of the reconstructed image J(x,y) is taken as the floating value α. float ;

[0052] α anchor With α float The ratio of α to β is used as the brightness scaling factor α, that is:

[0053]

[0054] Multiplying the restored image J(x,y) by the brightness scaling factor α will make J(x,y) achieve the optimal exposure brightness.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] 1. The underwater image enhancement method based on eight-neighborhood constraint polarization imaging provided by this invention is based on the physical model of underwater imaging. It considers the different propagation processes of light in turbid water and air, and according to the characteristics of polarization imaging, it uses different information obtained by retaining and filtering polarized light for image enhancement. This polarization information can better restore the details of the target object and improve the contrast.

[0057] 2. The underwater image enhancement method based on eight-neighborhood constraint provided by this invention proposes an eight-neighborhood constraint to address the single-value dependency problem in the process of obtaining the polarization degree of the background light, which greatly reduces the interference of random quantum noise of the camera on parameter acquisition and improves the accuracy of parameter estimation.

[0058] 3. The underwater image enhancement method based on polarization imaging with eight-neighborhood constraints provided by this invention proposes an adaptive brightness post-processing method for anchoring the original image to solve the problem of inaccurate exposure of the restored image. The proposed method makes the entire algorithm process free from human intervention.

[0059] Based on the above reasons, this invention can be widely applied in fields such as image processing. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of underwater optical imaging according to the present invention.

[0062] Figure 2 This is a flowchart of the method of the present invention.

[0063] Figure 3 The diagram illustrates the results provided in the embodiments of the present invention. Detailed Implementation

[0064] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0067] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0068] In the description of this invention, it should be understood that the orientation or positional relationship indicated by directional terms such as "front, back, up, down, left, right", "horizontal, vertical, horizontal" and "top, bottom" is generally based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. Unless otherwise stated, these directional terms do not indicate or imply that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the scope of protection of this invention. The directional terms "inner" and "outer" refer to the inner and outer contours relative to the outline of each component itself.

[0069] For ease of description, spatial relative terms such as "above," "over," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation besides the orientation of the device as described in the figures. For example, if the device in the figures is inverted, a device described as "above" or "above" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0070] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, the above terms have no special meaning and therefore should not be construed as limiting the scope of protection of this invention.

[0071] In underwater imaging models, the propagation of light is as follows: Figure 1 As shown, the image obtained by the camera underwater mainly consists of two parts: direct illumination of the object and backscattering of background light, as shown in the following formula:

[0072] I(x,y)=J(x,y)·t(x,y)+(1-t(x,y))·B ∞

[0073] Where x, y represent the coordinates of the underwater image, I(x, y) is the underwater image acquired by the camera, J(x, y) represents the light directly reflected from the target object, t(x, y) is the transmittance map, and B... ∞ This represents the background light intensity value at infinity. The image restored by this invention is J(x,y), and the formula for restoring the image can be derived from the above formula as follows:

[0074]

[0075] The unknowns in the above formula are key to solving the enhanced image J(x,y), and the method of this invention will solve them using polarization imaging. The detailed scheme is as follows:

[0076] like Figure 2 As shown, this invention provides a polarization imaging underwater image enhancement method based on eight-neighborhood constraints, comprising:

[0077] S1. Obtain four polarization images, denoted as I0(x,y), I... 45(x,y), I 90 (x,y), I 135 (x,y);

[0078] S2. Calculate the polarization angle and degree of polarization for each pixel in the image based on the Stokes vector.

[0079] S3. Select the polarization angle value that appears most frequently, obtain the corresponding pixel position, and perform eight-neighbor constraint.

[0080] S4. After applying the eight-neighborhood constraint, the average of the filtered polarization degree values ​​is taken as the background light polarization degree value p. B ;

[0081] S5. Calculate the background light intensity and the background light intensity of the polarized portion;

[0082] S6. Subtract the Stokes vector M to calculate the background light intensity at infinity;

[0083] S7. Calculate the transmittance based on the calculated background light intensity at infinity;

[0084] S8. The image is enhanced using the anchor brightness adaptive algorithm to obtain the enhanced image.

[0085] In a specific implementation, as a preferred embodiment of the present invention, in step S1, the polarizing mirror angles of the four polarized images are 0°, 45°, 90° and 135°, respectively, wherein the horizontal direction is taken as the 0° direction, and the polarizing mirror directions of the other images are rotated relative to the 0° direction.

[0086] In a specific implementation, as a preferred embodiment of the present invention, step S2, calculating the polarization angle and degree of polarization corresponding to each pixel of the image based on the Stokes vector, includes:

[0087] The corresponding Stokes vector is calculated using the following formula:

[0088] M(x,y)=I0(x,y)+I 90 (x,y)

[0089] Q(x,y)=I0(x,y)-I 90 (x,y)

[0090] U(x,y)=I 45 (x,y)-I 135 (x,y)

[0091] Based on the Stokes vector, the polarization angle and degree of polarization corresponding to each pixel of the image are calculated using the following formula:

[0092]

[0093]

[0094] Where θ(x,y) is the polarization angle value and p(x,y) is the degree of polarization value.

[0095] In practical implementation, as a preferred embodiment of the present invention, ideally, the maximum polarization degree value of the pixel region corresponding to the polarization angle value with the highest frequency of occurrence is the background light polarization degree. However, ideally, the background light polarization degree ultimately depends only on the value of a single pixel, that is, the maximum value among them. This approach is highly susceptible to quantum noise interference from the camera in actual operation, leading to inaccurate parameter estimation. Therefore, the present invention proposes an eight-neighborhood constraint to improve estimation accuracy. That is, in step S3, the specific rules of the eight-neighborhood constraint are as follows:

[0096] Select the region with the highest frequency of polarization angle values, sort the polarization degree values ​​of the region in descending order, take out the first one-thousandth and record the position of the first one-thousandth of pixels;

[0097] The eight-neighbor area is expanded from the center position of the previous one-thousandth of a pixel position;

[0098] Set two factors ω max ,ω min Determine whether the polarization degree of the pixels in this region satisfies the following formula:

[0099]

[0100] The number of pixels in the eight-neighborhood that do not meet the requirements is counted. If four or more pixels in the eight-neighborhood do not meet the requirements, the central pixel is considered to be subject to camera noise interference and is removed from the top 0.1%. In darker scenes, camera noise is more severe, ω max We can take 1.05, ω min 0.95 can be taken.

[0101] In a specific implementation, as a preferred embodiment of the present invention, step S4, calculating the background light intensity and the background light intensity of the polarized portion, includes:

[0102] The formula for calculating background light intensity is as follows:

[0103]

[0104] Among them, B p (x,y) represents the background light intensity of the polarized portion, p B This indicates the polarization degree of the background light.

[0105] The formula for calculating the background light intensity of the polarized portion is as follows:

[0106] Bpx (m,n)=B p (m,n)cos 2 [θ B (m,n)]

[0107]

[0108] Among them, B px (x,y) represents a component of the background light polarization on the x-axis.

[0109] In a specific implementation, as a preferred embodiment of the present invention, step S5, subtracting the Stokes vector M to calculate the background light intensity at infinity, includes:

[0110] In the Stokes vector M(x,y), the intensity value of the background light at infinity tends to the intensity of the corresponding position in the background light map B(x,y). Based on this principle, we can directly set a matrix A(x,y) of the same dimension to be equal to the absolute value of the difference between the background light map B(x,y) and the Stokes vector B(x,y).

[0111] Let ε equal 0.005, find the pixels in A(x,y) that are less than ε, and take the average of the pixel values ​​in A(x,y) that are less than ε, as the background light intensity value B at infinity. ∞ .

[0112] In a specific implementation, as a preferred embodiment of the present invention, step S6, calculating the transmittance based on the calculated background light intensity at infinity, includes:

[0113] The perspective plot t(x,y) is calculated using the following formula:

[0114]

[0115] The formula for calculating the restored image is as follows:

[0116] I(x,y)=J(x,y)·t(x,y)+(1-t(x,y))·B ∞

[0117]

[0118] Where x, y represent the coordinates of the underwater image, I(x, y) is the underwater image acquired by the camera, J(x, y) represents the light directly reflected from the target object, t(x, y) is the transmittance map, and B... ∞ It is the background light intensity value at infinity.

[0119] In a specific implementation, as a preferred embodiment of the present invention, t(x,y) is used as the denominator of the restoration formula. The value of t(x,y) varies point by point, and may be a small value at the background light position, which can lead to an overly bright background light portion of the restored image. To solve this problem, the present invention proposes a brightness adaptive post-processing method based on a priori principle: that is, in step S7, an anchor brightness adaptive algorithm is used to enhance the image to obtain an enhanced image, including:

[0120] Assume that the Stokes vector M(x,y) is under normal exposure and has reached the optimal exposure before image restoration;

[0121] Based on the above assumptions, the average intensity of the Stokes vector M(x,y) is taken as the brightness anchor point α. anchor Similarly, the mean intensity of the reconstructed image J(x,y) is taken as the floating value α. float ;

[0122] α anchor With α float The ratio of α to β is used as the brightness scaling factor α, that is:

[0123]

[0124] Multiplying the restored image J(x,y) by the brightness scaling factor α will make J(x,y) achieve the optimal exposure brightness.

[0125] like Figure 3 As shown, the results provided by this invention are illustrated in (a)-(d), which are four polarization images (0°, 45°, 90°, 135°), and are also the inputs of this invention; (e) is the superposition of the 0° image and the 90° image, which can be regarded as the effect of taking pictures with an ordinary camera; (f) is the effect image after processing by this invention; (g)-(h) are magnified images of (e)-(f), showing the before and after effects; (i) shows more before and after comparisons of the results of this invention.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A polarization imaging underwater image enhancement method based on eight-neighborhood constraint, characterized in that, Comprise: Four polarized images are acquired, respectively denoted as , , , ; According to the stokes vector calculation image each pixel corresponding to the polarization angle and degree of polarization; Select the highest frequency of polarization angle value, get the corresponding pixel position, eight neighborhood constraint; After the eight-neighbor constraint, the polarized degree values that pass the screening are averaged to be the background light polarized degree value ; Calculate the background light intensity and the background light intensity of the polarization part, including: Calculate the background light intensity, the calculation formula is as follows: wherein represents the background light intensity of the polarization part, represents the background light polarization degree value; with the stokes vector M Subtract, calculate the infinite remote background light intensity; Based on the calculated infinite background light intensity, calculate the perspective rate, calculate the restored image, the calculation formula is as follows: wherein, , represent coordinates of the underwater image, is an underwater image acquired by the camera, represent a restored image, is a perspective ratio map, is a background light intensity value at infinity; Enhance the image by using anchor brightness adaptive algorithm to obtain enhanced image, including: Assume the Stokes vector Normal exposure, and the best exposure before the image has been reached restoration; Based on the above assumptions, the intensity of the Stokes vector is averaged as the luminance anchor , and similarly, the intensity average of the restored image is taken as the floating value ; The ratio of to is used as a luminance scaling factor i.e. Let the reduced image Multiplying the luminance scaling factor That is, the exposure luminance Is optimized.

2. The polarization imaging underwater image enhancement method based on eight-neighborhood constraint according to claim 1, characterized in that, The four polarization images are 0°, 45°, 90° and 135°, wherein the horizontal direction is 0° direction, and the remaining picture polariscope direction is rotated relative to 0° direction.

3. The polarization imaging underwater image enhancement method based on octant constraint according to claim 1, characterized in that, The stokes vector calculation image each pixel corresponding to the polarization angle and degree of polarization, including: Calculate the corresponding stokes vector, the calculation formula is as follows: According to the stokes vector, calculate the polarization angle and degree of polarization corresponding to each pixel point of the image, the calculation formula is as follows: wherein is a value of the polarization angle, is a value of the degree of polarization.

4. The polarization imaging underwater image enhancement method based on octant constraint according to claim 1, characterized in that, The specific rules of the eight neighborhood constraint are as follows: Select the highest frequency of polarization angle value area, and arrange the polarization degree value of the area in descending order, take out the first one thousandth and record the pixel position of the first one thousandth; Select the highest frequency of polarization angle value area, and arrange the polarization degree value of the area in descending order, take out the first one thousandth and record the pixel position of the first one thousandth; Setting two pass factors , , determining whether the polarization degree of the region pixels satisfies the following formula: Take the first one thousandth pixel position as the center position to expand the eight neighborhood; 5. The polarization imaging underwater image enhancement method based on octant constraint according to claim 1, characterized in that, And the number of pixels that do not meet the eight neighborhood is counted, when there are more than four pixels in the eight neighborhood that do not meet, it is considered that the center pixel is the interference of camera noise, and it is excluded from the first one thousandth. The calculation of the background light intensity and the background light intensity of the polarization part includes: wherein represents a component of the background light polarization part on the x axis.

6. The polarization imaging underwater image enhancement method based on octant constraint according to claim 1, characterized in that, said with the stokes vector M Subtracting, calculating the infinite far background light intensity, including: In the Stokes vector the intensity value of the background light at infinity tends to the background light map The intensity of the corresponding position, based on this principle, directly makes a same dimension matrix equal to the background light map The absolute value of the difference with the Stokes vector ​ Let equal 0.005, find pixels in less than pixels in less than average the pixel values and use as the background light intensity value at infinity 7. The polarization imaging underwater image enhancement method based on octant constraint according to claim 1, characterized in that, Calculate the background light intensity of the polarization part, the calculation formula is as follows: The calculation of the background light intensity and the background light intensity of the polarization part includes: Calculate the background light intensity of the polarization part, the calculation formula is as follows: Computing a perspective ratio map The formula is as follows: wherein, , representing the coordinates of the underwater image, is a perspective ratio map, is a background light intensity value at infinity.

Images