A method for optimizing depth information of a scene based on boundary constraints of a transfer function

Abstract

The present invention discloses a method for optimizing depth information of a scene based on boundary constraints of a transmission function, which relates to the technical field of image processing. The method includes constructing an initial transmission function based on an atmospheric scattering model; constructing a polarization boundary constraint transmission function according to the initial transmission function and the boundary constraints of the atmospheric polarization degree; constructing a boundary constraint transmission function according to the initial transmission function, the polarization boundary constraint transmission function, a preset lower boundary threshold of the transmission function, and a maximum value function; constructing a weight constraint; constructing an objective function according to the difference between the boundary constraint transmission function value and the optimized transmission function value, the sum of all weight constraints in the image of the information, and a regularization parameter for balancing these two items, and solving for the optimal optimized transmission function value with the goal of minimizing the objective function; and obtaining the depth information of the optimized scene according to the optimal optimized transmission function value. The present invention improves the environmental perception ability of the scene and improves the adaptability and expandability of complex scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for optimizing depth information of a scene based on boundary constraints of a transmission function. Background Art

[0002] With the rapid development of technology, autonomous driving, robot navigation, and motion estimation technologies have been widely applied in various fields. However, in practical applications, especially in low-texture scenes such as haze, these technologies face huge challenges. Haze weather causes the image information captured by visual sensors to be blurred and have low contrast, seriously affecting the perception ability of unmanned platforms for scenes, and thus increasing the difficulty of motion estimation.

[0003] Traditional image dehazing methods mainly rely on image processing algorithms to enhance images. However, when dealing with haze images, these methods often have difficulty accurately restoring the true information of the images, resulting in limited dehazing effects. In recent years, with the rise of polarization imaging technology, it provides a new idea for the processing of haze images. Polarization imaging technology can obtain more image information by measuring the polarization state of light waves. Especially in scattering media such as haze, polarization information has a higher signal-to-noise ratio and stronger anti-interference ability.

[0004] However, relying solely on polarization imaging technology cannot completely solve the problem of dehazing haze images. In practical applications, due to the scattering effect of haze particles, accurate estimation of polarization information also faces certain difficulties. In addition, targets of different materials exhibit different polarization characteristics in polarization imaging, which also brings additional challenges to image processing.

[0005] Therefore, how to accurately estimate the transmission function using polarization imaging technology and effectively remove the influence of haze on images in low-texture scenes such as haze has become an urgent problem to be solved. Summary of the Invention

[0006] The present invention proposes a method for optimizing depth information of a scene based on boundary constraints of a transmission function to solve the problem that in low-texture scenes such as haze in the prior art, it is difficult to clearly perceive the effective information of the scene and the difficulty of motion estimation is large.

[0007] The present invention achieves the above object through the following technical solutions:

[0008] A method for optimizing depth information of a scene based on boundary constraints of a transmission function according to the present invention includes:

[0009] Obtaining the information of the scene detected by a polarization detector, where the information includes the polarization image of the scene and the atmospheric light intensity at infinity;

[0010] Construct an initial transmission function based on the degree of polarization of the atmosphere, the atmospheric light intensity at infinity, and the polarization image, based on the atmospheric scattering model;

[0011] Construct a polarization boundary constraint transmission function according to the initial transmission function and the boundary constraints of the degree of polarization of the atmosphere. The boundary constraints of the degree of polarization of the atmosphere include the color channels of the lower boundary of the degree of polarization of the atmosphere and the color channels of the upper boundary of the degree of polarization of the atmosphere;

[0012] Construct a boundary constraint transmission function according to the preset lower boundary threshold of the transmission function, the maximum value function, the initial transmission function, and the polarization boundary constraint transmission function;

[0013] Construct a weight constraint, which is obtained by multiplying the difference between the weighted function between two adjacent pixel points in the polarization image and the value of the boundary constraint transmission function of the two adjacent pixel points. The weighted function is constructed according to the difference in the degree of polarization of two adjacent pixel points;

[0014] Construct an objective function according to the difference between the value of the boundary constraint transmission function and the value of the optimized transmission function, the sum of all the weight constraints, and the regularization parameter that balances the two, and solve for the optimal value of the optimized transmission function with the goal of minimizing the objective function;

[0015] Obtain the depth information of the optimized scene according to the optimal value of the optimized transmission function.

[0016] Further, the initial transmission function is:

[0017]

[0018] where d A represents the degree of polarization of the atmosphere, represents the vertical polarization image of the polarization image, represents the parallel polarization image in the polarization image, represents obtaining the atmospheric light intensity at infinity, where x is the pixel point of the polarization image.

[0019] Further, the boundary constraints of the degree of polarization of the atmosphere are:

[0020]

[0021] x represents the pixel point of the polarization image, y represents the color channel y of the polarization image, represents the three color channels of the polarization image, K0 and K1 represent the lower boundary and the upper boundary of the degree of polarization of the atmosphere, and respectively represent the color channels y of the lower boundary K0 and the upper boundary K1, and Ω represents the local area of the polarization image, represents the degree of atmospheric polarization, represents any pixel point in the local region Ω belonging to the polarization image.

[0022] Furthermore, the polarization boundary constraint transfer function is :

[0023]

[0024] where c represents the color channel c of the polarization image, and respectively represent the color channel c of the lower boundary K0 and the upper boundary K1.

[0025] Furthermore, the boundary constraint transfer function is:

[0026]

[0027] represents the preset lower boundary threshold of the transfer function, that is, the average value of the lowest δ% transfer function values, calculated from the lowest values of δ%, where δ% is a preset proportional value.

[0028] Furthermore, the weight constraint is:

[0029]

[0030]

[0031] represents a weighting function, and represent two adjacent pixel points in the polarization image, and respectively represent and the corresponding boundary constraint transfer function values. When = 0, the weight constraint is invalid, and respectively represent two adjacent pixel points in the polarization image and degrees of polarization, and σ is a standard deviation parameter.

[0032] Furthermore, the objective function is:

[0033]

[0034] where m is a regularization parameter for balancing and of, and respectively represent the boundary constraint transfer function and the optimized transfer function value, where i and j represent any two adjacent pixel points in the polarization image, and respectively represent the boundary constraint transfer function values corresponding to i and j, represents the weighting function of i and j, represents the local region centered on j, and I represents the index set of the pixels of the polarization image, represents the Manhattan norm of.

[0035] The beneficial effects of the present invention are as follows:

[0036] 1) The technical solution of the present invention constructs a polarization transfer function using the degree of polarization and reconstructs the atmospheric scattering model. It improves the environmental perception ability of the scene.

[0037] 2) A boundary constraint polarization transfer function is proposed, a constraint combining the weighted L1 norm is established, and an optimization problem is modeled to estimate a more accurate transfer function value. It improves the adaptability and scalability of complex scenes, especially performs well in haze environments, can improve the accuracy of motion estimation of unmanned platforms, and reduce hardware costs.

[0038] 3) The algorithm flow is simple, the calculation amount is small, and it is easy to be implemented by programming. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flowchart of a method for optimizing the depth information of a scene based on the boundary constraint of the transfer function in this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. The components of the embodiments of the present invention usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0042] It should be noted that: similar reference numerals and letters denote similar items in the following drawings, and thus, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0043] The specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0044] As Figure 1 shown, a method for optimizing depth information of a scene based on boundary constraints of a transmission function includes:

[0045] S1: Obtain information of the scene detected by a polarization detector, where the information includes a polarization image of the scene and the atmospheric light intensity at infinity;

[0046] S2: Based on the degree of atmospheric polarization, the atmospheric light intensity at infinity, and the polarization image, construct an initial transmission function based on an atmospheric scattering model;

[0047] S3: According to the initial transmission function and the boundary constraints of the degree of atmospheric polarization, construct a polarization boundary constraint transmission function, where the boundary constraints of the degree of atmospheric polarization include the color channels of the lower boundary of the degree of atmospheric polarization and the color channels of the upper boundary of the degree of atmospheric polarization;

[0048] S4: According to a preset lower boundary threshold of the transmission function, a maximum value function, the initial transmission function, and the polarization boundary constraint transmission function, construct a boundary constraint transmission function;

[0049] S5: Construct a weight constraint, where the weight constraint is obtained by multiplying the difference between the weighted function between two adjacent pixel points in the polarization image and the value of the boundary constraint transmission function of the two adjacent pixel points, and the weighted function is constructed according to the difference in the degree of polarization of the two adjacent pixel points;

[0050] S6: Construct an objective function based on the difference between the value of the boundary constraint transmission function and the value of the optimized transmission function, the sum of all the weight constraints, and a regularization parameter for balancing these two items, and solve for the optimal value of the optimized transmission function with the goal of minimizing the objective function;

[0051] S7: Obtain the depth information of the optimized scene according to the optimal value of the optimized transmission function.

[0052] In some embodiments, under hazy weather conditions, the light intensity reaching the detector mainly includes two parts: 1) the reflected light of the scene target, also known as direct transmitted light, which contains the intensity information of the scene target; 2) the stray light caused by scattering of haze particles, also known as atmospheric light, which is the main interference factor in optical imaging in a hazy environment. First, when the reflected light intensity L(x) of the scene target passes through the hazy area, it will be strongly scattered and absorbed by haze particles, and the direct transmitted light intensity D(x) reaching the detector decays exponentially with the transmission distance:

[0053] (1)

[0054] where β is the attenuation coefficient and z(x) is the distance from the detection target to the detector. Generally, the transmission function is defined as:

[0055] (2)

[0056] Different from the direct transmitted light, the atmospheric light is caused by the direct scattering of sunlight by haze particles, and the light intensity A(x) when it reaches the detector increases exponentially with the transmission distance:

[0057] (3)

[0058] where A ∞ is the atmospheric light intensity at infinity, representing the atmosphere without a target. The total light intensity I(x) when it reaches the detector is the incoherent superposition of the direct transmitted light intensity and the atmospheric light intensity, and can be expressed as:

[0059] (4)

[0060] The transmission function t r (x) in the atmospheric scattering model can be obtained as:

[0061] (5)

[0062] The degree of polarization of the atmosphere defined in the atmospheric scattering model is:

[0063] (6)

[0064] Combining equations (5) and (6) to eliminate the atmospheric light A, the expression of the initial transmission function A represented by the degree of polarization of the atmosphere d can be obtained:

[0065] (7)

[0066] After obtaining the estimated values of the polarization transmission function and the atmospheric light at infinity using the polarization information, the defogged image can be expressed as:

[0067] (8)

[0068] In some embodiments, when calculating the degree of polarization of the atmosphere d A , it is observed in the experiment that d A may be negative or the calculated value is greater than 1. Generally, d A is between 0 and 1, so it is necessary to eliminate the outliers. Considering that the degree of polarization of the atmosphere in the scene is always bounded, without loss of generality, considering the case of the y color channel, where , K0 and K1 are expressed as the lower and upper boundaries of the degree of polarization of the atmosphere, and and represent the color channels of the lower boundary K0 and the upper boundary K1 respectively. The purpose of the boundary constraint is to eliminate the atmospheric polarization degree d of the scene A outliers caused by errors, and obtain the exact transmission function values corresponding to the boundaries. The boundary constraint of the atmospheric polarization degree is as follows:

[0069] (9)

[0070] In the formula, Ω is the local area of a given image, and the boundary values are set to and , represents the atmospheric polarization degree, represents any pixel point in the local area Ω belonging to the polarization image.

[0071] The boundary constraint for the atmospheric polarization degree can be further expressed as the boundary constraint for the transmission function. Since the transmission function describes the influence of haze on image pixels, the boundary constraint for the transmission function aims to obtain the transmission function value when the dehazed pixel exactly reaches the given lower boundary or the upper boundary . Further, for the obtained transmission function value, the maximum value among its three color channels represents the transmission function value that makes the dehazed pixel exactly reach the lower boundary K0 or the upper boundary K1 in one color channel. Assume that the polarization difference image and as well as the atmospheric light intensity A at infinity ∞ are given. Incorporating the polarization difference images and , the atmospheric polarization degree d of the polarization image A and the atmospheric light intensity A at infinity ∞ into the calculation error of the transmission function described in Equation (10), from Equations (7) and (9), we can obtain the polarization boundary constraint transmission function of

[0072] (10)

[0073] where c represents the color channel c of the polarization image, and represent the color channels c of the lower boundary K0 and the upper boundary K1 respectively.

[0074] Through the design of transmission boundary constraints, it is ensured that the present invention can restore the true scene information when the polarization information in some regions cannot be accurately estimated, improving the adaptability of the defogging algorithm to various haze scenes. Secondly, the strategy of the present invention is to use the polarization degree of the highest frequency as the global scalar, resulting in a lower estimated value of the transmission function in the sky region. However, a lower transmission will cause serious noise and distortion in the defogging result. Therefore, it is difficult to effectively restore rich scene information only by using the polarization vector amplification strategy. So, a threshold is further set on the lower boundary to adaptively restore the undistorted defogged image, and the boundary constraint transmission function is:

[0075] (11)

[0076] In the formula, represents the preset lower boundary threshold of the transmission function, which is the average value of the lowest δ% transmission function values, calculated from the lowest values of δ%, and δ% is a preset proportional value. For example, it can be derived from the 6% lowest values in , that is, δ = 6. According to formula (8) and formula (11), the restored scene brightness can be obtained as:

[0077] (12)

[0078] When performing boundary constraints on the atmospheric polarization degree d A , the present invention assumes that in a local area, the polarization degree is the same. However, there are always exceptions. For example, when there are targets with multiple materials in the scene, there will be a situation of polarization degree jump. If continuity is still assumed at this time, edge artifact phenomena will occur. The present invention adds a weight function to the above assumption.

[0079] (13)

[0080] represents the weighting function, and represent two adjacent pixel points, and respectively represent and the corresponding boundary constraint transmission function values. In order to select a reasonable weight function , the present invention notes that the best is related to the polarization degree difference between and . That is, if the and polarization degree difference between is large, It must be very small, and vice versa. Generally, the degree of polarization jump usually appears at the targets of different materials in the image. Therefore, the present invention can calculate the difference in the degree of polarization of local pixels to construct a weighting function, that is:

[0081] (14)

[0082] where σ is the standard deviation parameter, and respectively represent two adjacent pixel points and of the degree of polarization. By adjusting the value of σ, the sensitivity of the algorithm to these differences can be controlled. In order to find the optimal transmission function, the present invention obtains it by optimizing the following objective function:

[0083] (15)

[0084] where m is the regularization parameter for balancing and , and respectively represent the boundary constraint transmission function and the optimized transmission function value. i and j represent any two adjacent pixel points in the polarization image. and respectively represent the boundary constraint transmission function values corresponding to i and j. represents the weighting function of i and j. represents the local area centered on j, I represents the index set of the pixels of the polarization image. represents of the Manhattan norm.

[0085] The beneficial effects of the present invention are as follows:

[0086] 1) The technical solution of the present invention constructs a polarization transmission function with the degree of polarization and reconstructs the atmospheric scattering model. It improves the environmental perception ability of the scene.

[0087] 2) A boundary constraint polarization transmission function is proposed, a constraint combining the weighted L1 norm is established, and an optimization problem is modeled to estimate a more accurate transmission function value. It improves the adaptability and scalability of complex scenes, especially performs well in haze environments, can improve the accuracy of motion estimation of unmanned platforms, and reduce hardware costs.

[0088] 3) The algorithm flow is simple, the calculation amount is small, and it is easy to be programmed and implemented.

[0089] The depth information optimization method for the scene based on the boundary constraint of the transmission function proposed by the present invention solves.

[0090] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for optimizing depth information of a scene based on boundary constraints of a transfer function, characterized in that Including: Obtaining information of a scene detected by a polarization detector, where the information includes a polarization image of the scene and the atmospheric light intensity at infinity; Constructing an initial transmission function based on an atmospheric scattering model according to the degree of atmospheric polarization, the atmospheric light intensity at infinity, and the polarization image; Constructing a polarization boundary constraint transmission function according to the initial transmission function and the boundary constraint of the degree of atmospheric polarization, where the boundary constraint of the degree of atmospheric polarization includes the color channels of the lower boundary of the degree of atmospheric polarization and the color channels of the upper boundary of the degree of atmospheric polarization; Constructing a boundary constraint transmission function according to a preset lower boundary threshold of the transmission function, a maximum value function, the initial transmission function, and the polarization boundary constraint transmission function; Constructing a weight constraint, where the weight constraint is obtained by multiplying the difference between a weighting function between two adjacent pixel points in the polarization image and the value of the boundary constraint transmission function of the two adjacent pixel points, and the weighting function is constructed according to the difference in the degree of polarization of the two adjacent pixel points; Constructing an objective function according to the difference between the value of the boundary constraint transmission function and the value of the optimized transmission function, the sum of all the weight constraints, and a regularization parameter for balancing these two items, and solving for the optimal value of the optimized transmission function with the goal of minimizing the objective function; Obtaining the depth information of the optimized scene according to the optimal value of the optimized transmission function; The initial transfer function is as follows: ， where d A represents the degree of polarization of the atmosphere, represents the vertical polarization image of the polarization image, represents the parallel polarization image in the polarization image, represents the atmospheric light intensity at infinity, where x represents the pixel point of the polarization image; The objective function is: ， Among them, m is the regularization parameter for balance and the regularization parameter, and respectively represent the boundary constraint transfer function and the optimized transfer function value. i and j represent any two adjacent pixel points in the polarization image, and respectively represent the boundary constraint transfer function values corresponding to i and j, represents the weighting function of i and j, represents the local region centered on j, and I represents the index set of the pixels in the polarization image, represents the Manhattan norm of.

2. The depth information optimization method for a scenario based on the boundary constraint of the transfer function according to claim 1, wherein The boundary constraint of the degree of atmospheric polarization is: ， x represents the pixel point of the polarization image, and y represents the color channel of the polarization image. represent the three color channels of the polarization image. K0 and K1 represent the lower and upper boundaries of the atmospheric polarization degree. and represent the color channels y of the lower boundary K0 and the upper boundary K1 respectively. Ω represents the local area of the polarization image. represents the atmospheric polarization degree. represents any pixel point belonging to the local area Ω of the polarization image.

3. The depth information optimization method for a scenario based on the boundary constraint of the transfer function according to claim 2, wherein The polarization boundary constraint transfer function is :[[]]END]] ， where c represents the color channel of the polarization image, and respectively represent the color channel c of the lower boundary K0 and the upper boundary K1.

4. The depth information optimization method for a scenario based on transmission function boundary constraints according to claim 3, characterized in that The boundary constraint transmission function is: ， Represents the lower boundary threshold of the preset transfer function, which is the average value of the lowest δ% transfer function values, calculated from the lowest values, where δ% is a preset proportional value.

5. The depth information optimization method for a scenario based on transmission function boundary constraints according to claim 4, characterized in that The weight constraint is: ， ， represents a weighting function, and represent two adjacent pixel points in the polarization image, and respectively represent and the corresponding boundary constraint transfer function values. When = 0, the weight constraint is invalid, and respectively represent two adjacent pixel points and in the polarization image, and σ is the standard deviation parameter.

Images