Scene depth information optimization method based on transmission function boundary constraint

Abstract

The invention discloses a depth information optimization method of a scene based on transmission function boundary constraint, which relates to the technical field of image processing, and comprises the following steps: constructing an initial transmission function based on an atmospheric scattering model; constructing a polarization boundary constraint transmission function according to the initial transmission function and the boundary constraint of the atmospheric polarization degree; constructing a boundary constraint transmission function according to the initial transmission function, the polarization boundary constraint transmission function, a preset transmission function lower boundary threshold value and a maximum value function; constructing a weight constraint; constructing a target function according to the difference between the boundary constraint transmission function value and the optimized transmission function value, the sum of all weight constraints in the image of the information and a regularization parameter for balancing the boundary constraint transmission function value and the optimized transmission function value, and solving an optimal optimized transmission function value by taking minimization of the target function as a target; and obtaining depth information of the optimized scene according to the optimal optimization transmission function value. According to the invention, the environment perception capability of the scene is improved, and the adaptability and expansibility of the complex scene are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for optimizing depth information of a scene based on a transfer function boundary constraint. Background Art

[0002] With the rapid development of science and technology, autonomous driving, robot navigation and motion estimation technologies have been widely used in various fields. However, these technologies face huge challenges in practical applications, especially in low-texture scenes such as haze. Haze weather causes the image information captured by visual sensors to be blurred and low-contrast, which seriously affects the unmanned platform's ability to perceive the scene, thereby increasing the difficulty of motion estimation.

[0003] Traditional image dehazing methods mainly rely on image processing algorithms to enhance images, but these methods often have difficulty in accurately restoring the true information of images when processing haze images, resulting in limited dehazing effects. In recent years, with the rise of polarization imaging technology, new ideas have been provided for the processing of haze images. Polarization imaging technology can obtain more image information by measuring the polarization state of light waves, especially in scattering media such as haze, where polarization information has a higher signal-to-noise ratio and stronger anti-interference ability.

[0004] However, relying solely on polarization imaging technology cannot completely solve the problem of dehazing haze images. In practical applications, due to the scattering effect of haze particles, accurate estimation of polarization information also faces certain difficulties. In addition, targets of different materials exhibit different polarization characteristics in polarization imaging, which also brings additional challenges to image processing.

[0005] Therefore, how to use polarization imaging technology to accurately estimate the transfer function in low-texture scenes such as haze and effectively remove the impact of haze on the image has become a problem that needs to be solved urgently. Summary of the invention

[0006] The present invention proposes a method for optimizing the depth information of a scene based on a transfer function boundary constraint, so as to solve the problem that it is difficult to clearly perceive the effective information of the scene and motion estimation is difficult in low-texture scenes such as haze in the prior art.

[0007] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0008] The present invention provides a method for optimizing depth information of a scene based on a transfer function boundary constraint, comprising:

[0009] Acquiring information of a scene detected by a polarization detector, the information including a polarization image of the scene and atmospheric light intensity at infinity;

[0010] Constructing an initial transmission function based on an atmospheric scattering model according to the atmospheric polarization degree, the atmospheric light intensity at infinity and the polarization image;

[0011] Constructing a polarization boundary constraint transfer function according to the initial transfer function and the boundary constraint of the atmospheric polarization degree, wherein the boundary constraint of the atmospheric polarization degree includes a color channel of a lower boundary of the atmospheric polarization degree and a color channel of an upper boundary of the atmospheric polarization degree;

[0012] Constructing a boundary-constrained transmission function according to a preset lower boundary threshold of the transmission function, a maximum value function, the initial transmission function, and the polarization boundary-constrained transmission function;

[0013] Constructing a weight constraint, wherein the weight constraint is obtained by multiplying a weighting function between two adjacent pixels in the polarization image by a difference between boundary constraint transfer function values ​​of the two adjacent pixels, and the weighting function is constructed according to a difference in polarization degree between the two adjacent pixels;

[0014] Constructing an objective function according to the difference between the boundary constraint transfer function value and the optimized transfer function value, the sum of all the weight constraints, and a regularization parameter that balances the two items, and solving the optimal optimized transfer function value with the goal of minimizing the objective function;

[0015] The depth information of the optimized scene is obtained according to the optimal optimized transfer function value.

[0016] Furthermore, the initial transfer function for:

[0017]

[0018] Among them, d A represents the degree of atmospheric polarization, a vertical polarization image representing the polarization image, represents the parallel polarization image in the polarization image, It means obtaining the atmospheric light intensity at infinity, where x is the pixel point of the polarization image.

[0019] Furthermore, the boundary constraint of the atmospheric polarization degree is:

[0020]

[0021] x represents a pixel point of the polarization image, y represents a color channel y of the polarization image, Represents the three color channels of the polarization image, K 0 and K 1 Expressed as the lower and upper boundaries of the atmospheric polarization degree, and They are represented as the lower boundary K0 and the upper boundary K 1 The color channel y,Ω represents the local area of ​​the polarization image, represents the degree of atmospheric polarization, represents any pixel in the local area Ω belonging to the polarization image.

[0022] Furthermore, the polarization boundary constraint transfer function is :

[0023]

[0024] Wherein, c represents the color channel c of the polarization image, and They are represented as the lower boundary K 0 and the upper boundary K 1 The color channel c.

[0025] Furthermore, the boundary constraint transfer function is:

[0026]

[0027] Indicates the preset lower boundary threshold of the transfer function, that is, the average value of the lowest δ% transfer function value. The minimum value is calculated, and δ% is the preset ratio value.

[0028] Furthermore, the weight constraint is:

[0029]

[0030]

[0031] represents the weighting function, and represents two adjacent pixel points in the polarization image, and Respectively and The corresponding boundary constraint transfer function value is =0, the weight constraint is invalid. and Respectively represent two adjacent pixel points in the polarization image and The polarization degree of , σ is the standard deviation parameter.

[0032] Furthermore, the objective function is:

[0033]

[0034] Among them, m is the balance and The regularization parameter of and represent the boundary constraint transfer function and the optimized transfer function value respectively, i and j represent any two adjacent pixels in the polarization image, and represent the boundary constraint transfer function values ​​corresponding to i and j respectively, represents the weighting function of i and j, represents the local area centered at j, I represents the index set of the polarization image pixels, express The Manhattan norm of .

[0035] The beneficial effects of the present invention are:

[0036] 1) The technical solution of the present invention constructs the polarization transfer function using the polarization degree and reconstructs the atmospheric scattering model, thereby improving the environmental perception capability of the scene.

[0037] 2) A boundary-constrained polarization transfer function is proposed, a constraint combined with the weighted L1 norm is established, and an optimization problem is modeled to estimate a more accurate transfer function value. The adaptability and scalability of complex scenes are improved, especially in haze environments, which can improve the accuracy of unmanned platform motion estimation and reduce hardware costs.

[0038] 3) The algorithm flow is simple, the amount of calculation is small, and it is easy to program and implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of the depth information optimization method of the scene based on the transfer function boundary constraint of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0043] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0044] like Figure 1 As shown, a method for optimizing depth information of a scene based on a transfer function boundary constraint includes:

[0045] S1: Acquire information of a scene detected by a polarization detector, the information including a polarization image of the scene and atmospheric light intensity at infinity;

[0046] S2: constructing an initial transmission function based on an atmospheric scattering model according to the atmospheric polarization degree, the atmospheric light intensity at infinity and the polarization image;

[0047] S3: constructing a polarization boundary constraint transfer function according to the initial transfer function and the boundary constraint of the atmospheric polarization degree, wherein the boundary constraint of the atmospheric polarization degree includes a color channel of a lower boundary of the atmospheric polarization degree and a color channel of an upper boundary of the atmospheric polarization degree;

[0048] S4: constructing a boundary constraint transmission function according to a preset lower boundary threshold of the transmission function, a maximum value function, the initial transmission function, and the polarization boundary constraint transmission function;

[0049] S5: constructing a weight constraint, wherein the weight constraint is obtained by multiplying a weighting function between two adjacent pixels in the polarization image by a difference between boundary constraint transfer function values ​​of the two adjacent pixels, and the weighting function is constructed according to the difference in polarization degree between the two adjacent pixels;

[0050] S6: constructing an objective function according to the difference between the boundary constraint transfer function value and the optimized transfer function value, the sum of all the weight constraints, and a regularization parameter that balances the two items, and solving the optimal optimized transfer function value with the goal of minimizing the objective function;

[0051] S7: Obtaining depth information of the optimized scene according to the optimal optimized transfer function value.

[0052] In some embodiments, under haze weather conditions, the light intensity reaching the detector mainly includes two parts: 1) the reflected light of the scene target, also known as direct transmitted light, which contains the intensity information of the scene target; 2) the stray light caused by the scattering of haze particles, also known as atmospheric light, which is the main interference factor for optical imaging in haze environments. First, the reflected light intensity L(x) of the scene target passes through the haze area and is strongly scattered and absorbed by the haze particles. The direct transmitted light intensity D(x) reaching the detector decays exponentially with the transmission distance:

[0053] (1)

[0054] Where β is the attenuation coefficient and z(x) is the distance from the target to the detector. In general, the transfer function is defined as:

[0055] (2)

[0056] Different from directly transmitted light, atmospheric light is caused by haze particles directly scattering sunlight. When it reaches the detector, the light intensity A(x) increases exponentially with the transmission distance:

[0057] (3)

[0058] Among them A ∞ is the atmospheric light intensity at infinity, indicating the atmospheric light without a target. The total light intensity I(x) reaching the detector is the incoherent superposition of the direct transmission light intensity and the atmospheric light intensity, which can be expressed as:

[0059] (4)

[0060] From equation (3), we can get the transfer function t in the atmospheric scattering model: r (x) is:

[0061] (5)

[0062] The atmospheric polarization degree defined in the atmospheric scattering model is:

[0063] (6)

[0064] Combining formulas (5) and (6) to eliminate the atmospheric light A, we can obtain the atmospheric polarization degree d A represents the initial transfer function The expression is:

[0065] (7)

[0066] After using polarization information to obtain the polarization transfer function and the estimated value of the atmospheric light at infinity, the dehazed image can be expressed as:

[0067] (8)

[0068] In some embodiments, when calculating the atmospheric polarization degree d A When d A There will be negative values ​​or calculated values ​​greater than 1. Generally, d AIt is between 0 and 1, so outliers need to be eliminated. Considering that the atmospheric polarization in the scene is always bounded, without loss of generality, considering the y color channel, , K 0 and K 1 Expressed as the lower and upper boundaries of the atmospheric polarization degree, and They are represented as the lower boundary K 0 and the upper boundary K 1 The purpose of the boundary constraint is to eliminate the atmospheric polarization of the scene. A The outliers caused by the error are used to obtain the accurate transfer function value corresponding to the boundary. The boundary constraint of the atmospheric polarization degree is:

[0069] (9)

[0070] Where Ω is the local area of ​​a given image, and the boundary values ​​are set to and , represents the degree of atmospheric polarization, represents any pixel in the local area Ω belonging to the polarization image.

[0071] The boundary constraint on the atmospheric polarization degree can be further expressed as a boundary constraint on the transfer function. Since the transfer function describes the impact of haze on image pixels, the boundary constraint on the transfer function aims to obtain the dehazed pixels just reaching the given lower boundary. or upper boundary The corresponding transfer function value is , and further, for the obtained transfer function value, its maximum value in the three color channels means that the defogging pixel just reaches the lower boundary K in one color channel 0 Or upper boundary K 1 The transfer function value of . Assuming that the polarization difference image is given and and the atmospheric light intensity A at infinity ∞ . The polarization difference image and , atmospheric polarization degree d of polarization image A and the atmospheric light intensity A at infinity ∞ The resulting calculation error is constrained to the transfer function described by equation (10), and from equations (7) and (9) we can get The polarization boundary constraint transfer function is:

[0072] (10)

[0073] Wherein, c represents the color channel c of the polarization image, and They are represented as the lower boundary K 0 and the upper boundary K 1 The color channel c.

[0074] By designing the transmission boundary constraint, it is ensured that the present invention can restore the real scene information when the polarization information in certain areas cannot be accurately estimated, thereby improving the adaptability of the defogging algorithm to various haze scenes. Secondly, the strategy of the present invention is to use the highest frequency polarization degree as the global scalar, which leads to a lower estimate of the transmission function value in the sky area, and the lower transmission will cause serious noise and distortion in the defogging result. Therefore, it is difficult to effectively restore rich scene information using only the polarization vector amplification strategy. Therefore, a threshold is further set on the lower boundary to adaptively restore the undistorted defogging image. The boundary constraint transfer function is:

[0075] (11)

[0076] In the formula, Indicates the preset lower boundary threshold of the transfer function, that is, the average value of the lowest δ% transfer function value. The lowest value is calculated from the default value, and δ% is the preset ratio. The lowest value of 6% is derived , that is, δ=6. According to equations (8) and (11), the restored scene brightness can be obtained as:

[0077] (12)

[0078] The atmospheric polarization degree d A When performing boundary constraints, the present invention assumes that the polarization degree is the same in a local area, but there are always exceptions. For example, when there are targets of various materials in the scene, polarization degree jumps will occur. If continuity is still assumed at this time, edge artifacts will appear. The present invention adds a weight function to the above assumptions.

[0079] (13)

[0080] represents the weighting function, and Represents two adjacent pixels. and Respectively and The corresponding boundary constraint transfer function value. In order to select a reasonable weight function , note that the best and and That is, if and There is a big difference in polarization between them. must be small, and vice versa. Generally, polarization jumps usually occur at different material targets in the image. Therefore, the present invention can calculate the polarization difference of local pixels to construct a weighting function, that is:

[0081] (14)

[0082] Where σ is the standard deviation parameter, and Represents two adjacent pixels respectively and By adjusting the value of σ, the sensitivity of the algorithm to these differences can be controlled. In order to find the optimal transmission function, the present invention optimizes the following objective function:

[0083] (15)

[0084] Among them, m is the balance and The regularization parameter of and represent the boundary constraint transfer function and the optimized transfer function value respectively, i and j represent any two adjacent pixels in the polarization image, and represent the boundary constraint transfer function values ​​corresponding to i and j respectively, represents the weighting function of i and j, represents the local area centered at j, I represents the index set of the polarization image pixels, express The Manhattan norm of .

[0085] The beneficial effects of the present invention are:

[0086] 1) The technical solution of the present invention constructs the polarization transfer function using the polarization degree and reconstructs the atmospheric scattering model, thereby improving the environmental perception capability of the scene.

[0087] 2) A boundary-constrained polarization transfer function is proposed, a constraint combined with the weighted L1 norm is established, and an optimization problem is modeled to estimate a more accurate transfer function value. The adaptability and scalability of complex scenes are improved, especially in haze environments, which can improve the accuracy of unmanned platform motion estimation and reduce hardware costs.

[0088] 3) The algorithm flow is simple, the amount of calculation is small, and it is easy to program and implement.

[0089] The present invention proposes a method for optimizing scene depth information based on transfer function boundary constraints.

[0090] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for optimizing depth information of a scene based on a transfer function boundary constraint, characterized in that: include: Acquiring information of a scene detected by a polarization detector, the information including a polarization image of the scene and atmospheric light intensity at infinity; Constructing an initial transmission function based on an atmospheric scattering model according to the atmospheric polarization degree, the atmospheric light intensity at infinity and the polarization image; Constructing a polarization boundary constraint transfer function according to the initial transfer function and the boundary constraint of the atmospheric polarization degree, wherein the boundary constraint of the atmospheric polarization degree includes a color channel of a lower boundary of the atmospheric polarization degree and a color channel of an upper boundary of the atmospheric polarization degree; Constructing a boundary-constrained transmission function according to a preset lower boundary threshold of the transmission function, a maximum value function, the initial transmission function, and the polarization boundary-constrained transmission function; Constructing a weight constraint, wherein the weight constraint is obtained by multiplying a weighting function between two adjacent pixels in the polarization image by a difference between boundary constraint transfer function values ​​of the two adjacent pixels, and the weighting function is constructed according to a difference in polarization degree between the two adjacent pixels; Constructing an objective function according to the difference between the boundary constraint transfer function value and the optimized transfer function value, the sum of all the weight constraints, and a regularization parameter that balances the two items, and solving the optimal optimized transfer function value with the goal of minimizing the objective function; The depth information of the optimized scene is obtained according to the optimal optimized transfer function value.

2. The method for optimizing depth information of a scene based on a transfer function boundary constraint according to claim 1, characterized in that: The initial transfer function for: ， Among them, d A represents the degree of atmospheric polarization, a vertical polarization image representing the polarization image, represents the parallel polarization image in the polarization image, represents the atmospheric light intensity at infinity, where x represents the pixel point of the polarization image.

3. The method for optimizing scene depth information based on transfer function boundary constraints according to claim 2, characterized in that: The boundary constraint of the atmospheric polarization degree is: ， x represents a pixel point of the polarization image, y represents a color channel y of the polarization image, represents the three color channels of the polarization image, K0 and K1 represent the lower and upper boundaries of the atmospheric polarization degree, and The color channels y are represented as the lower boundary K0 and the upper boundary K1, Ω represents the local area of ​​the polarization image, represents the degree of atmospheric polarization, represents any pixel in the local area Ω belonging to the polarization image.

4. The method for optimizing scene depth information based on transfer function boundary constraints according to claim 3, characterized in that: The polarization boundary constraint transfer function is : ， Wherein, c represents the color channel c of the polarization image, and The color channels c are represented as the lower boundary K0 and the upper boundary K1 respectively.

5. The method for optimizing scene depth information based on transfer function boundary constraints according to claim 4, characterized in that: The boundary constraint transfer function is: ， Indicates the preset lower boundary threshold of the transfer function, that is, the average value of the lowest δ% transfer function value. The minimum value is calculated, and δ% is the preset ratio value.

6. The method for optimizing scene depth information based on transfer function boundary constraints according to claim 5, characterized in that: The weight constraint is: ， ， represents the weighting function, and represents two adjacent pixel points in the polarization image, and Respectively and The corresponding boundary constraint transfer function value is =0, the weight constraint is invalid. and Respectively represent two adjacent pixels in the polarization image and The polarization degree of , σ is the standard deviation parameter.

7. The method for optimizing scene depth information based on transfer function boundary constraints according to claim 6, characterized in that: The objective function is: ， Among them, m is the balance and The regularization parameter of and represent the boundary constraint transfer function and the optimized transfer function value respectively, i and j represent any two adjacent pixels in the polarization image, and represent the boundary constraint transfer function values ​​corresponding to i and j respectively, represents the weighting function of i and j, represents the local area centered at j, I represents the index set of the polarization image pixels, express The Manhattan norm of .

Images