A dynamic scene polarization three-dimensional reconstruction method fusing image optical flow

Abstract

The application discloses a dynamic scene polarization three-dimensional reconstruction method fusing image light flow, and comprises the following steps: a polarization three-dimensional reconstruction system is built, and dynamic scene polarization images of an object to be reconstructed at different angles are acquired; the polarization degree of the surface of the object is calculated by using the dynamic scene polarization images at different angles; a three-dimensional surface function of the target object is reconstructed according to the mapping relationship between the three-dimensional profile information of the target surface micro-patch and the target surface normal vector; the dynamic scene is reconstructed by using an image light flow algorithm, and a three-dimensional surface reconstruction result of the target object is obtained; the three-dimensional surface reconstruction result obtained by fusing the image light flow is used to correct the polarization three-dimensional reconstruction result obtained by using the target micro-patch normal vector, and finally, a three-dimensional reconstruction result is obtained. The application corrects the reconstruction result in the dynamic scene obtained by polarization three-dimensional imaging by fusing the image light flow, and effectively improves the accuracy of polarization three-dimensional imaging in the dynamic scene.

Description

Technical Field

[0001] This invention belongs to the field of optical imaging technology, specifically relating to a dynamic scene polarization three-dimensional reconstruction method that fuses image optical flow. Background Technology

[0002] 3D reconstruction is an important research area in optical engineering technology. It's a key technology that reconstructs the 3D shape of a target by analyzing 2D images and combining this with computer vision knowledge. Based on this, 3D imaging technology is increasingly widely used in industrial production, facial recognition, and security detection. Polarization 3D imaging technology, as an important branch of 3D imaging technology, has advantages such as simple equipment, high precision, and few application limitations. Therefore, research on polarization 3D imaging technology is of great significance. However, in dynamic scenes, polarization 3D imaging technology suffers from severe polarization image noise and blurred reconstruction normal azimuth angles, which greatly limits its application in fields such as robot vision, aerospace, military, and medical research.

[0003] Existing monocular polarization 3D reconstruction methods achieve 3D reconstruction of object surfaces based on polarization information obtained from a camera. These methods provide relatively rich details in the reconstruction results and have certain applications in the field of 3D reconstruction technology. The monocular polarization 3D reconstruction process is as follows: First, a polarization 3D reconstruction system is built. By rotating a polarizer, polarization images of the target at 0°, 45°, 90°, and 135° are acquired. From the acquired polarization images, a light intensity variation curve is obtained by fitting the relationship between the polarizer rotation angle and the light intensity value. Then, the angle corresponding to the maximum light intensity in the curve is obtained as the azimuth angle of the target surface normal. Four Stokes vectors of the target are obtained from the four polarization images, and the polarization degree of the target is calculated. Next, the zenith angle of the target surface normal is obtained based on the relationship between the target polarization degree and the incident angle. The target surface normal gradient field is obtained based on the acquired surface normal azimuth angle and zenith angle, and then the target surface shape is obtained through 3D reconstruction using the gradient field.

[0004] Monocular polarization-based 3D reconstruction methods suffer from errors in normal direction due to the uncertainty of the normal azimuth angle during the acquisition of the target surface normal. This leads to distortion and inaccuracies in the reconstructed 3D object surface, making this method challenging and resulting in low accuracy. Particularly in dynamic scenes, polarization-based 3D reconstruction using this method suffers from increased polarization image noise, further blurring the reconstructed normal azimuth angle. Relying solely on this method makes it difficult to obtain highly accurate 3D reconstruction results. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention provides a dynamic scene polarization 3D reconstruction method that integrates image optical flow. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] This invention provides a dynamic scene polarization 3D reconstruction method based on image optical flow fusion, comprising:

[0007] S1: Build a polarization 3D reconstruction system to obtain dynamic scene polarization images of the object to be reconstructed at different angles;

[0008] S2: Calculate the degree of polarization of the object's surface using the dynamic scene polarization images from different angles;

[0009] S3: Reconstruct the three-dimensional surface function of the target object based on the mapping relationship between the three-dimensional contour information of the target surface micro-element and the target surface normal vector;

[0010] S4: Use the image optical flow algorithm to reconstruct the dynamic scene and obtain the three-dimensional surface reconstruction result of the target object;

[0011] S5: The three-dimensional surface reconstruction result obtained by fusing image optical flow is corrected to the polarization three-dimensional reconstruction result obtained by using the target micro-surface element normal vector to obtain the final three-dimensional reconstruction result.

[0012] In one embodiment of the present invention, S3 includes:

[0013] S3.1: Utilizing polarization images I0 and I at different angles 45 I 90 and I 135 The azimuth angle is calculated based on the Stokes vector.

[0014] S3.2: The zenith angle θ is calculated based on the relationship between the degree of polarization P and the zenith angle θ;

[0015] S3.3: Based on the mapping relationship between the three-dimensional contour information of the target surface micro-element and the target surface normal vector, reconstruct the target three-dimensional surface function z(u).

[0016] In one embodiment of the present invention, S3.3 includes:

[0017] S3.31: Construct the target micro-element normal vector using the azimuth angle and the zenith angle:

[0018]

[0019] Where, n x (u), n y (u), n z (u) represent the target surface normal vectors respectively. The components in the x, y, and z axes, u = (x, y) represent the pixels on the image corresponding to the target surface;

[0020] S3.32: Obtain the correspondence between the target micro-element normal vector and the target surface function:

[0021]

[0022] in, Let represent the gradient fields of the surface normal vectors of the target object's micro-element in the x and y directions, respectively, and z(u) represent the height at point u on the surface of the target object. Represents the partial derivative in the x-direction. This represents the partial derivative in the y-direction;

[0023] S3.33: By using Fourier orthogonal basis decomposition, the discrete gradient field information and surface function information in the spatial domain are linked in the Fourier domain to reconstruct the surface function z(u) of the target object.

[0024] In one embodiment of the present invention, S4 includes:

[0025] S4.1: Obtain the three-dimensional motion velocity of the target object and the correspondence between the depth coordinates of the target object's surface and the optical flow of the image;

[0026] S4.2: Obtain the three-dimensional motion detection energy function ε based on the correspondence between the target object's three-dimensional motion velocity, the target object's surface depth coordinates, and the image optical flow;

[0027] S4.3: Calculate the three-dimensional motion detection energy function to obtain the relative three-dimensional motion [ω] of the moving object's surface. x ,ω y ,ω z ,T x ',T y ',T z '];

[0028] S4.4: According to the translational velocity T'=[T x ',T y ',T z '] T Obtain the relative depth coordinate Z of the target object.

[0029] In one embodiment of the present invention, S4.1 includes:

[0030] S4.11: Assume point P is a point on the surface of a moving object in three-dimensional space, with three-dimensional coordinates (X, Y, Z), and point p is the corresponding projection point of point P onto the camera's imaging plane, with coordinates (x, y). Then, the following holds:

[0031]

[0032] Wherein, the camera focal length is f;

[0033] S4.12: Setting f = 1, we obtain the three-dimensional equation of motion for point P:

[0034]

[0035] Where (X',Y',Z') represents the velocity of point P, ω(ω x ,ω y ,ω z T(T) represents the three-dimensional rotational velocity of a moving object in three-dimensional space. x ,T y ,T z () represents the translational velocity of a moving object in three-dimensional space;

[0036] S4.13: Assuming the optical flow at point p on the projection plane is (u, v), then the following holds:

[0037] (u,v)=(x',y')

[0038] Where (x',y') represents the velocity of point p;

[0039] S4.14: Obtain the correspondence between image optical flow, 3D motion velocity, and object surface depth coordinates:

[0040]

[0041]

[0042] Where Z represents the surface depth coordinates of the target object.

[0043] In one embodiment of the present invention, the energy function is:

[0044]

[0045] Where λ is the coefficient of the smoothing term. It is the spatial gradient smoothing factor.

[0046] a = [a x ,a y ,a z ] T =[-xyf x -(1+y 2 )f y ,(1+x 2 )f x +xyf y ,-yf x+xf y ] T

[0047] b = [b x ,b y ,b z ] T =[f x ,f y ,-xf x -yf y ] T

[0048]

[0049] Among them, f x f y f t These represent the gradients of the gray level of pixel p(x,y) in the camera's imaging plane along the x-axis, y-axis, and time t, respectively.

[0050] In one embodiment of the present invention, S5 includes:

[0051] The gradients of the three-dimensional surface function of the target object reconstructed in step S3 and the three-dimensional surface reconstruction result reconstructed in step S4 are calculated respectively to obtain their respective gradient information. The gradient information of the three-dimensional surface based on image optical flow is used to correct the three-dimensional surface reconstruction result obtained by polarization to obtain the final three-dimensional surface reconstruction result of the target object.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] 1. This invention proposes a polarization-based 3D reconstruction method that integrates image optical flow, which can effectively eliminate the influence of dynamic scenes on the polarization-based 3D reconstruction results. It comprehensively considers the relevant issues in polarization-based 3D imaging technology, making the dynamic object details in the 3D reconstruction richer and expanding the application scope of 3D reconstruction technology.

[0054] 2. This invention corrects the reconstruction results of dynamic scenes obtained by polarization 3D imaging by fusing image optical flow, overcoming the problems of severe noise in the acquired images and blurred reconstruction normal azimuth angles encountered in polarization 3D imaging under dynamic scenes. It effectively improves the accuracy of polarization 3D imaging under dynamic scenes and makes the application range of polarization 3D reconstruction method wider.

[0055] 3. The present invention uses the fusion of optical flow information to perform polarization 3D reconstruction of dynamic scenes. In dynamic scenes, the reconstruction results of image optical flow are used to correct the polarization 3D reconstruction results, overcoming the problems of severe noise in the acquired image and blurred reconstruction normal azimuth angle encountered in polarization 3D imaging in dynamic scenes, and improving the accuracy of polarization 3D reconstruction in such scenes.

[0056] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0057] Figure 1 This is a flowchart of a dynamic scene polarization 3D reconstruction method that integrates image optical flow, provided by an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of a polarization three-dimensional reconstruction system provided in an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of a normal characterization model of a target surface provided in an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of a camera imaging plane provided in an embodiment of the present invention. Detailed Implementation

[0061] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following describes in detail a dynamic scene polarization three-dimensional reconstruction method based on image optical flow proposed in accordance with the present invention, in conjunction with the accompanying drawings and specific embodiments.

[0062] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0063] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0064] Please see Figure 1 , Figure 1 This is a flowchart of a dynamic scene polarization 3D reconstruction method based on fused image optical flow, provided by an embodiment of the present invention. The dynamic scene polarization 3D reconstruction method includes:

[0065] S1: Build a polarization 3D reconstruction system to obtain dynamic scene polarization images of the object to be reconstructed at different angles.

[0066] Please see Figure 2 , Figure 2 This is a schematic diagram of a polarization-based 3D reconstruction system provided in an embodiment of the present invention. The system simulates a dynamic scene; in a laboratory environment, the object to be reconstructed is placed on an electrically driven displacement platform. In a natural light environment, a polarization camera collects reflected light from the object's surface, and the camera acquires polarization images I0, I10, I20, and I30 from four angles of the dynamic object. 45 I 90 and I 135 .

[0067] S2: Calculate the degree of polarization P of the object surface using the dynamic scene polarization images at different angles.

[0068] Stokes vector representation is a commonly used method for representing polarization characteristics. It states that the polarization state of a beam of light can be completely described by four fixed parameters, called the Stokes vector. Since each Stokes parameter is expressed in terms of light intensity, it can be directly measured using certain photoelectric instruments. The Stokes vector is represented as:

[0069]

[0070] Among them, E x and E y Let I represent the components of the electric field vector of the reflected light from the object's surface along the x and y axes, respectively. L and I R These represent the light intensity of left-handed circularly polarized light and the light intensity of right-handed circularly polarized light, respectively.

[0071] The formula for calculating the degree of polarization P using the Stokes vector is:

[0072]

[0073] S3: Reconstruct the three-dimensional surface function of the target based on the mapping relationship between the three-dimensional contour information of the target surface micro-element and the target surface normal vector.

[0074] Please see Figure 3 , Figure 3 This is a schematic diagram of a normal characterization model of a target surface provided in an embodiment of the present invention. Step S3 specifically includes:

[0075] S3.1: Utilizing polarization images I0 and I at different angles 45 I 90 and I 135 The azimuth angle of the incident light on the object surface is calculated based on the Stokes vector. The calculation formula is:

[0076]

[0077] Because the light intensity of the polarized images obtained from two rotation angles spaced 180° apart is the same during the rotation of the polarizer, there is a 180° uncertainty in the incident azimuth angle of the incident light on the surface of the object to be reconstructed compared to the actual incident azimuth angle. This leads to uncertainty in the direction of the surface normal obtained from the polarization information, thus requiring correction of the surface normal.

[0078] S3.2: Based on the relationship between the degree of polarization P and the zenith angle θ (the angle of incidence of the incident light on the object surface), the zenith angle θ is calculated using the following formula:

[0079]

[0080] Where n represents the refractive index of the object's surface, and in this embodiment, the refractive index of the object's surface is generally taken as 1.5.

[0081] S3.3: Reconstruct the three-dimensional surface function of the target based on the mapping relationship between the three-dimensional contour information of the target surface micro-element and the target surface normal vector.

[0082] Specifically, based on the mapping relationship between the three-dimensional contour information of the target surface micro-element and its normal vector, the three-dimensional contour of the target can be reconstructed by solving for the normal vector information of the micro-element. There is a close relationship between the normal vector of the object surface micro-element and the polarization characteristics of the reflected light, such as... Figure 3 As shown, the normal vector The direction can be determined by two polarization characteristic parameters: zenith angle θ and azimuth angle. Constraints, then the normal vector With zenith angle θ and azimuth angle The mapping relationship between them can be represented as:

[0083]

[0084] Where u = (x, y) represents the pixel point on the image corresponding to the target object, and n x (u), n y (u), n z (u) represent the surface normal vectors of the target object. The components along the x, y, and z axes. In 3D reconstruction, it is typically assumed that the target surface is convex, meaning the normal vectors of the target micro-element all point towards the detector side. Therefore, the zenith angle θ and azimuth angle are defined. The value space of θ is [0°, 90°]. Therefore, in the actual process of solving the three-dimensional contour of the target surface, the polarization characteristics of the reflected light from the target surface are used to determine the zenith angle θ and azimuth angle. By solving the problem, the three-dimensional contour of the target can be reconstructed.

[0085] Specifically, the zenith angle θ and azimuth angle of the micro-surface element are obtained by utilizing the polarization characteristics of the reflected light from the target surface. Next, obtain the normal vector n(u) of the target micro-surface element:

[0086]

[0087] in, Let represent the gradient fields of the surface normal vectors of the target object's micro-element in the x and y directions, respectively, and z(u) represent the height at point u on the surface of the target object. Represents the partial derivative in the x-direction. This represents the partial derivative in the y-direction.

[0088] Furthermore, the gradient field on the surface of the target object can be written as For globally continuous normal gradient fields, local or global integration methods are typically used to reconstruct the 3D contour of the target. However, for the recovery of normal gradient fields of general targets, during the 3D surface solution process, due to system noise interference or the complexity of the target surface, discrete non-integrable regions exist on the target surface, making it impossible to directly reconstruct the 3D surface through integration. Based on the Frankot-Chellappa 3D surface restoration function, by projecting the non-integrable regions in the target normal gradient field information onto the integrable surface slope subspace, the distance function between the normal gradient field and the continuous integrable micro-element is defined as:

[0089] d{(p,q),(z x ,z y )}=∫∫|z x -p| 2 +|z y -q| 2 dxdy

[0090] Among them, z x Let z represent the gradient field of the target surface in the x-direction. y Let represent the gradient field of the target surface in the y direction, p represent the gradient field of the target micro-surface element normal vector in the x direction, and q represent the gradient field of the target micro-surface element normal vector in the y direction.

[0091] When the above equation is minimized, the normal gradient field and the integrable infinitesimal element are orthogonal projections. Then the surface function z(u) can be expressed as a series of orthogonal basis functions. A linear combination, w = (w x ,w y() represents a two-dimensional index. Therefore, through a linear combination of orthogonal basis functions, the surface function z(u) can be expressed in the form shown in the following equation:

[0092]

[0093] Where C(w) are the expansion coefficients of z(u), and the gradient field of the surface function is expressed in the orthogonal function expansion form shown in the following equation:

[0094]

[0095]

[0096] in, and Let q represent the optimal sets of coefficients in the x and y directions, respectively. g (u) and p g (u) represents the gradient fields of the surface function in the x and y directions, respectively. and Let represent the orthogonal basis functions in the x and y directions, respectively. To simplify the algorithm implementation, Fourier coefficients with a complete orthogonal basis are chosen for expansion, and their basis functions can be characterized as:

[0097]

[0098] Where M and N represent the dimensions of the two-dimensional image, respectively, by performing a Fourier transform on the basis functions and combining it with the above formula, the gradient field of the surface function can be obtained as shown in the following equation:

[0099]

[0100] Finally, the surface function z(u) of the object is obtained with respect to the gradient field (p) of the micro-element. g ,q g The representation of ) is as follows:

[0101]

[0102] Where m and n represent the Fourier coefficients of the discrete microoperator in the x and y directions, respectively; F{} and F -1 {} denote the Discrete Fourier Transform and the Inverse Discrete Fourier Transform, respectively, and j represents the imaginary identity element. This model uses Fourier orthogonal basis decomposition to establish a connection between the discrete gradient field information and surface function information in the spatial domain within the Fourier domain, thereby reconstructing the surface function z(u) and achieving the recovery of the target surface.

[0103] The output is a point cloud image, which can be directly processed to obtain the gradient information of the target's 3D result.

[0104] S4: Use the image optical flow algorithm to reconstruct the dynamic scene and obtain the three-dimensional surface reconstruction result of the target object.

[0105] Specifically, step S4 in this embodiment includes:

[0106] S4.1: Obtain the three-dimensional motion velocity of the target object and the correspondence between the depth coordinates of the target object's surface and the optical flow of the image.

[0107] Please see Figure 4 , Figure 4 This is a schematic diagram of a camera imaging plane provided in an embodiment of the present invention. The camera lens is positioned at the origin O of a three-dimensional coordinate system, with the Z-axis of the coordinate system coinciding with the camera's optical axis, and the camera's focal length is f. A moving object exists in the three-dimensional space within the camera's field of view, and the three-dimensional rotational speed of this moving object is ω(ω... x ,ω y ,ω z The translational speed is T(T x ,T y ,T z ).

[0108] Assuming point P is a point on the observable surface of the moving object, with three-dimensional coordinates (X, Y, Z), and point p is the corresponding projection point of point P in three-dimensional space onto the camera's imaging plane, with coordinates (x, y), the correspondence between their coordinates is as follows:

[0109]

[0110] The camera's focal length is f.

[0111] Given f = 1, the three-dimensional equations of motion for point P are as follows:

[0112]

[0113] Where (X',Y',Z') represents the velocity of point P, specifically the velocity components in three directions.

[0114] Optical flow refers to the instantaneous velocity generated by the movement of pixels on the image plane. Assuming the optical flow at point p on the projection plane is (u, v), then:

[0115] (u,v)=(x',y')

[0116] Where (x',y') represents the derivative of the x and y coordinates of point p with respect to time, that is, the speed of movement of pixel p.

[0117] The resulting equations show the correspondence between image optical flow, 3D motion velocity, and the depth coordinate Z of the object's surface:

[0118]

[0119]

[0120] Where Z represents the surface depth coordinates of the target object.

[0121] S4.2: Obtain the three-dimensional motion detection energy function based on the correspondence between the three-dimensional motion velocity of the target object, the depth coordinates of the target object's surface, and the optical flow of the image.

[0122] Let f(x,y,t) be the gray value at image point p(x,y) at time t. Then the gray value conservation equation for optical flow can be expressed as:

[0123] f x u+f y v+f t =0

[0124] Among them, f x f y f t These represent the gradients of the gray level of pixel p(x,y) in the camera's imaging plane along the x-axis, y-axis, and time t, respectively.

[0125] Based on the correspondence between the image optical flow, three-dimensional motion velocity, and object surface depth coordinates obtained in S4.1, we get:

[0126] a x ω x +a y ω y +a z ω z +b x T x '+b y T y '+b z T z '+f t =0

[0127] in:

[0128] a = [a x ,a y ,a z ] T =[-xyf x -(1+y 2 )f y ,(1+x 2 )f x +xyf y ,-yf x +xf y ] T

[0129] b = [b x ,b y ,b z ] T =[f x ,f y ,-xf x -yf y ] T

[0130]

[0131] The above equation is a conservation assumption regarding the three-dimensional motion velocity of pixels on the surface of a moving object in an image. The equation contains six unknown motion parameters [ω]. x ,ω y ,ω z ,T x ',T y ',T z As can be seen from the image, the three-dimensional motion velocity of the pixels on the surface of the moving object is related to the optical flow of the image.

[0132] The above formula can be simplified as follows:

[0133] a T ·ω+b T ·T'+f t =0

[0134] Design reasonable and smooth constraints:

[0135]

[0136] in, Let be the spatial gradient smoothing factor, expressed by the following formula:

[0137]

[0138]

[0139] Where ω represents the three-dimensional rotational velocity of the moving object, and T' represents the translational velocity of the moving object.

[0140] Finally, we obtain an energy function with an unknown parameter representing the three-dimensional motion velocity of the pixels on the surface of the moving target object:

[0141]

[0142] Where λ is the smoothing coefficient, the value of which must take into account the noise in the image. It adopts a functional form that varies with the image gray-level gradient:

[0143]

[0144] in, It is the gradient of pixels in the image of a moving object obtained by the camera. A, B, and σ are constants, and can be taken as A = B = 200 and σ = 2.

[0145] S4.3: Calculate the three-dimensional motion detection energy function to obtain the relative three-dimensional motion [ω] of the moving object's surface. x ,ω y ,ω z ,T x ',T y ',T z ').

[0146] The energy function ε is applied to the three-dimensional motion velocity [ω] x ,ω y ,ω z ,T x ',T y ',T z Taking the partial derivative, we get:

[0147]

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] That is, the Euler-Lagrange equation corresponding to the energy function ε. Let Let be the mean of the ω-neighborhood. Let T' be the mean of its neighborhood, and set... They are respectively:

[0154]

[0155]

[0156] Substituting the above expression into the equation above and simplifying, we get:

[0157]

[0158]

[0159]

[0160]

[0161]

[0162]

[0163] In the above formula, a = [a x ,a y ,a z ] T b = [b x ,b y ,b z ] T λ can be obtained from the image. The iterative equations are obtained by solving and rearranging the equations using the Gauss-Seidel iteration:

[0164]

[0165]

[0166]

[0167]

[0168]

[0169]

[0170] The relative three-dimensional motion [ω] of the surface of a moving object can be obtained using the above iterative formula. x ,ω y ,ω z ,T x ',T y ',T z ']. Also:

[0171]

[0172] Then we have, T = T'Z = (T x 'Z,T y 'Z,T z 'Z). Based on the previously calculated pixel translation speed T' = [T x ',T y ',T z '] T A relative translational velocity T(T) containing the pixel's depth coordinate Z can be obtained. x ,T y ,T z ).

[0173] S4.4: According to the translational velocity T'=[T x ',T y ',T z ']T Obtain the relative depth coordinate Z of the target object.

[0174] First set Then we have:

[0175]

[0176] The relative depth coordinates Z and relative three-dimensional coordinates of the moving target object are obtained by processing:

[0177]

[0178]

[0179]

[0180] Where k is a constant factor, and usually k is 1.

[0181] S5: The three-dimensional surface reconstruction result obtained by fusing image optical flow is corrected to the polarization three-dimensional reconstruction result obtained by using the target micro-surface element normal vector to obtain the final three-dimensional reconstruction result.

[0182] In this step, the gradients of the 3D surface function of the target object reconstructed in step S3 and the 3D surface reconstruction result reconstructed in step S4 are calculated to obtain their respective gradient information. The gradient information of the 3D surface based on image optical flow is then used to correct the 3D surface reconstruction result obtained using polarization to obtain the final 3D surface reconstruction result of the target object. The specific steps are as follows:

[0183] Based on the azimuth and incident angle (zenith angle) of the incident light on the object's surface, normalize the depth direction (i.e., the z-axis component) and construct the normal vector:

[0184]

[0185] Where, θ and These are the zenith angle and azimuth angle of the incident light on the target surface, respectively.

[0186] Correct the normal vector to eliminate the multivaluedness problem and the problem of some points being undesirable:

[0187] For the multivalued problem, the relationship between the gradient field of the reflected light intensity of the target and its contour is used to provide a reference for the value of the binary operator Λ. By converting the normal vector information into gradient field information, the correction process for the multivalued normal vector problem in a multi-target polarization 3D imaging scene can be expressed as:

[0188]

[0189] Among them, G NG represents the gradient field variation reference information obtained by solving the 3D surface reconstruction results using the image optical flow algorithm. polar This represents the gradient field parameters obtained by solving using the polarization information of the reflected light from the target. If This indicates that the estimated azimuth angle is accurate, meaning the micro-element normal vector information obtained directly from the polarization characteristics of the reflected light from the target is accurate; conversely, if The azimuth angle obtained using polarization characteristics needs to be corrected to achieve a 180° flip of the azimuth angle value. The azimuth angle correction for the target surface micro-element can be expressed by the following formula:

[0190]

[0191] in, This represents the azimuth information of the target micro-facets after correction. Ultimately, the normal vector information of the target surface micro-facets can be uniquely solved. The corrected gradient recovers the three-dimensional shape of the target, thus obtaining the dynamic target reconstruction based on the optical flow of the fused image.

[0192] Furthermore, for the problem of some points being undesirable, specifically the case where the tanθ parameter is introduced into the expression for the normal vector, causing tanθ to become infinite when θ is around 90°, an optimization algorithm is proposed, as follows:

[0193] make Let the input data be f(x), and let its three values ​​be T1, T0, and T1 respectively. * T0 approaches T * .

[0194] set up:

[0195] The output value is:

[0196] in,

[0197] For the above algorithm, the zenith angle θ is taken as the input data f(x), T1 is 85.5°, and T... * If T0 is 84.5°, then after optimization by the above algorithm, all zenith angles that were originally between 85° and 90° are reduced to 85°, while those near and below 85° remain unchanged. Thus, this algorithm effectively solves the problem of some points being undesirable.

[0198] This invention proposes a polarization-based 3D reconstruction method that fuses image optical flow. This method effectively eliminates the influence of dynamic scenes on the polarization-based 3D reconstruction results, comprehensively considers relevant issues in polarization-based 3D imaging technology, enriches the detail information of dynamic objects in the 3D reconstruction, and expands the application scope of 3D reconstruction technology. This invention corrects the reconstruction results of dynamic scenes obtained through polarization-based 3D imaging by fusing image optical flow, overcoming problems such as severe noise in the acquired images and blurred reconstruction normal azimuth angles encountered in dynamic scenes. This effectively improves the accuracy of polarization-based 3D imaging in dynamic scenes, broadening the application range of polarization-based 3D reconstruction methods. This invention fuses optical flow information for polarization-based 3D reconstruction of dynamic scenes, and uses the reconstruction results of image optical flow to correct the polarization-based 3D reconstruction results in dynamic scenes. This overcomes problems such as severe noise in the acquired images and blurred reconstruction normal azimuth angles encountered in dynamic scenes, improving the accuracy of polarization-based 3D reconstruction in such scenes.

[0199] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A dynamic scene polarization three-dimensional reconstruction method fusing image optical flow, characterized in that, include: S1: Build a polarization 3D reconstruction system to obtain dynamic scene polarization images of the object to be reconstructed at different angles; S2: Calculate the degree of polarization of the object's surface using the dynamic scene polarization images from different angles; S3: Reconstruct the three-dimensional surface function of the target object based on the mapping relationship between the three-dimensional contour information of the target surface micro-element and the target surface normal vector; S4: Use the image optical flow algorithm to reconstruct the dynamic scene and obtain the three-dimensional surface reconstruction result of the target object; S5: The 3D surface reconstruction result obtained by fusing image optical flow is corrected against the polarization 3D reconstruction result obtained using the target micro-surface element normal vector to obtain the final 3D reconstruction result. S4 includes: S4.1: Obtain the three-dimensional motion velocity of the target object and the correspondence between the depth coordinates of the target object's surface and the optical flow of the image; S4.2: obtaining a three-dimensional motion detection energy function according to the corresponding relationship between the three-dimensional motion speed of the target object, the surface depth coordinates of the target object and the image optical flow ; S4.3: calculating the three-dimensional motion detection energy function to obtain the relative three-dimensional motion of the surface of the moving object , , , ], denotes the three-dimensional rotational velocity of the moving object in the three-dimensional space; S4.4: According to the translation speed T’ [ , , ] T Obtaining relative depth coordinates of the target object Z .

2. The dynamic scene polarization 3D reconstruction method fusing image light flow according to claim 1, characterized in that, S3 includes: S3.1: Utilizing polarized images of different angles I 0、 I 45 、 I 90 and I 135 azimuth angle is calculated from the Stokes vector φ ; S3.2: Based on the degree of polarization P and the zenith angle θ The zenith angle is obtained by calculating the relationship. θ ; S3.3: Based on the mapping relationship between the three-dimensional contour information of the target surface micro-element and the target surface normal vector, reconstruct the target three-dimensional surface function. z ( u ).

3. The dynamic scene polarization 3D reconstruction method based on fused image optical flow according to claim 2, characterized in that, S3.3 includes: S3.31: Construct the target micro-element normal vector using the azimuth angle and the zenith angle: ， in, , , They represent the target surface normal vectors respectively. exist x axis, y shaft and z Components in the axial direction, u =( x , y ) represents the pixel on the image corresponding to the target surface; S3.32: Obtain the correspondence between the target micro-element normal vector and the target surface function: ， in, , respectively represent the normal vectors of the micro-element surfaces of the target object at... x and y Gradient field in the direction, z ( u ) represents the surface of the target object. u The height at the point, express x Partial derivative in direction, express y Partial derivative in direction; S3.33: By using Fourier orthogonal basis decomposition, the discrete gradient field information and surface function information in the spatial domain are linked in the Fourier domain to reconstruct the surface function of the target object. z ( u ).

4. The dynamic scene polarization 3D reconstruction method based on fused image optical flow according to claim 3, characterized in that, S4.1 includes: S4.11: Assume point P is a point on the surface of a moving object in three-dimensional space, and the three-dimensional coordinates of point P are ( X , Y , Z ), p Point P is the corresponding projection point of point P onto the camera's imaging plane. Assume... p The coordinates of the point are ( x , y If ), then it exists: The camera focal length is f ; S4.12: Settings f =1, thus obtaining the three-dimensional motion equation of point P: in,( X’ , Y’ , Z’ () represents the velocity of point P. This represents the three-dimensional rotational velocity of a moving object in three-dimensional space. It represents the translational velocity of a moving object in three-dimensional space; S4.13: Assume the projection plane p The optical flow at point ( u , v If ), then it exists: in, express p The speed of movement of the point; S4.14: Obtain the correspondence between image optical flow, 3D motion velocity, and object surface depth coordinates: ， in, Z Represents the surface depth coordinates of the target object.

5. The dynamic scene polarization 3D reconstruction method based on fused image optical flow according to claim 4, characterized in that, The energy function is: Where λ is the coefficient of the smoothing term. It is the spatial gradient smoothing factor. ， in, f x , f y , f t These represent the pixels in the camera's imaging plane. p ( x , y grayscale along x axis, y Axis and Time t Gradient of direction.

6. The dynamic scene polarization 3D reconstruction method based on fused image optical flow according to claim 4, characterized in that, S5 includes: The gradients of the three-dimensional surface function of the target object reconstructed in step S3 and the three-dimensional surface reconstruction result reconstructed in step S4 are calculated respectively to obtain their respective gradient information. The gradient information of the three-dimensional surface based on image optical flow is used to correct the three-dimensional surface reconstruction result obtained by polarization to obtain the final three-dimensional surface reconstruction result of the target object.

Images