Stereo matching method based on polarization vector similarity

Abstract

The application provides a stereomatching method based on polarization vector similarity, comprising obtaining camera parameters, performing epipolar rectification on two image sets respectively by using the camera parameters to obtain a first rectified image set and a second rectified image set. Two types of polarization information corresponding to the two rectified image sets are calculated, two zenith angles are inversely calculated, and polarization vector fields on two sides of a target to be reconstructed are calculated. The polarization vector similarity of each matching window in the two rectified image sets is calculated, a disparity map is calculated according to the maximum value of the polarization vector similarity, and a depth map of the target to be reconstructed is calculated by using the disparity map. The application estimates disparity based on polarization vector similarity, improves the polarization information to the vector dimension, constructs polarization vector similarity with geometric constraint capability, significantly enhances the distinguishing ability in a complex scene, and improves the stereomatching precision and robustness of a textureless target.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computational imaging technology, and in particular to a stereo matching method based on polarization vector similarity. Background Technology

[0002] Stereo vision technology can acquire absolute 3D depth information of a scene under passive conditions. This unique advantage has made it one of the most widely used technologies in the field of 3D measurement. Stereo matching is the core of stereo vision, and its key lies in accurately determining the disparity of the same spatial feature point between stereo image pairs taken from different viewpoints. The reliability of disparity estimation directly determines the accuracy and completeness of the final 3D reconstruction information. Therefore, stereo matching has long been an important research direction in the field of stereo vision.

[0003] Based on different matching strategies, traditional stereo matching algorithms are mainly divided into two categories. The first category is global methods, which achieve disparity estimation by constructing and minimizing an energy function containing data terms and smoothing terms. Typical algorithms include Belief Propagation (BP), Dynamic Programming (DP), and Graph Cuts (GC). These methods can jointly consider data consistency and disparity smoothing constraints globally, usually obtaining disparity results with good continuity, but their computational complexity is high, making them difficult to meet the needs of real-time applications. The second category is local methods, which complete the matching by comparing grayscale or gradient information within a local window, such as the squared difference, absolute difference, normalized cross-correlation, sum of squared differences, and sum of absolute differences based on a single pixel. These methods are computationally efficient and simple to implement, and have been widely used in real-time stereo vision tasks.

[0004] However, both global and local methods essentially rely on grayscale variations and texture information in the image for their matching process. Therefore, in practical applications, targets with no or weak texture remain a core challenge in stereo matching. When the target surface lacks significant grayscale variations or feature points, the intensity differences between different pixels are extremely small, making it difficult for traditional methods to establish reliable correspondences, easily leading to incorrect disparity estimation or large-area matching holes. In such scenarios, the matching performance of both global and local methods degrades significantly.

[0005] To address the challenge of matching textureless targets, researchers have continuously explored and optimized matching methods to improve reliability. Some studies employ active projection methods, artificially adding matchable features to the image by projecting structured light or coded patterns onto the target surface, thereby improving the stability of parallax search. These methods typically require additional projection equipment and are susceptible to ambient lighting and measurement distance. Meanwhile, because intensity information lacks spatial discriminability in textureless regions, researchers have begun to focus on higher-dimensional physical information neglected by traditional methods, hoping to obtain more stable matching criteria from new observational dimensions. The polarization characteristics of target surfaces have attracted increasing attention. Polarization stereo imaging systems, by introducing polarization information into binocular systems, provide an additional observational dimension beyond traditional intensity imaging. Unlike intensity information, polarization information is closely related to the geometric features of the object's surface; even regions that appear uniformly textureless in intensity images can exhibit discriminative spatial variations in the polarization domain. Based on this characteristic, existing research has attempted to introduce polarization information to enhance the robustness of matching, utilizing the polarization characteristics of light reflected from the target surface to provide additional discriminative cues. However, most existing methods only use polarization degree or polarization angle as auxiliary features, which essentially still belong to similarity comparison based on scalar information. Although they improve the matching effect to some extent, the expressive power of scalar form is limited, and they still face the problem of insufficient matching reliability in complex or textureless scenes.

[0006] Therefore, how to fully utilize the richer structural features in polarization information to improve the stereo matching accuracy and robustness of textureless targets without active projection remains a pressing technical problem in the field of stereo vision. Summary of the Invention

[0007] To overcome the problems existing in related technologies, the purpose of this invention is to provide a stereo matching method based on polarization vector similarity, which can make full use of the richer structural features in polarization information without active projection, thereby improving the stereo matching accuracy and robustness of textureless targets.

[0008] Stereo matching methods based on polarization vector similarity include: A camera is used to take pictures at different angles on both sides of the target to be reconstructed, resulting in a first image set and a second image set; the two image sets contain the same number of polariton images. Obtain camera parameters, and use the camera parameters to perform epipolar correction on two image sets respectively to obtain a first corrected image set and a second corrected image set; Calculate the first polarization information of the first corrected image set and the second polarization information of the second corrected image set; The first type of zenith angle is derived from the first polarization information, and the second type of zenith angle is derived from the second polarization information. Based on the two types of polarization information and the zenith angle of the corresponding category, the polarization vector fields on both sides of the target to be reconstructed are calculated; Based on the polarization vector fields on both sides of the target to be reconstructed, calculate the polarization vector similarity of each matching window in the two corrected image sets; A disparity map is calculated based on the maximum value of the polarization vector similarity, and the depth map of the target to be reconstructed is calculated using the disparity map.

[0009] In a preferred embodiment of the present invention, calculating the first polarization information of the first corrected image set includes: The polarization angle and degree of polarization of the first corrected image set are calculated using the following formula: ; ; in, The polarization angle of the first corrected image set. Let be the polarization degree of the first corrected image set, and arctan be the arctangent function. This is the polariton image of linearly polarized light in the 0° direction from the first image set. This is a polariton image of linearly polarized light at a 45° angle from the first image set. This is a polariton image of linearly polarized light in the 90° direction from the first image set. The polariton image of linearly polarized light in the 135° direction for the first image set.

[0010] In a preferred embodiment of the present invention, the step of retrieving the first type of zenith angle based on the first polarization information includes: Establish a nonlinear model of the polarization degree and the first type zenith angle of the first corrected image set: ; in, n The refractive index of the target to be reconstructed. This is the first type of zenith angle.

[0011] In a preferred embodiment of the present invention, calculating the polarization vector fields on both sides of the target to be reconstructed based on two types of polarization information and the corresponding zenith angles includes: The polarization vector field on the first side of the target to be reconstructed is calculated using the following formula: ; in, The polarization vector field of the first side of the target to be reconstructed. Here, T represents the first type of azimuth angle, and T is the matrix transpose operation. The polarization vector field on the second side of the target to be reconstructed is calculated using the following formula: ; in, The polarization vector field of the second side of the target to be reconstructed. This is a second-type zenith angle. This is the second type of azimuth. and Each includes multiple polarization vectors.

[0012] In a preferred embodiment of the present invention, the polarization vector similarity of each matching window in the two corrected image sets is calculated based on the polarization vector fields on both sides of the target to be reconstructed, including: The polarization vector similarity of each matching window is calculated using the following formula: ; in, For the first image set, the... i The first window v The polarization vector of each pixel. For the second image set j The first window v The polarization vector of one pixel; || represents the vector modulo operation. W To match the window, For the first image set, the... i The first window and the second image set j The similarity of polarization vectors of each window.

[0013] In a preferred embodiment of the present invention, the step of calculating a disparity map based on the maximum value of the polarization vector similarity, and using the disparity map to calculate the depth map of the target to be reconstructed, includes: The first image set is calculated based on the maximum value of the polarization vector similarity. i The parallax of each window; where the first image set is... i The disparity of the nth window is the first image set. i The window is shifted to the matching second image set. j The number of grid points in each window; The disparity of all windows in the first image set is used to form a disparity map; the center pixel of each window is the pixel in the polarization sub-image of the first image set; Multiply the focal length by the baseline length to obtain the depth parallax scaling factor; camera parameters include the focal length and the baseline length; Divide the depth disparity scaling factor by the disparity map to obtain the depth map.

[0014] In a preferred embodiment of the present invention, the step of performing epipolar correction on two image sets using the camera parameters to obtain a first corrected image set and a second corrected image set includes: Construct the left rotation matrix and the right rotation matrix based on the rotation matrix and translation vector of the camera parameters, respectively; The first image set is remapped using the left rotation matrix and the intrinsic parameter matrix to obtain the first corrected image set. The second image set is remapped using the right rotation matrix and the intrinsic parameter matrix to obtain the second corrected image set.

[0015] In a preferred embodiment of the present invention, before obtaining the camera parameters, the method further includes: Multiple target images were obtained by photographing the checkerboard calibration board from multiple angles using a camera. The camera is calibrated using all the target images to obtain camera parameters; the camera parameters include intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficients.

[0016] In a preferred embodiment of the present invention, the step of taking pictures at different angles on both sides of the target to be reconstructed using a camera to obtain a first image set and a second image set includes: A camera is used to capture images of the target to be reconstructed at four angles: 0°, 45°, 90°, and 135° on the first side of the target, resulting in a first image set; the first image set contains four polarization images. A camera is used to capture images of the target to be reconstructed at four angles: 0°, 45°, 90°, and 135° on the second side of the target, resulting in a second image set; the second image set contains four polariton images.

[0017] In a preferred embodiment of the present invention, the step of calculating the first image set based on the maximum value of the polarization vector similarity is... i The parallax of each window includes: Fix the sequence number of the first image collection window i Iterate through the indexes of the second image collection window. j The optimal sequence number of the second image set window when the polarization vector similarity reaches its maximum value. j max ; The optimal sequence number of the second image set window j max Subtract the index of the first image collection window. i The first image set is obtained. i Parallax of each window.

[0018] The beneficial effects of this invention are as follows: The stereo matching method based on polarization vector similarity provided by this invention includes taking pictures at different angles on both sides of the target to be reconstructed using a camera to obtain polariton images at different polarization angles, resulting in a first image set and a second image set. Camera parameters are obtained, and epipolar correction is performed on the two image sets using these parameters, resulting in a first corrected image set and a second corrected image set. The ordinate of the same pixel in both the first and second corrected image sets is consistent, reducing the stereo matching problem from a two-dimensional search to a one-dimensional search along a horizontal scan line, thus significantly reducing the computational complexity of matching and providing standardized image pairs as input for dense disparity estimation based on polarization information. The first polarization information of the first corrected image set and the second polarization information of the second corrected image set are calculated. Even in regions where the intensity image appears uniform and textureless, rich spatial variation features are preserved in the polarization domain, allowing previously unmatchable regions to obtain highly recognizable matching primitives, effectively avoiding mismatches and matching gaps. The first type of zenith angle is inverted based on the first polarization information, and the second type of zenith angle is inverted based on the second polarization information. The zenith angle determines the tilt of the polarization vector, while the polarization angle determines its orientation. Specifically, the zenith angle provides the elevation and / or tilt angle of the polarization vector, and the polarization angle provides the angle of rotation of the polarization vector around the line of sight. Based on these two types of polarization information and the corresponding zenith angle, the polarization vector fields on both sides of the target to be reconstructed are calculated. These polarization vector fields are dense polarization vector maps corresponding to the views from both sides, laying the foundation for stereo matching and 3D shape reconstruction based on polarization vectors. Based on the polarization vector fields on both sides of the target to be reconstructed, the polarization vector similarity of each matching window in the two corrected image sets is calculated. A disparity map is calculated based on the target matching window corresponding to the maximum value of the polarization vector similarity, and the depth map of the target to be reconstructed is calculated using the disparity map. This invention estimates disparity based on polarization vector similarity, elevating polarization information to the vector dimension and constructing a polarization vector similarity with geometric constraints. This significantly enhances the discrimination ability in complex scenes and improves the stereo matching accuracy and robustness of textureless targets. Attached Figure Description

[0019] Figure 1 This is a flowchart of the stereo matching method based on polarization vector similarity of the present invention; Figure 2 This is a schematic diagram of the polarization angle of the present invention; Figure 3 This is a schematic diagram of the degree of polarization of the present invention; Figure 4 This is a schematic diagram of polarization vector similarity matching according to the present invention; Figure 5 This is a schematic diagram of the results of polarization stereoscopic imaging in a real-world scenario according to the present invention. Detailed Implementation

[0020] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0021] Example 1 like Figure 1 As shown, this embodiment provides a stereo matching method based on polarization vector similarity, including: S1: Take pictures at different angles on both sides of the target to be reconstructed using a camera to obtain a first image set and a second image set; the two image sets contain the same number of polariton images; S2: Obtain camera parameters, and use the camera parameters to perform epipolar correction on the two image sets respectively to obtain the first corrected image set and the second corrected image set; S3: Calculate the first polarization information of the first corrected image set and the second polarization information of the second corrected image set; S4: Retrieve the first type of zenith angle based on the first polarization information, and retrieve the second type of zenith angle based on the second polarization information; S5: Calculate the polarization vector fields on both sides of the target to be reconstructed based on the two polarization information and the zenith angle of the corresponding category; S6: Based on the polarization vector fields on both sides of the target to be reconstructed, calculate the polarization vector similarity of each matching window in the two corrected image sets; S7: Calculate the disparity map based on the maximum value of the polarization vector similarity, and use the disparity map to calculate the depth map of the target to be reconstructed.

[0022] By taking pictures at different angles on both sides of the target to be reconstructed using a camera, a first image set and a second image set are obtained, including: S11: Using a camera, the target to be reconstructed is photographed at four angles of 0°, 45°, 90° and 135° on the first side to obtain a first image set; the first image set contains four polarization images; S12: Using a camera, the target to be reconstructed is photographed at four angles of 0°, 45°, 90° and 135° on the second side of the target to be reconstructed, to obtain a second image set; the second image set contains four polariton images.

[0023] The camera used in this invention is a focal plane polarization camera. In a single acquisition, it obtains polariton images from four different directions at two different viewpoints (left and right). Specifically, it captures images of the target to be reconstructed from four angles: 0°, 45°, 90°, and 135°, on both sides determined by an external reference point along the length of the target, resulting in a total of eight polariton images. The four polariton images from the first side (left) of the target are combined into a first image set, and the four polariton images from the second side (right) are combined into a second image set. The focal plane polarization camera has a resolution of 2448*2048, two lenses with the same focal length (f=50mm), and is a binocular camera with a baseline distance of b=50mm.

[0024] The target to be reconstructed is smooth plastic, porcelain, or a billiard ball, excluding transparent / semi-transparent and highly reflective objects, and characterized by a smooth, featureless material. The first polarization information of the first corrected image set is calculated, including: The polarization angle and degree of polarization of the first corrected image set are calculated using the following formula: (1) (2) in, The polarization angle of the first corrected image set. Let be the polarization degree of the first corrected image set, and arctan be the arctangent function. This is the polariton image of linearly polarized light in the 0° direction from the first image set. This is a polariton image of linearly polarized light at a 45° angle from the first image set. This is a polariton image of linearly polarized light in the 90° direction from the first image set. The polariton image of linearly polarized light in the 135° direction for the first image set.

[0025] The polarization angle characterizes the azimuth angle, the direction of the principal axis of birefringence, and the absolute orientation, while the degree of polarization characterizes the surface tilt, material, and type of scattering event. Figure 2 This is a schematic diagram of the polarization angle of the present invention, which represents the polarization vibration direction of the light reflected from the surface of the target to be reconstructed, and is directly related to the azimuth angle of the normal to the surface of the target to be reconstructed. Figure 3 This is a schematic diagram of the degree of polarization of the present invention, reflecting the degree of polarization of the reflected light from the surface of the target to be reconstructed. A definite nonlinear relationship is established between this polarization and the zenith angle of the normal to the surface of the target to be reconstructed, using a Fresnel reflection model. The polarization angle and degree of polarization together constitute the physical feature field, which is closely related to the geometry of the target surface and is resolved from the polariton image. This field forms the basis for calculating the polarization vector field.

[0026] Similarly, by replacing the polariton image in the first corrected image set with the polariton image in the second corrected image set in formulas (1)-(2), the polarization angle and polarization degree of the second corrected image set can be calculated. The polarization angle calculated by this invention is the set of polarization angles of all pixels in the polariton image, and the polarization degree is the set of polarization degrees of all pixels in the polariton image. The polarization angle and polarization degree of this invention are both from an image, and the size is the same as the size of the polariton image.

[0027] The first kind of zenith angle is derived from the first polarization information, including: Establish a nonlinear model of the polarization degree and the first type zenith angle of the first corrected image set: (3) in, n The refractive index of the target to be reconstructed. This refers to the first type of zenith angle. According to Fresnel reflection theory, there is a definite nonlinear functional relationship between the degree of polarization and the zenith angle of the surface polarization vector in the case of diffuse reflection. In this invention, the polarization vector is the normal vector of the target surface point to be reconstructed, and the polarization vector is used to characterize the orientation of the target surface point to be reconstructed in three-dimensional space.

[0028] In formula (3), the refractive index n The polarization degree of the first corrected image set is known, and the first-kind zenith angle can be inverted by looking up a table or by numerical calculation. .

[0029] Similarly, by replacing the polarization degree of the first corrected image set in formula (3) with the polarization degree of the second corrected image set, the second type zenith angle can be derived. The first type of zenith angle includes the zenith angles of all pixels in the first corrected image set, and the second type of zenith angle includes the zenith angles of all pixels in the second corrected image set.

[0030] The polarization degree and first type zenith angle of the first corrected image set reflect the polarization characteristics of the first side of the target to be reconstructed, i.e., the left side of the selected reference object. The polarization degree and second type zenith angle of the second corrected image set reflect the polarization characteristics of the second side of the target to be reconstructed, i.e., the right side of the selected reference object.

[0031] Based on the two types of polarization information and the corresponding zenith angles, the polarization vector fields on both sides of the target to be reconstructed are calculated, including: The polarization vector field on the first side of the target to be reconstructed is calculated using the following formula: (4) in, The polarization vector field of the first side of the target to be reconstructed. Here, T represents the first type of azimuth angle, and T is the matrix transpose operation. The polarization vector field on the second side of the target to be reconstructed is calculated using the following formula: (5) in, The polarization vector field of the second side of the target to be reconstructed. This is a second-type zenith angle. This is the second type of azimuth. It contains the polarization vectors of all pixels in the four polarizer images of the first corrected image set. It contains the polarization vectors of all pixels in the four polariton images of the second corrected image set.

[0032] In diffuse reflection, the azimuth angle and polarization angle of the target surface to be reconstructed have the following relationship: (6) in, Let be the azimuth angle of the target surface to be reconstructed. The azimuth angles of the pixels on the first and second sides of the target surface to be reconstructed can be calculated according to formula (6).

[0033] Steps S1-S6 of this invention are stereo matching based on polarization vector similarity. Step S7 is an application after stereo matching, namely, calculating the disparity map by the maximum value of polarization vector similarity, and using the disparity map to calculate the depth map of the target to be reconstructed, so as to make full use of the higher-dimensional structured constraints in polarization information and improve the accuracy of three-dimensional reconstruction of the target to be reconstructed in textureless or low-texture scenes.

[0034] Existing methods for incorporating polarization information only treat polarization degree or polarization angle as independent scalar features to assist matching, resulting in limited information representation capabilities. This invention, for the first time, elevates polarization information to the vector dimension, constructing a polarization vector similarity with geometric constraints, significantly enhancing the ability to distinguish between different scenarios.

[0035] This invention employs a completely passive polarization imaging method, which eliminates the need to project structured light or coded patterns onto the target surface. It relies entirely on the reflective polarization characteristics of the target itself, avoiding the additional equipment required by active projection methods and reducing the sensitivity of active projection methods to ambient light and measurement distance. While maintaining the system's simplicity, low cost, and ease of operation, it is suitable for three-dimensional imaging tasks of textureless targets at medium to long distances, in large-scale outdoor scenes.

[0036] Example 2 This embodiment provides a stereo matching method based on polarization vector similarity. This embodiment further describes the scheme of Embodiment 1 based on Embodiment 1.

[0037] The step of calculating the polarization vector fields on both sides of the target to be reconstructed based on two types of polarization information and the corresponding zenith angles includes: The polarization vector field on the first side of the target to be reconstructed is calculated using the following formula: (7) in, The polarization vector field of the first side of the target to be reconstructed. Here, T represents the first type of azimuth angle, and T is the matrix transpose operation. The polarization vector field on the second side of the target to be reconstructed is calculated using the following formula: (8) in, The polarization vector field of the second side of the target to be reconstructed. This is a second-type zenith angle. This is the second type of azimuth. and Each includes multiple polarization vectors.

[0038] This is the set of polarization vectors of the target surface to be reconstructed, taken from the left side of the reference object. This polarization vector field represents the orientation of the pixels on the target surface to be reconstructed in three-dimensional space when taken from the left side. The polarization vector field is a set of polarization vectors on the surface of the target to be reconstructed, which is the second side of the target, i.e., the right side of the reference object. This polarization vector field represents the orientation of the pixels on the surface of the target to be reconstructed in three-dimensional space when photographed from the right side. The polarization vector field is a vector field.

[0039] equal That is, polarization vector field N1 exist x The component along the axial direction represents... x The rate of change of the gradient in the direction; equal That is, polarization vector field N1 exist y The component along the axial direction represents... y The rate of change of the gradient in the direction; equal That is, polarization vector field N1 exist z The component along the axial direction represents... z The rate of change of the gradient in the direction.

[0040] Similarly, the polarization vector field can be calculated. N2 exist x axis, y shaft and zThe components along the axial direction. From this, dense polarization vector diagrams corresponding to the left and right viewpoints can be obtained, laying the foundation for stereo matching and 3D topography reconstruction based on polarization vectors.

[0041] This invention no longer relies on the easily invalidated grayscale information in traditional methods. Instead, it constructs a structured polarization feature tensor that includes zenith and azimuth angles, thereby obtaining a polarization vector field containing multiple polarization vectors. Even in uniform, textureless regions, the polarization domain retains rich spatial variation features, enabling previously unmatchable regions to acquire highly recognizable matching primitives, effectively avoiding mismatches and matching gaps.

[0042] The step of calculating the polarization vector similarity of each matching window in the two corrected image sets based on the polarization vector fields on both sides of the target to be reconstructed includes: The polarization vector similarity of each matching window is calculated using the following formula: (9) in, For the first image set, the... i The first window v The polarization vector of each pixel. For the second image set j The first window v The polarization vector of one pixel; || represents the vector modulo operation. W To match the window, For the first image set, the... i The first window and the second image set j The polarization vector similarity of the first window. The summation symbol in formula (9) represents the first image set. i The first window, and the second image set j The sum of the cosine similarity of the polarization vectors of all pixels within a window, that is, the sum of the cosine similarity of the polarization vectors of all pixels within two windows, is equal to the polarization vector similarity of the two windows.

[0043] To address the insufficient strength matching discrimination capability of traditional methods in textureless scenes, formula (9) is used to calculate the polarization vector similarity. The four polarization sub-images in the first corrected image set are located in the first coordinate system, and the same position, for example, row 1, column 1, corresponds to the same position on the first side of the target to be reconstructed. The four polarization sub-images in the second corrected image set are located in the second coordinate system, and the same position, for example, row 1, column 1, corresponds to the same position on the second side of the target to be reconstructed. The first image set... i The first window v polarization vector of each pixel Represented as: (10) in, The polarizer image in the first corrected image set is the first one. i The first window v The zenith angle of one pixel, The polarizer image in the first corrected image set is the first one. i The first window v The azimuth angle of each pixel.

[0044] Second image set j The first window v polarization vector of each pixel Represented as: (11) in, For the polarizer image in the second corrected image set, j The first window v The zenith angle of one pixel, For the polarizer image in the second corrected image set, j The first window v The azimuth angle of each pixel.

[0045] like Figure 4 As shown, j = i + d That is, the polariton image in the first corrected image set. i The window was moved. d The polariton image in the second corrected image set j The windows overlap. After epipolar correction, for Figure 4 A specific matching window on the left (by number) i (Represented by...) Each pixel within the window corresponds to a three-dimensional polarization vector calculated from the zenith angle and azimuth angle. These polarization vectors are visualized on a three-dimensional coordinate axis, visually representing their spatial orientation. During matching, the set of polarization vectors of this matching window is compared with a series of candidate windows (numbered sequentially) sliding along the horizontal polar line in the right figure. j The similarity is evaluated by calculating the sum of the dot products of the polarization vectors of all corresponding pixels within the window.

[0046] Relative displacement between the two windows d The value of is greater than or equal to 0 and less than or equal to the width of the polarizer image. The width of the polarizer image in the first corrected image set is the same as the width of the polarizer image in the second corrected image set, and the height of the polarizer image in the first corrected image set is the same as the height of the polarizer image in the second corrected image set.

[0047] A disparity map is calculated based on the maximum value of the polarization vector similarity, and a depth map of the target to be reconstructed is calculated using the disparity map, including: S71: Calculate the first image set based on the maximum value of the polarization vector similarity. i The parallax of each window; where the first image set is... i The disparity of the nth window is the first image set. i The window is shifted to the matching second image set. j The number of grid points in each window; S72: Combine the disparities of all windows in the first image set to form a disparity map; the center pixel of each window is a pixel in the polarization image of the first image set; S73: Multiply the focal length by the baseline length to obtain the depth parallax scaling factor; camera parameters include the focal length and the baseline length; S74: Divide the depth disparity scaling factor by the disparity map to obtain the depth map.

[0048] The first image set is calculated based on the maximum value of the polarization vector similarity. i The parallax of each window includes: S711: Fix the sequence number of the first image set window i Iterate through the indexes of the second image collection window. j The optimal sequence number of the second image set window when the polarization vector similarity reaches its maximum value. j max ; S712: Set the optimal sequence number of the second image set window. jmax Subtract the index of the first image collection window. i The first image set is obtained. i Parallax of each window.

[0049] (12) Use formula (12) to search when When the maximum value is obtained i and jmax ,in i fixed, jmax To and i The second image set window with the highest matching degree. At this point... jmax minus i The difference is used as the first image set. i Parallax of each window.

[0050] A window in the first image set corresponds to a pixel in the polariton image. The disparities of all windows in the first image set, i.e., all pixels in the polariton images of the first image set, are combined to form a disparity map. The disparity map and the polariton image have the same size.

[0051] Calculate depth map based on disparity map: (13) in, D For depth map, f Focal length b Baseline length disp The disparity map, depth map, and disparity map are the same size. After camera calibration, all pixels have the same focal length and baseline length.

[0052] This invention directly embeds structured polarization vectors into the 3D reconstruction matching framework to replace the traditional matching cost calculation method. It does not require disruptive modifications to the hardware system and algorithm architecture, and achieves deep integration of polarization information and the matching process. While improving the robustness of 3D reconstruction of textureless targets, it is compatible with mainstream matching strategies such as local and global matching strategies, and has good engineering promotion value.

[0053] Example 3 This embodiment provides a stereo matching method based on polarization vector similarity. This embodiment further describes the scheme of Embodiment 1 based on Embodiment 1.

[0054] Before obtaining the camera parameters, the process also includes: S11': Use a camera to photograph the checkerboard calibration board from multiple angles to obtain multiple target images; S12': The camera is calibrated using all the target images to obtain camera parameters; the camera parameters include intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficients.

[0055] This embodiment adopts the Zhang Zhengyou calibration method based on multiple target images. First, the corner points in each target image are extracted and the corner point coordinates are recorded. The calibration steps are as follows: (1) For each target image, the homography matrix under the viewpoint is calculated using the corner point coordinates and its corresponding checkerboard space coordinates. (2) Using the orthogonality of the rotation matrix, a linear equation system is constructed from the homography matrices of multiple viewpoints. Solving this equation system can yield the initial value of the camera's intrinsic parameter matrix. (3) After obtaining the initial value of the intrinsic parameter matrix, for each target image, the rotation matrix and translation vector under the viewpoint are inverted through the initial values ​​of the homography matrix and the intrinsic parameter matrix to obtain the initial value of the extrinsic parameter matrix. (4) The parameters calculated in steps (1)-(3) are used as initial values. All parameters are optimized by maximum likelihood estimation. The optimization goal is to minimize the reprojection error, that is, to minimize the sum of squared errors between the pixel coordinates of all three-dimensional corner points projected back to the image plane and the actual detected corner point coordinates, thus obtaining the intrinsic parameter matrix and the extrinsic parameter matrix. (5) Estimate the distortion coefficients, which include radial distortion coefficients and tangential distortion coefficients.

[0056] The step of performing epipolar correction on two image sets using the camera parameters to obtain a first corrected image set and a second corrected image set includes: S22: Construct the left rotation matrix and the right rotation matrix based on the rotation matrix and translation vector of the camera parameters, respectively; S23: The first image set is remapped using the left rotation matrix and the intrinsic parameter matrix to obtain the first corrected image set; S24: The second image set is remapped using the right rotation matrix and the intrinsic parameter matrix to obtain the second corrected image set.

[0057] Before step S22, there is also step S21: obtaining camera parameters.

[0058] In binocular stereo vision, the optical axes of the left and right cameras are not parallel. Therefore, epipolar correction is an important processing step. The first image set from the left side of the reference object and the second image set from the right side of the reference object are remapped so that the polariton image rows in the two image sets are aligned, that is, the vertical coordinates of corresponding pixels are the same. This reduces the search space of stereo matching from two dimensions to one dimension, which greatly improves the computational efficiency and accuracy.

[0059] The epipolar correction step employs classic algorithms such as Bouguet's. Its core is to calculate a correction rotation matrix—a left rotation matrix and a right rotation matrix—for the first image set (left view) and the second image set (right view), respectively. These two rotation matrices are derived from the calibration parameters of the stereo system (including the rotation matrix R and translation vector t between the left and right cameras). The purpose is to reproject the image planes of the left and right cameras onto a common virtual plane parallel to the line connecting their optical centers (baseline). Subsequently, the first image set (left view) is remapped using the left rotation matrix and the camera's intrinsic parameters, and the second image set (right view) is remapped using the right rotation matrix and the camera's intrinsic parameters, resulting in a first and second corrected image set with strictly aligned rows. This process restricts the search of corresponding pixels to the same horizontal line, achieving a dimensionality reduction of the search space from two dimensions to one dimension.

[0060] Example 4 Figure 5 This is a schematic diagram of the results of polarization stereoscopic imaging in a real-world scenario according to the present invention. Figure 5 (a) is a diagram of the experimental setup. Figure 5 (b) is the target image. Figure 5 (c) is a polarization three-dimensional imaging result. Figure 5 (d) shows the polarization stereo vision imaging result. Figure 5 (e) is a 3D tilted visualization. Figure 5 (f) is a 3D frontal visualization.

[0061] Figure 5 (a) shows the experimental setup, with a smooth, textureless target object to be reconstructed on the left and a binocular focal plane polarization camera system for data acquisition on the right. The scale placed below provides an absolute reference for the subsequent reconstruction results.

[0062] Figure 5 (b) shows the original light intensity image (I0) captured by the system. Figure 5 (c) and Figure 5 (d) The results of the three-dimensional reconstruction obtained by the traditional monocular polarization three-dimensional reconstruction method and the method proposed in this patent are shown respectively, which intuitively demonstrate the significant advantages of the present invention in restoring the complete and coherent three-dimensional shape of the object.

[0063] Figure 5 (e) and Figure 5 (f) further converts the matched disparity map into depth information with absolute scale, and visualizes it as a depth map in the form of a 3D point cloud; where Figure 5 (e) is a tilted viewpoint, which facilitates observation of the three-dimensional morphology. Figure 5(f) is the frontal view. Combined with the scale, the accuracy of the reconstruction depth on the absolute scale can be clearly verified, which fully confirms that the method of the present invention can achieve effective reconstruction from polariton images to high-precision and absolute-scale three-dimensional models.

[0064] This embodiment provides an electronic device including a processor, a memory, a network interface, and a database connected via a system bus. The processor in this electronic device provides computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operating devices, computer programs, and a database. When the computer program is executed by the processor, it implements the stereo matching methods based on polarization vector similarity described in Embodiments 1-3.

[0065] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0066] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A stereo matching method based on polarization vector similarity, characterized in that, include: A camera is used to take pictures from both sides of the target to be reconstructed at different angles to obtain a first image set and a second image set; The two image sets contain the same number of polariton images; Obtain camera parameters, and use the camera parameters to perform epipolar correction on two image sets respectively to obtain a first corrected image set and a second corrected image set; Calculate the first polarization information of the first corrected image set and the second polarization information of the second corrected image set; The first type of zenith angle is derived from the first polarization information, and the second type of zenith angle is derived from the second polarization information. Based on the two types of polarization information and the zenith angle of the corresponding category, the polarization vector fields on both sides of the target to be reconstructed are calculated; Based on the polarization vector fields on both sides of the target to be reconstructed, calculate the polarization vector similarity of each matching window in the two corrected image sets; A disparity map is calculated based on the maximum value of the polarization vector similarity, and the depth map of the target to be reconstructed is calculated using the disparity map.

2. The stereo matching method based on polarization vector similarity according to claim 1, characterized in that, The calculation of the first polarization information of the first corrected image set includes: The polarization angle and degree of polarization of the first corrected image set are calculated using the following formula: ； ； in, The polarization angle of the first corrected image set. Let be the polarization degree of the first corrected image set, and arctan be the arctangent function. This is the polariton image of linearly polarized light in the 0° direction from the first image set. This is a polariton image of linearly polarized light at a 45° angle from the first image set. This is a polariton image of linearly polarized light in the 90° direction from the first image set. The polariton image of linearly polarized light in the 135° direction for the first image set.

3. The stereo matching method based on polarization vector similarity according to claim 2, characterized in that, The step of retrieving the first type of zenith angle based on the first polarization information includes: Establish a nonlinear model of the polarization degree and the first type zenith angle of the first corrected image set: ； in, n The refractive index of the target to be reconstructed. This is the first type of zenith angle.

4. The stereo matching method based on polarization vector similarity according to claim 3, characterized in that, The step of calculating the polarization vector fields on both sides of the target to be reconstructed based on two types of polarization information and the corresponding zenith angles includes: The polarization vector field on the first side of the target to be reconstructed is calculated using the following formula: ； in, The polarization vector field of the first side of the target to be reconstructed. Here, T represents the first type of azimuth angle, and T is the matrix transpose operation. The polarization vector field on the second side of the target to be reconstructed is calculated using the following formula: ； in, The polarization vector field of the second side of the target to be reconstructed. This is a second-type zenith angle. This is the second type of azimuth. and Each includes multiple polarization vectors.

5. The stereo matching method based on polarization vector similarity according to claim 1, characterized in that, The step of calculating the polarization vector similarity of each matching window in the two corrected image sets based on the polarization vector fields on both sides of the target to be reconstructed includes: The polarization vector similarity of each matching window is calculated using the following formula: ； in, For the first image set, the... i The first window v The polarization vector of each pixel. For the second image set j The first window v The polarization vector of one pixel; || represents the vector modulo operation. W To match the window, For the first image set, the... i The first window and the second image set j The similarity of polarization vectors of each window.

6. The stereo matching method based on polarization vector similarity according to claim 5, characterized in that, The step of calculating a disparity map based on the maximum value of the polarization vector similarity, and using the disparity map to calculate the depth map of the target to be reconstructed, includes: The first image set is calculated based on the maximum value of the polarization vector similarity. i The parallax of each window; where the first image set is... i The disparity of the nth window is the first image set. i The window is shifted to the matching second image set. j The number of grid points in each window; The disparity of all windows in the first image set is used to form a disparity map; the center pixel of each window is the pixel in the polarization sub-image of the first image set; Multiply the focal length by the baseline length to obtain the depth parallax scaling factor; camera parameters include the focal length and the baseline length; Divide the depth disparity scaling factor by the disparity map to obtain the depth map.

7. The stereo matching method based on polarization vector similarity according to claim 1, characterized in that, The step of performing epipolar correction on two image sets using the camera parameters to obtain a first corrected image set and a second corrected image set includes: Construct the left rotation matrix and the right rotation matrix based on the rotation matrix and translation vector of the camera parameters, respectively; The first image set is remapped using the left rotation matrix and the intrinsic parameter matrix to obtain the first corrected image set. The second image set is remapped using the right rotation matrix and the intrinsic parameter matrix to obtain the second corrected image set.

8. The stereo matching method based on polarization vector similarity according to claim 1, characterized in that, Before obtaining the camera parameters, the process also includes: Multiple target images were obtained by photographing the checkerboard calibration board from multiple angles using a camera. The camera is calibrated using all the target images to obtain camera parameters; the camera parameters include intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficients.

9. The stereo matching method based on polarization vector similarity according to claim 1, characterized in that, The method involves taking photos from both sides of the target to be reconstructed at different angles using a camera to obtain a first image set and a second image set, including: A camera is used to capture images of the target to be reconstructed at four angles: 0°, 45°, 90°, and 135° on the first side of the target, resulting in a first image set; the first image set contains four polarization images. A camera is used to capture images of the target to be reconstructed at four angles: 0°, 45°, 90°, and 135° on the second side of the target, resulting in a second image set; the second image set contains four polariton images.

10. The stereo matching method based on polarization vector similarity according to claim 6, characterized in that, The first image set is calculated based on the maximum value of the polarization vector similarity. i The parallax of each window includes: Fix the sequence number of the first image collection window i Iterate through the indexes of the second image collection window. j The optimal sequence number of the second image set window when the polarization vector similarity reaches its maximum value. j max ; The optimal sequence number of the second image set window j max Subtract the index of the first image collection window. i The first image set is obtained. i Parallax of each window.

Images