Image rectification method based on binocular parallax

Abstract

The application provides a kind of image rectification method based on binocular disparity, comprising: obtaining the first view image and the second view image of different depth of field of target scene;The first view image and the second view image are adjusted to the same preset size;Determine the relative position attitude of the first view image and the second view image;According to the relative position attitude, the corresponding pixel points in the first view image and the second view image are adjusted to be aligned in horizontal direction;The first view image and the second view image are binocular depth estimation to obtain the first image, and the first view image or the second view image with relatively smaller depth of field is monocular depth estimation to obtain the second image;The first image and the second image are image fusion.The rectification method provided by the application can successfully generate long-range and near view disparity map, and is obviously superior to monocular disparity estimation method in accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to an image correction method based on binocular parallax. Background Technology

[0002] Existing binocular parallax estimation algorithms all use standard stereo camera devices. Due to the influence of the camera device's own parameters, they cannot achieve high-definition monitoring of objects at both near and far distances, especially when the focal length difference of the camera device is greater than 0.5mm, which needs to be improved. Summary of the Invention

[0003] This invention addresses at least some of the aforementioned problems by providing an image correction method based on binocular parallax, which can successfully generate far-field and near-field parallax maps with different focal depths and is significantly more accurate than monocular parallax estimation methods.

[0004] This invention provides an image correction method based on binocular parallax, comprising:

[0005] Acquire first-person and second-person perspective images of the target scene at different depths of field;

[0006] Adjust the first perspective image and the second perspective image to the same preset size;

[0007] Determine the relative position and pose of the first view image and the second view image;

[0008] Based on the relative position and posture, adjust the corresponding pixels in the first view image and the second view image to align in the horizontal direction;

[0009] A first image is obtained by performing binocular depth estimation on the first view image and the second view image, and a second image is obtained by performing monocular depth estimation on the first view image or the second view image with a relatively small depth of field.

[0010] The first image and the second image are then merged.

[0011] In one embodiment, image fusion of the first image and the second image includes:

[0012] Determine the vector difference between a pixel in the first image and the corresponding pixel in the second image;

[0013] Adjust other pixels in the second image that do not correspond to the first image based on the vector difference;

[0014] The third image is obtained by combining the pixels in the first image with the other adjusted pixels in the second image.

[0015] In one embodiment, obtaining a third image by combining pixels from the first image with other adjusted pixels from the second image includes:

[0016] According to the formula Fill the top and bottom pixels of the first image. Represents the first image. Represents the second image, D r This represents the disparity map after filling, where u represents the number of columns and j represents the number of rows.

[0017] According to the formula The first image is filled with left and right pixels, where Represents the first image. Represents the second image, D r This represents the disparity map after filling, where i represents the number of rows and v represents the number of columns.

[0018] In one embodiment, adjusting the first viewpoint image and the second viewpoint image to the same preset size includes:

[0019] Based on the image with a relatively larger depth of field between the first and second perspective images, the image with a relatively smaller depth of field is reduced in size, and the corresponding target objects in the first and second perspective images are adjusted to the same size.

[0020] Add pixels to the first or second view image with a relatively small depth of field, and adjust the first and second view images to the same size.

[0021] In one embodiment, adjusting the corresponding target objects in the first and second viewpoint images to the same size, based on the one with a relatively larger depth of field between the first and second viewpoint images, and reducing the size of the other with a relatively smaller depth of field, includes:

[0022] Calculate k = f l / f r , where f l f is the focal length of the first viewpoint image. r The focal length of the second-view image;

[0023] When k < 1, the second perspective image is reduced by a factor of k;

[0024] When k=1, no operation is performed;

[0025] When k>1, the first viewpoint image is reduced by a factor of 1 / k.

[0026] In one embodiment, supplementing pixels to the first or second view image with a relatively small depth of field, and adjusting the first and second view images to the same size, includes:

[0027] Compare I l >I′ r , where I l I′ represents the size of the first viewpoint image. r Indicates the size of the second-view image;

[0028] If I l >I′ r , with I l Based on, I′ r Adjust to the same size by adding pixels;

[0029] If I l <I′ r , with I′ r Based on, I l Adjust to the same size by adding pixels;

[0030] If I′ r =I l No image adjustments are required.

[0031] In one embodiment, the relative position attitude includes:

[0032] The vector difference and rotation angle between corresponding pixels in the first viewpoint image and the second viewpoint image.

[0033] In one embodiment, the vector difference and the rotation angle are calculated using the essential matrix E and the fundamental matrix F;

[0034] F = K -T EK -1 , where K l K r Built-in parameters of the camera device

[0035] The fundamental matrix F is the relationship matrix between the pixels of the first-view image and the second-view image. Take any pair of matching points x1 and x′1, and define x1 as a feature point in the first-view image with coordinates x1 = [u1, v1, 1]. T x′1 is the feature point corresponding to x1, and its coordinates are x1′=[u′1,v′1,1] T According to polar constraints It can be known that:

[0036]

[0037] Let f = [F 11 F 12 F 13 F 21 F 22 F 23 F 31 F 32 F 33 ] T We get [u1u′1 u1v′1 u1 v1u′1 v1v′1 v1u′1 v′1 1]f=0, when there are n pairs of matching points.

[0038]

[0039] Because the essential matrix has favorable properties, its singular values ​​are of the form [σ,σ,0], and the external parameters R and T of the binocular camera device can be obtained through singular value decomposition; the singular value decomposition of the essential matrix E is:

[0040] E=U∑V T

[0041] Where U is an orthogonal matrix of column vectors, meaning that any two column vectors in it are orthogonal; V T ∑ is an orthogonal matrix of row vectors, meaning that any two row vectors in it are orthogonal; ∑ is a diagonal matrix with rank 2;

[0042] After decomposing the essential matrix, the values ​​of the extrinsic parameter R are obtained as follows:

[0043] R = UW T V T Or R = UWV T

[0044] in,

[0045] The translation vector T takes the following values:

[0046] T = u3 or T = -u3

[0047] R and T can take four values:

[0048]

[0049] Based on the actual positional relationship between the first and second viewpoint images, one of the geometric relationships can be determined to be correct, i.e., by selecting an observation point, the true solution of the vector difference and rotation angle can be obtained.

[0050] The image correction method based on binocular parallax provided by this invention addresses the issue that the first-view image and the second-view image have different depths of field, resulting in different sizes of objects in the two images. By adjusting the first-view image and the second-view image to a preset size, the object sizes in the two images can be made approximately the same. This reduces the computational range and computational load during subsequent parallax estimation, improving decoding efficiency. Furthermore, by adjusting the horizontal position of the images, the objects in the two images are roughly aligned, further reducing the computational load for binocular depth estimation, monocular depth estimation, and image fusion, thus facilitating the acquisition of clearer images. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating an image correction method according to an embodiment of the present invention;

[0052] Figure 2 This is a flowchart illustrating the image correction method in another embodiment;

[0053] Figure 3 This is a flowchart illustrating the image correction method in yet another embodiment;

[0054] Figure 4 This is a flowchart illustrating the image correction method in other embodiments;

[0055] Figure 5 This is a schematic diagram illustrating image adjustment based on focal length ratio.

[0056] Figure 6 The position diagrams show the four possible values ​​for R and T of the camera device:

[0057] Figure 7 This is a schematic diagram of a binocular camera device. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] like Figure 1 As shown, in one embodiment, an image correction method based on binocular parallax includes steps S10, S20, S30, S40, S50 and S60.

[0060] Step S10: Obtain first-view and second-view images of the target scene at different depths of field.

[0061] Images are acquired using a camera device. In this embodiment, the focal lengths of the camera device are 6mm and 16mm, thus establishing a binocular system with different focal lengths. For ease of description, the image obtained with a focal length of 6mm is taken as the first-view image, and the image obtained with a focal length of 16mm is taken as the second-view image. It can be understood that in other embodiments, the camera device can also have other focal lengths, as long as the focal length difference between the first-view image and the second-view image is greater than 0.5mm.

[0062] like Figure 2 Step S20: Adjust the first view image and the second view image to the same preset size.

[0063] The preset size can be a size determined according to specific working conditions, or a size based on first-view or second-view images with different focal lengths and depths of field.

[0064] Step S21: Based on the image with a relatively larger depth of field between the first view image and the second view image, reduce the size of the image with a relatively smaller depth of field, and adjust the corresponding target objects in the first view image and the second view image to the same size.

[0065] In this embodiment, the size of a first-view image with a focal length of 6mm and a large depth of field is used as the preset size. Therefore, when adjusting the image size, only the second-view image needs to be adjusted, which reduces the amount of program calculation and helps to improve the efficiency of encoding calculation. Furthermore, since the image with a large depth of field is used as the benchmark, subsequent disparity estimation and image fusion can obtain a more comprehensive image.

[0066] Specifically, if the focal length is known, the image size can be adjusted according to the focal length ratio. Sometimes, the focal length of the camera device is not always known. In such cases, an uncalibrated stereo correction method based on the quasi-Euclidean method can be used. In this embodiment, the focal length of the camera device has been calibrated using the Zhang Zhengyou calibration method. The image is then directly adjusted according to the focal length ratio so that the size of the same target object in the first-view image and the second-view image are basically the same.

[0067] For the camera device model, the image formation equation (1) is as follows:

[0068]

[0069] Where (x1, y1) are the coordinates of a 3D point (X, Y, Z) in the world's 3D coordinate system within an image captured by a camera with a focal length of f1. Similarly, in a camera with a focal length of f2, the following equation can be obtained:

[0070]

[0071] The relationship between image pixels can be derived from the two equations above:

[0072] x1=kx2,y1=ky2 (3)

[0073] See Figure 5 Where k = f1 / f2, representing the focal length ratio of the camera device acquiring the first-view image and the second-view image. Thus, the influence of different focal lengths can be compensated for using the focal length ratio in perspective geometry. If we perform the correction process without compensation, it will lead to large correction and distortion errors. Therefore, a focal length ratio compensation process is needed to obtain a better corrected image pair. The specific steps are as follows:

[0074] 1) Calculate the focal length ratio k = f l / f r , where f l f is the focal length of the first image. r The focal length of the second-view image;

[0075] 2) When k < 1, the second-view image I r Reduced by a factor of k; when k = 1, no operation is performed on the binocular image; when k > 1, the first-view image I is reduced. l Reduce by a factor of 1 / k.

[0076] This allows the target object in the first-view image and the second-view image to be adjusted to approximately the same size, reducing the computational load during subsequent image processing of the target object and facilitating the obtaining of a clear image. It is worth noting that the target object refers to the image of the same object in both the first-view image and the second-view image, that is, the pixel points corresponding to the same object.

[0077] Step S22: Add pixels to the first view image or the second view image with a relatively small depth of field, and adjust the first view image and the second view image to the same size.

[0078] In the above steps, the target objects in the first-view image and the second-view image are roughly the same size, but the overall size of the first-view image and the second-view image are not the same, which will affect the subsequent disparity estimation, so further adjustments are needed.

[0079] In this embodiment, since the selected focal length satisfies k < 1, the second-view image is scaled by a factor of k, and the first-view image I is then... l Second-person perspective image I′ r Although both images are at the same scale, it's clear that the second-view image has changed size after being scaled. Let's define the size of the image at this point as w′. r ×h′ rTo ensure that the binocular images are of consistent size, the adjusted images need to be padded. The padding process mainly includes the following steps:

[0080] 1) Determine the image size after binocular adjustment;

[0081] 2) If I l >I′ r A blank black background is pre-set, the size of which is the same as the first-view image I. l Keep the size consistent, and adjust the image I′ r Fill the background image with the top-left corner at coordinates (u1, v1) and a size of w′. r ×h′ r Where u1, v1 can be represented as:

[0082]

[0083] The filled image is I″ r .

[0084] 3) If I′ r >I l A blank black background is pre-defined, the size of which is the same as the adjusted image I′. r Keep the size consistent, fill the background image with the first image, and perform the remaining operations as in step 2.

[0085] 4) If I′ r =I l At this point, the two images are the same size, so no adjustment is needed.

[0086] By using this method, the first-view image and the second-view image can be adjusted to approximately the same size. The target object in the first-view image and the second-view image is also approximately the same size, which facilitates subsequent algorithm processing and helps reduce the amount of computation.

[0087] Step S30: Determine the relative position and pose of the first view image and the second view image.

[0088] Figure 3 Step S31: The vector difference and rotation angle of corresponding pixels in the first view image and the second view image.

[0089] Specifically, the binocular camera device consists of two monocular camera device models. See also Figure 7The extrinsic parameters R and T of a binocular camera device refer to the positional relationship between the two optical centers. These extrinsic parameters can be obtained by decomposing the essential matrix E. In solving for the essential matrix E, it can be calculated using the fundamental matrix F or directly from the normalized image pixel coordinates. The method using the fundamental matrix F involves the fundamental matrix being an intrinsic constraint of the binocular system, related only to the system parameters and independent of the actual application scenario. While the method using normalized image pixel coordinates does not require intrinsic parameters, it involves many unknowns, making it difficult to obtain accurate intrinsic and extrinsic parameters and requiring assumptions such as an initial focal length and the principal point being at the image center. Therefore, the method used is to calculate the essential matrix E using the fundamental matrix F. The essential matrix E contains the intrinsic parameters K of the binocular camera system. l K r And the external parameters R and T. Given that the intrinsic parameters of the camera device have been calibrated, the relationship between the fundamental matrix F and the essential matrix E is:

[0090] F = K -T EK -1 (5)

[0091] As can be seen from the formula, the essential matrix is ​​only related to the external parameters R and T of the binoculars, and is not related to the intrinsic parameters of the camera device.

[0092] The fundamental matrix F is the relationship matrix between pixels in the stereo image. Take any pair of matching points x1 and x′1 in the stereo image, and define x1 as a feature point in the left image with coordinates x1 = [u1, v1, 1]. T x′1 is the feature point corresponding to x1, and its coordinates are x1′=[u′1,v′1,1] T According to polar constraints It can be known that:

[0093]

[0094] Let f = [F 11 F 12 F 13 F 21 F 22 F 23 F 31 F 32 F 33 ] T ,have to:

[0095] [u1u′1 u1v′1 u1 v1u′1 v1v′1 v1 u′1 v′1 1]f=0 (7)

[0096] When there are n pairs of matching points

[0097]

[0098] Since the unknown f has 9 degrees of freedom, at least 8 known matching points are needed to obtain a unique solution.

[0099] The intrinsic parameter matrices of the two cameras in the binocular camera system are K and K, respectively. l K r A three-dimensional coordinate system is established using the coordinate system of the camera device in the first-view image as a reference, and the coordinate system of the camera device in the second-view image is made to coincide with the world coordinate system. The pose of the camera device in the second-view image relative to the three-dimensional points is the pose relationship between the two camera devices. Due to the favorable properties of the essential matrix, its singular values ​​are of the form [σ,σ,0]. The external parameters R and T of the binocular camera device can be obtained through singular value decomposition. The singular value decomposition of the essential matrix E is as follows:

[0100] E=U∑V T (9)

[0101] Where U is an orthogonal matrix of column vectors, meaning that any two column vectors in it are orthogonal; V T ∑ is an orthogonal matrix of row vectors, meaning that any two row vectors in it are orthogonal; ∑ is a diagonal matrix with rank 2.

[0102] After decomposing the essential matrix, the values ​​of the extrinsic parameter R are obtained as follows:

[0103] R = UW T V T Or R = UWV T (10)

[0104] in,

[0105]

[0106] The translation vector T takes the following values:

[0107] T = u3 or T = -u3 (12)

[0108] Therefore, R and T can take four values:

[0109]

[0110] The geometric positional relationships of the four possible values ​​are as follows: Figure 6 As shown:

[0111] Based on the actual positional relationship between the two camera devices, only one geometric relationship is correct. Therefore, it is only necessary to select one observation point to obtain the true solution for the extrinsic parameters of the camera devices. Of course, in other embodiments, other existing methods can also be used to calculate the extrinsic parameters of the camera devices.

[0112] S40 adjusts the corresponding pixels in the first view image and the second view image to be aligned in the horizontal direction according to the relative position and posture.

[0113] Based on the external parameters of the camera device obtained above, the position of the target object in the first-view image and the second-view image is adjusted. Specifically, in order to minimize the distortion of image reprojection, the planes of the two camera devices are rotated towards the center. At this time, the rotation matrix R is divided into two parts, and the rotation matrices of the camera devices are defined as r. l r r At this time, r l r r The following relationships should be satisfied:

[0114]

[0115] To transform the pole of the camera device in the first-person view image to infinity and align the epipolar lines horizontally, a rotation matrix R is constructed. rect The rotation matrix starts from the direction of the pole e1 and sets the principal point (c) as the pivot. x ,c y The origin of the imaging plane of the camera device, which serves as the first-person view image, has its pole direction as the direction of the translation vector between the projection centers of the two camera devices, which can be expressed as:

[0116]

[0117] The next vector e2 is orthogonal to e1. ​​e2 is obtained by the cross product of e1 and the direction of the principal ray, and then normalized, and is expressed as:

[0118]

[0119] The third vector e3 is obtained by the cross product of e2 and e1. Therefore, the poles in the camera device that capture the first-person view image are transformed into a matrix R at infinity. l R r The matrix can be represented as:

[0120]

[0121] Thus, by using this method, the corresponding points in the binocular images are located on the same horizontal line after correction, and the target objects in the first-view image and the second-view image are roughly at the same horizontal position. Subsequent disparity calculation can be simplified to a one-dimensional line search, which is beneficial for simplifying image processing.

[0122] S50 performs binocular depth estimation on the first view image and the second view image to obtain a first image, and performs monocular depth estimation on the first view image or the second view image with a relatively small depth of field to obtain a second image.

[0123] There are many methods and algorithms for binocular and monocular depth estimation, and no detailed limitations are provided here, as long as they can achieve the corresponding depth estimation purpose. In this embodiment, since the depth of field of the second-view image is relatively small, monocular depth estimation is performed on the second-view image.

[0124] S60 performs image fusion between the first image and the second image.

[0125] Figure 4 S61 determines the vector difference between the pixels in the first image and the corresponding pixels in the second image;

[0126] S62 adjusts other pixels in the second image that do not correspond to the first image according to the vector difference;

[0127] S63 combines the pixels in the first image with the other adjusted pixels in the second image to obtain the third image.

[0128] Specifically, using camera devices with different focal lengths can provide disparity information for near and far scenes. Since short-focal-length images contain the necessary information for stereo matching calculations, calculating the disparity map of long-focal-length images using binocular disparity estimation methods is straightforward. However, images with different focal lengths lack sufficient information to calculate the disparity map of short-focal-length images using binocular methods. Their disparity maps are calculated using monocular depth estimation methods. However, the accuracy of monocular depth estimation methods is limited and significantly lower than that of binocular disparity estimation methods. Because binocular disparity estimation methods require corresponding points from stereo images to calculate disparity, they can calculate a complete disparity map for first-view images but only a partial disparity map for second-view images. Therefore, monocular disparity estimation is used as supporting information to accurately calculate the complete disparity map of the corrected right image.

[0129] The panoramic parallax adjustment process first involves creating a blank parallax map template. The optimized binocular parallax map (i.e., the first image) is then overlaid on the same area, and the blank area is filled with the monocular parallax estimation result (i.e., the second image). Four lines are then created along the boundaries of the filled image: two horizontal lines and two vertical lines. The monocular parallax map is then used... This indicates that binocular parallax diagrams are used... This indicates that the disparity map after filling is represented by D. r The fusion process consists of two steps, the purpose of which is to interpolate pixels outside the rectangular region. The interpolation process is as follows:

[0130] 1) When filling the upper and lower parts, the fill depth data is as follows:

[0131]

[0132] Where ΔD ru It can be represented as:

[0133]

[0134] u represents the column number, and j represents the row number; it can be understood as the average of the differences between the estimated binocular and monocular disparities in the u-th column within the binocular disparity coverage area.

[0135] 2) Fill the left and right portions, with the fill depth data as follows:

[0136]

[0137] Where ΔD rv It can be represented as:

[0138]

[0139] i represents the column number, and v represents the row number; it can be understood as the average of the differences between the binocular disparity and monocular disparity estimates in the v-th row within the binocular disparity coverage area.

[0140] In this way, a third image can be obtained. The third image is equivalent to selecting the clearer part of the second image obtained by binocular depth estimation, and selecting the other parts of the first image excluding the parts displayed in the second image. Furthermore, by adjusting the pixels of the selected first image, the third image is made even clearer.

[0141] The image correction method based on binocular parallax provided by this invention addresses the issue that the first-view image and the second-view image have different depths of field, resulting in different sizes of objects in the two images. By adjusting the first-view image and the second-view image to a preset size, the object sizes in the two images can be made approximately the same. This reduces the computational range and computational load during subsequent parallax estimation, improving decoding efficiency. Furthermore, by adjusting the horizontal position of the images, the objects in the two images are roughly aligned, further reducing the computational load for binocular depth estimation, monocular depth estimation, and image fusion, thus facilitating the acquisition of clearer images.

[0142] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. An image correction method based on binocular parallax, characterized in that, include: Acquire first-person and second-person perspective images of the target scene at different depths of field; Adjust the first perspective image and the second perspective image to the same preset size; Determine the relative position and pose of the first view image and the second view image; The relative position pose includes: the vector difference and rotation angle of corresponding pixels in the first view image and the second view image; Through the essential matrix With the fundamental matrix Calculate the vector difference and the rotation angle; Based on the relative position and posture, adjust the corresponding pixels in the first view image and the second view image to align in the horizontal direction; A first image is obtained by performing binocular depth estimation on the first view image and the second view image, and a second image is obtained by performing monocular depth estimation on the one with the smaller depth of field between the first view image and the second view image. The first image and the second image are then merged.

2. The image correction method based on binocular parallax as described in claim 1, characterized in that, Image fusion of the first image and the second image includes: Determine the vector difference between a pixel in the first image and the corresponding pixel in the second image; Adjust other pixels in the second image that do not correspond to the first image based on the vector difference; The third image is obtained by combining the pixels in the first image with the other adjusted pixels in the second image.

3. The image correction method based on binocular parallax as described in claim 2, characterized in that, The third image is obtained by combining the pixels in the first image with the other adjusted pixels in the second image, including: According to the formula Fill the top and bottom pixels of the first image. , Represents the first image. This represents the second image. This represents the disparity map after filling. Indicates the column number. Indicates the row number; According to the formula The first image is filled with left and right pixels, where , Represents the first image. This represents the second image. This represents the disparity map after filling. Indicates the row number. Indicates the column number.

4. The image correction method based on binocular parallax as described in claim 1, characterized in that, Adjusting the first viewpoint image and the second viewpoint image to the same preset size includes: Based on the image with a relatively larger depth of field between the first and second perspective images, the image with a relatively smaller depth of field is reduced in size, and the corresponding target objects in the first and second perspective images are adjusted to the same size. Add pixels to the image with the smaller depth of field in the first view image and the second view image, and adjust the first view image and the second view image to the same size.

5. The image correction method based on binocular parallax as described in claim 4, characterized in that, Using the image with a relatively larger depth of field between the first and second viewpoint images as a reference, reducing the size of the image with a relatively smaller depth of field, and adjusting the corresponding target objects in the first and second viewpoint images to the same size includes: calculate ,in, Let be the focal length of the first perspective image. The focal length of the second-view image; when At that time, the second perspective image is reduced in size. times; when At that time, no operation is performed; when At that time, the first viewpoint image is zoomed out. times.

6. The image correction method based on binocular parallax as described in claim 1, characterized in that, ,in, These are the built-in parameters of the camera device; Essential matrix The singular value decomposition is as follows: in It is a column vector orthogonal matrix, meaning that any two column vectors in it are orthogonal. It is a row vector orthogonal matrix, meaning that any two row vectors in it are orthogonal. It is a diagonal matrix with rank 2; After decomposing the essential matrix, the extrinsic parameters are obtained. The value can be: in, Translation vector The value can be: , There are four possible values: Based on the actual positional relationship between the first and second viewpoint images, one of the geometric relationships can be determined to be correct, i.e., by selecting an observation point, the true solution of the vector difference and rotation angle can be obtained. It is the third column vector of the column vector orthogonal matrix U.

Images