A 3D Reconstruction System and Method Based on Panoramic Binocular Stereo Vision

Abstract

The present invention belongs to the technical field of computer vision three-dimensional reconstruction, and specifically is a three-dimensional reconstruction system and method based on panoramic binocular stereo vision. The three-dimensional reconstruction system of the present invention includes two panoramic cameras, two multi-dimensional adjustment frames and a computer processing system. The panoramic camera is composed of an image sensor and a quadratic catadioptric panoramic lens. According to the imaging relationship of the quadratic catadioptric panoramic camera, a plane-cylindrical projection model is established. The three-dimensional reconstruction method of the present invention includes: panoramic image acquisition and image processing to obtain a panoramic annular image; panoramic camera calibration; feature point detection of the panoramic annular image using a feature point extraction algorithm; feature matching using a balanced binary tree method; calculating the three-dimensional coordinates of actual object points to achieve panoramic three-dimensional reconstruction of sparse features. The present invention can quickly and efficiently achieve three-dimensional reconstruction of 360° annular scenes, and has broad application prospects in robot navigation, automatic driving, non-contact three-dimensional measurement and other aspects.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision three-dimensional reconstruction, and in particular relates to a three-dimensional reconstruction system and method based on panoramic binocular stereo vision. Background Art

[0002] Image-based 3D reconstruction technology is to obtain a 3D model of a scene from an image sequence. It is one of the means to solve 3D modeling in the field of computer vision. It has very important practical significance for real-time large-scale complex scene 3D modeling. Among them, binocular stereo vision uses two cameras to obtain two digital images of the same 3D scene from different angles, and calculates the position deviation (i.e., parallax) between the pixels of the two images through stereo matching to obtain the 3D geometric information and depth information of the 3D scene, so as to achieve the purpose of reconstructing the 3D shape and position of the scene. Binocular stereo vision directly simulates the way the human eye processes scenes, and has been widely used in many fields, such as robot navigation and aerial survey, autonomous driving, augmented reality, non-contact 3D measurement, etc.

[0003] The 3D reconstruction technology of panoramic space has important applications in many fields, but the traditional imaging system is limited by the field of view and it is difficult to quickly and directly restore the depth information of all object points in the 360° ring scene. At present, panoramic stereo imaging is mainly divided into three categories: stitching panoramic stereo imaging, fisheye panoramic stereo imaging and catadioptric panoramic stereo imaging. Stitching panoramic stereo imaging mainly includes two methods: single camera rotation shooting and multi-camera surround shooting. The single camera rotation shooting method uses the camera to collect image sequences in sequence around the axis 360°, and then stitch them into a panoramic ring image, but its imaging speed is slow and not real-time; the multi-camera imaging method is to use multiple cameras to surround a circle, each camera fixedly shoots a certain angle to obtain a fixed angle image, and then stitches it into a panoramic ring image. Although it can meet the real-time requirements, the stitching system is not only more expensive but also more complicated, and the stitching process will also produce stitching errors, so the 3D reconstruction accuracy of stitching panoramic stereo imaging is low. Fisheye panoramic stereo imaging uses an ultra-wide-angle objective lens to achieve panoramic imaging. Its focal length is very short, so it can obtain a hemispherical scene with a field of view close to 180°. However, fisheye images have a large barrel distortion, which is difficult to correct later, resulting in low calibration accuracy, thus affecting the accuracy of 3D reconstruction. Catadioptric panoramic stereo imaging uses reflective optical elements to expand the field of view, and can obtain a horizontal 360° panoramic ring image, eliminating the steps of rotation and stitching multiple cameras, and has the advantages of simple system structure and flexible design.

[0004] According to the number of reflections, catadioptric panoramic imaging can be divided into single-reflection panoramic imaging and secondary catadioptric panoramic imaging. Single-reflection panoramic imaging has a simple structure and low cost, but it has disadvantages such as large size of system reflector, difficult integration, short focal length, serious stray light, large distortion, etc., resulting in low three-dimensional reconstruction accuracy. In contrast, secondary catadioptric panoramic imaging uses a panoramic annular lens to integrate the refractive and reflective surfaces together, which has the advantages of compact structure, easy integration, long focal length, large field of view, low distortion, etc., and high three-dimensional reconstruction accuracy. Therefore, the present invention is based on a secondary catadioptric panoramic stereo imaging system, which can perform fast, efficient, and high-precision three-dimensional reconstruction of 360° annular scenes, and has broad application prospects in robot navigation and aerial survey, automatic driving, augmented reality, non-contact three-dimensional measurement, etc. Summary of the invention

[0005] The purpose of the present invention is to provide a three-dimensional reconstruction system and method based on panoramic binocular stereo vision, which can perform fast, efficient and high-precision panoramic three-dimensional reconstruction of a 360° ring scene.

[0006] The three-dimensional reconstruction system based on panoramic binocular stereo vision provided by the present invention comprises: two panoramic cameras, two multi-dimensional adjustment frames and a computer processing system; wherein:

[0007] The panoramic camera is composed of an image sensor and a secondary catadioptric panoramic lens, the relative positions of which are adjusted and fixed by screw threads, so that the panoramic lens forms an image on the image sensor, and the collected image information is transmitted to a computer processing system;

[0008] The two panoramic cameras are respectively fixed on two multi-dimensional adjustment frames, and the multi-dimensional adjustment frames are adjusted so that the two panoramic cameras have the same optical axis and have overlapping fields of view. The two panoramic cameras are used to collect two panoramic images with parallax for three-dimensional reconstruction of a 360° ring scene.

[0009] The computer processing system is used to drive the two panoramic cameras to collect panoramic images, calculate the calibration parameters of the two panoramic cameras, and complete the three-dimensional reconstruction of the 360° ring scene.

[0010] The quadratic catadioptric panoramic lens mapping relationship adopts a plane-cylindrical projection model based on Taylor series expansion, and can project a 360° annular scene onto a two-dimensional plane.

[0011] The optical system of the secondary catadioptric panoramic lens is composed of a panoramic annular lens, an aperture diaphragm and a relay lens group arranged in sequence along the optical axis. The incident light is reflected and refracted twice inside the panoramic annular lens, passes through the aperture diaphragm and the relay lens group, and is finally received by the image sensor to generate a panoramic image; the image sensor outputs the panoramic image to a computer processing system for three-dimensional reconstruction of a 360° annular scene.

[0012] Based on the above three-dimensional reconstruction system, the present invention also provides a three-dimensional reconstruction method based on panoramic binocular stereo vision, and the specific steps are as follows:

[0013] Step 1, panoramic image acquisition and image processing. Place the panoramic binocular stereo vision system in the scene, use the computer processing system to drive two panoramic cameras to simultaneously shoot a 360° ring scene, and obtain two panoramic images with a certain parallax; filter the panoramic image to remove noise; cut the blind area of ​​the panoramic image to generate a panoramic ring image;

[0014] Step 2, panoramic camera calibration. The panoramic camera imaging model is a plane-cylindrical projection model based on Taylor series expansion, including a forward projection model and a reverse projection model. The relationship between the three-dimensional point coordinates of a certain object point in space and its projection point coordinates in the panoramic ring image is determined through this projection model; the three-dimensional point coordinates in the world coordinate system and the two-dimensional point coordinates in the pixel coordinate system corresponding to all corner points of the chessboard are obtained through a chessboard calibration plate; the panoramic camera calibration parameters are obtained by the least squares method, including internal parameters, external parameters and distortion parameters, which will be used for the three-dimensional scene reconstruction of the panoramic binocular stereo vision system.

[0015] Step 3: Feature point extraction. The panoramic ring images generated by the two panoramic cameras are used as the original image and the target image respectively. The feature point extraction algorithm is used to detect the feature points of the original image and the target image, and the feature description vectors of all feature points are generated. The feature description vectors will be used to calculate the feature similarity in the feature point matching process.

[0016] Step 4, feature point matching. Establish a panoramic common path constraint, map the center point of the first panoramic annular image to the corresponding radius of the second panoramic annular image according to the position transformation matrix of the two panoramic cameras, and limit the search range to the fan-shaped area where the corresponding effective radius is located, so as to reduce the complexity of matching, effectively shorten the calculation time, and reduce the mismatch rate. Use the balanced binary tree method (Adel'Son-Vel'Skii GM. An algorithms for the organization of Information [J]. Soviet Math, 1962, 3.) to find matching feature points and output the matching results.

[0017] Step 5: Calculate the three-dimensional coordinates. The panoramic annular images generated by the two panoramic cameras are converted to the same camera coordinate system. Based on the reverse projection model, the feature points on the image plane are reversely projected into the direction vectors of the corresponding object points. The two direction vectors restored from the two panoramic annular images intersect, and the coordinate value of the intersection point solved by the least squares method is the three-dimensional coordinate of the actual object point. The three-dimensional coordinates of each pair of feature points are calculated to achieve panoramic three-dimensional reconstruction of sparse features.

[0018] The beneficial effects of the present invention are:

[0019] The present invention designs a binocular stereo vision system based on a quadratic catadioptric panoramic camera, which can obtain a panoramic image of a 360° annular scene through a single imaging; a plane-cylindrical projection model based on Taylor series expansion is established to calibrate the panoramic binocular stereo vision system with high precision; feature point detection is performed on the panoramic annular image through a feature point extraction algorithm, and a panoramic common path constraint is established in the feature point matching process to narrow the search range, greatly improve the search efficiency, reduce the mismatch rate, and improve the precision and efficiency of 3D reconstruction. The present invention has important research significance in the fields of robot navigation and aerial survey, autonomous driving, augmented reality, and non-contact 3D measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention is a schematic diagram of the structure of a three-dimensional reconstruction system based on panoramic binocular stereo vision disclosed in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of imaging of a quadratic catadioptric panoramic camera disclosed in an embodiment of the present invention.

[0022] Figure 3 This is a flowchart of a 3D reconstruction algorithm based on panoramic binocular stereo vision disclosed in an embodiment of the present invention.

[0023] Figure 4 The present invention is a flowchart of a panoramic camera calibration algorithm disclosed in an embodiment of the present invention.

[0024] Figure 5 A simplified model diagram of three-dimensional reconstruction based on panoramic binocular stereo vision disclosed in an embodiment of the present invention.

[0025] Numbers in the figure: 1 is the first image sensor, 2 is the first secondary catadioptric panoramic lens, 3 is the first multi-dimensional adjustment bracket; 4 is the second image sensor, 5 is the second secondary catadioptric panoramic lens, 6 is the second multi-dimensional adjustment bracket; 7 is the computer processing system; 8 is the panoramic annular lens, 9 is the aperture stop, 10 is the relay lens group, 11 is the optical axis; 12 is the object point, 13 is the image plane, 14 is the image sensor, 15 is the imaging area, 16 is the blind area, 17 is the non-imaging area; 18 is the first panoramic cylindrical coordinate system, and 19 is the second panoramic cylindrical coordinate system. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. However, it should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0027] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0028] like Figure 1 As shown, an embodiment of the present invention provides a three-dimensional reconstruction system based on panoramic binocular stereo vision, including: a first panoramic camera composed of a first image sensor 1 and a first quadratic catadioptric panoramic lens 2; a first multi-dimensional adjustment bracket 3; a second panoramic camera composed of a second image sensor 4 and a second quadratic catadioptric panoramic lens 5; a second multi-dimensional adjustment bracket 6; and a computer processing system 7. The first panoramic camera is composed of a first image sensor 1 and a first quadratic catadioptric panoramic lens 2, and is fixed by threads; the second panoramic camera is composed of a second image sensor 4 and a second quadratic catadioptric panoramic lens 5, and is fixed by threads. The first panoramic camera and the second panoramic camera are respectively fixed on the first multi-dimensional adjustment bracket 3 and the second multi-dimensional adjustment bracket 6, and the two multi-dimensional adjustment brackets are adjusted so that the first panoramic camera and the second panoramic camera have the same optical axis and have an overlapping field of view.

[0029] Figure 2 The imaging schematic diagram of the quadratic catadioptric panoramic camera disclosed in the embodiment of the present invention includes: an optical axis 11, an object point 12, a panoramic annular lens 8, an aperture stop 9, a relay lens group 10, an image plane 13, an image sensor 14, an imaging area 15, a blind area 16, and a non-imaging area 17. Figure 2As shown, the incident light emitted by a certain point 12 in space is reflected and refracted twice in the panoramic annular lens 8, passes through the aperture diaphragm 9 and the relay lens group 10, and is finally received by the image sensor 14 to generate a panoramic image. The panoramic image is composed of a blind area 16, an imaging area 15, and a non-imaging area 17 from the inside to the outside. The image sensor outputs the panoramic image to the computer processing system 7 for three-dimensional reconstruction of the 360° annular scene.

[0030] The quadratic catadioptric panoramic camera mapping relationship adopts a plane-cylindrical projection model, which can project a 360° annular scene onto a two-dimensional plane.

[0031] According to the imaging relationship of the quadratic catadioptric panoramic camera, a plane-cylindrical projection model based on Taylor series expansion is established. The specific description of the model is as follows:

[0032] (1) Project the object point P in the world coordinate system into the camera coordinate system. Assume that light rays from different incident directions all pass through a single effective viewpoint O. Use this as the origin to establish the camera coordinate system O-XYZ. The object point P is transformed into the camera coordinate system through a rigid body transformation. The incident vector (x, y, z) and the outgoing vector (u, v, w) are obtained. Normalize the incident vector in the horizontal direction to obtain the unit circle based on xy. Similarly, the unit circle of the outgoing vector based on uv can be obtained. Therefore, the following holds:

[0033]

[0034] Let the scaling factor λ be:

[0035]

[0036] (2) The nonlinear function g is used to fit the projection relationship, and the relationship between the image plane point coordinates (u, v) and the incident vector p is obtained, where p is the vector from the viewpoint O to the field point P:

[0037] g(u,v)=(u,v,f(u,v)),

[0038] Where f is rotationally symmetric about the image sensor, and the specific expression depends on the shape of the panoramic annular lens:

[0039] f(u,v)=α0+α1ρ+α2ρ 2 +...+α N ρ N ,

[0040]

[0041] The use of polynomial fitting not only overcomes the lack of knowledge about the parameter model of fisheye or quadratic catadioptric cameras, but also puts the distortion and camera intrinsic parameters together to directly correct the radial distortion.

[0042]

[0043] Right now:

[0044]

[0045] And according to the ρ value, the pixel coordinates of the reprojection point can be calculated as

[0046]

[0047] (3) In the process of converting the image plane point coordinates (u, v) to pixel coordinates (u′, v′), the affine transformation is used to fit the uneven deformation caused by the lens installation error and the different pixel sizes in practice. Therefore, the actual pixel coordinate value is:

[0048]

[0049] Among them, (u0, v0) is the image center point. The affine transformation matrix elements (c, d, e, f) are obtained by calculation.

[0050] The embodiment of the present invention provides a three-dimensional reconstruction method based on panoramic binocular stereo vision. Figure 3 FIG. 1 is a flowchart of a 3D reconstruction algorithm based on panoramic binocular stereo vision disclosed in an embodiment of the present invention. Figure 3 As shown in Figure 1, the 3D reconstruction algorithm mainly includes the following five steps:

[0051] Step 1, panoramic image acquisition and image processing. Place the panoramic binocular stereo vision system in the scene, use the computer processing system to drive two panoramic cameras to simultaneously shoot a 360° ring scene, and obtain two panoramic images with a certain parallax; filter the panoramic image, and use the spatial domain smoothing method to remove unnecessary noise in the image while retaining the image details to the greatest extent; cut the blind area of ​​the panoramic image to generate a panoramic ring image, reduce the mismatch rate, and improve the efficiency and accuracy of feature point matching.

[0052] Step 2, panoramic camera calibration. The panoramic camera imaging model is a plane-cylindrical projection model based on Taylor series expansion, including a forward projection model and a reverse projection model. The projection model is used to determine the relationship between the three-dimensional point coordinates of a certain object point in space and its projection point coordinates in the panoramic ring image; the three-dimensional point coordinates in the world coordinate system and the two-dimensional point coordinates in the pixel coordinate system corresponding to all feature points are obtained through the calibration plate. The least squares method is used to obtain the panoramic camera calibration parameters, including internal parameters, external parameters and distortion parameters, which will be used for the three-dimensional scene reconstruction of the panoramic binocular stereo vision system.

[0053] Figure 4 This is a flowchart of a panoramic camera calibration algorithm disclosed in an embodiment of the present invention, and the specific contents are as follows:

[0054] Input the panoramic annular image sequence of the calibration plate into the subsequent steps; determine and input the known parameters of the system, including the coordinates of the center point of the panoramic annular image and the actual size of the chessboard, and obtain the three-dimensional point coordinates of the corner points of the chessboard in the world coordinate system; extract the pixel coordinates of the corner points of the chessboard, that is, the two-dimensional point coordinates of the object point on the image plane; perform Taylor series expansion on the projection relationship between the camera coordinate system and the image plane coordinate system, and solve the internal and external parameters in turn through the linear least squares method to obtain the initial solution, and establish an ideal model without distortion; calculate the initial solution of the distortion parameter based on the difference between the ideal model and the actual value, and obtain the actual camera model; with the minimum reprojection error as the constraint condition, perform iterative nonlinear optimization on the initial solution of the internal parameters, external parameters and distortion parameters of the panoramic camera, and obtain the optimal solution as the geometric model parameters of the panoramic camera. Then calculate the position transformation matrix of the first panoramic camera and the second panoramic camera based on the external parameter matrix of the first panoramic camera and the second panoramic camera.

[0055] Step 3: Feature point extraction. The panoramic ring images generated by the two panoramic cameras are used as the original image and the target image respectively. The scale-invariant feature transform (SIFT) algorithm ([2] DGLowe, et al. Object recognition from local scale-invariant features [C], Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, pp. 1150-1157 vol. 2.) is used to detect feature points of the original image and the target image, and generate feature description vectors of all feature points. The feature description vectors will be used to calculate feature similarity in the feature point matching process.

[0056] Among them, the SIFT algorithm is preferably used for feature point extraction, specifically: constructing an image pyramid and an image difference pyramid, and performing preliminary extreme value detection in the difference pyramid; performing Taylor series expansion near the detected extreme value points to achieve sub-pixel positioning; sampling a small area near the feature point in the pyramid from the spatial position of the feature point to determine the main direction angle of the feature point; sampling a small area near the feature point in the pyramid from the spatial position of the feature point to generate a descriptor by gradient histogram statistics, and the descriptor is normalized to form a final description vector; performing the above operations on all feature points in the image to generate a feature point description vector for the entire image.

[0057] Step 4, feature point matching. Establish a panoramic common path constraint: Assuming that the first panoramic camera and the second panoramic camera are coaxial, the center point of the first panoramic ring image is mapped to the corresponding radius of the second panoramic ring image according to the position transformation matrix of the two panoramic cameras. At this time, the global search is reduced to a two-dimensional radius search, which effectively shortens the calculation time and greatly improves the efficiency of feature point matching. The extremely small fan-shaped area where the two-dimensional radius is located is used as the search area to improve the accuracy of the matching results.

[0058] The balanced binary tree rule is used to find matching feature points. When the ratio of the minimum Euclidean distance to the second minimum Euclidean distance between the 128-dimensional feature description vectors of the key points of the two images is less than a certain threshold, the feature points are considered to match, and the matching results of all key points are output.

[0059] Step 5, calculating the three-dimensional coordinates. Converting the panoramic ring images generated by the first panoramic camera and the second panoramic camera to the same camera coordinate system; Figure 5 A simplified model diagram of three-dimensional reconstruction based on panoramic binocular stereo vision disclosed in an embodiment of the present invention is shown in FIG. Figure 5 As shown, object points P in the three-dimensional space are projected onto their respective image planes through the center points of the first panoramic cylindrical coordinate system 18 and the second panoramic cylindrical coordinate system 19 respectively. Based on the reverse projection model, the feature points on the image plane are reversely projected into direction vectors of the corresponding object points. The two direction vectors restored from each pair of feature points intersect, and the coordinate value of the intersection point solved by the least squares method is the three-dimensional coordinate of the actual object point. The three-dimensional coordinates of each pair of feature points are calculated to achieve panoramic three-dimensional reconstruction of sparse features.

[0060] The above is a detailed introduction to a three-dimensional reconstruction system and method based on panoramic binocular stereo vision disclosed in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0061] References:

[0062] [1]Adel'Son-Vel'Skii GM.An algorithms for the organization ofInformation[J].Soviet Math,1962,3.

[0063] [2]D.G.Lowe,et al.Object recognition from local scale-invariantfeatures[C],Proceedings of the Seventh IEEE International Conference onComputer Vision,1999,pp.1150-1157 vol.2。

Claims

1. A three-dimensional reconstruction method based on panoramic binocular stereo vision, characterized in that: The specific steps are as follows: Step 1, panoramic image acquisition and image processing; A panoramic binocular stereo vision system is placed in a scene, and two panoramic cameras are driven by a computer processing system to simultaneously shoot a 360° circular scene to obtain two panoramic images with a certain parallax; the panoramic images are filtered to remove noise; the blind areas of the panoramic images are cut to generate a panoramic circular image; Step 2: panoramic camera calibration; The panoramic camera imaging model is a plane-cylindrical projection model based on Taylor series expansion, including forward projection model and reverse projection model. The relationship between the three-dimensional point coordinates of a certain object point in space and its projection point coordinates in the panoramic ring image is determined through this projection model; the three-dimensional point coordinates in the world coordinate system and the two-dimensional point coordinates in the pixel coordinate system corresponding to all corner points of the chessboard are obtained through the chessboard calibration plate; the panoramic camera calibration parameters, including internal parameters, external parameters and distortion parameters, are obtained by the least squares method. These parameters will be used for the three-dimensional scene reconstruction of the panoramic binocular stereo vision system; Step 3, feature point extraction; The panoramic ring images generated by the two panoramic cameras are used as the original image and the target image respectively. The feature point extraction algorithm is used to detect the feature points of the original image and the target image, and the feature description vectors of all the feature points are generated. The feature description vectors will be used to calculate the feature similarity in the feature point matching process. Step 4, feature point matching; A panoramic common path constraint is established, and according to the position conversion matrix of the two panoramic cameras, the center point of the first panoramic annular image is mapped to the corresponding radius of the second panoramic annular image, and the search range is limited to the sector area where the corresponding effective radius is located; Use the balanced binary tree method to find matching feature points and output the matching results; Step 5, calculate the three-dimensional coordinates; The panoramic annular images generated by the two panoramic cameras are converted to the same camera coordinate system. Based on the reverse projection model, the feature points on the image plane are reversely projected into direction vectors of the corresponding object points. The two direction vectors restored from the two panoramic annular images intersect, and the coordinate value of the intersection point solved by the least squares method is the three-dimensional coordinate of the actual object point. The three-dimensional coordinates of each pair of feature points are calculated to achieve panoramic three-dimensional reconstruction of sparse features.

2. The three-dimensional reconstruction method based on panoramic binocular stereo vision according to claim 1, characterized in that: The algorithm flow of the panoramic camera calibration described in step 2 is as follows: input the panoramic annular image sequence of the calibration plate into the subsequent steps; determine and input the known parameters of the system, including the coordinates of the center point of the panoramic annular image and the actual size of the chessboard, and obtain the three-dimensional point coordinates of the corner points of the chessboard in the world coordinate system; extract the pixel coordinates of the corner points of the chessboard, that is, the two-dimensional point coordinates of the object point on the image plane; perform Taylor series expansion on the projection relationship between the camera coordinate system and the image plane coordinate system, and solve the internal and external parameters in turn by the linear least squares method to obtain the initial solution, and establish an ideal model without distortion; calculate the initial solution of the distortion parameter based on the difference between the ideal model and the actual value, and obtain the actual camera model; with the minimum reprojection error as the constraint condition, perform iterative nonlinear optimization on the initial solution of the internal parameters, external parameters and distortion parameters of the panoramic camera, and obtain the optimal solution as the geometric model parameters of the panoramic camera; and then calculate the position transformation matrix of the first panoramic camera and the second panoramic camera based on the external parameter matrix of the first panoramic camera and the second panoramic camera.

3. The three-dimensional reconstruction method based on panoramic binocular stereo vision according to claim 1, characterized in that: The feature point extraction described in step 3 adopts the SIFT algorithm, and the specific process is: construct an image pyramid and an image difference pyramid, and perform preliminary extreme value detection in the difference pyramid; perform Taylor series expansion near the detected extreme value points to achieve sub-pixel positioning; from the spatial position of the feature point back to the pyramid, sample a small area near the feature point to determine the main direction angle of the feature point; from the spatial position of the feature point back to the pyramid, sample a small area near the feature point to perform gradient histogram statistics to generate a descriptor, and the descriptor is normalized to form a final description vector; perform the above operations on all feature points in the image to generate a feature point description vector for the entire image.

4. A three-dimensional reconstruction system based on panoramic binocular stereo vision, using the three-dimensional reconstruction method based on panoramic binocular stereo vision as claimed in any one of claims 1 to 3, characterized in that: include: Two panoramic cameras, two multi-dimensional adjustment frames and a computer processing system; among which: The panoramic camera is composed of an image sensor and a secondary catadioptric panoramic lens, the relative positions of which are adjusted and fixed by screw threads, so that the panoramic lens forms an image on the image sensor, and the collected image information is transmitted to a computer processing system; The two panoramic cameras are respectively fixed on two multi-dimensional adjustment frames, and the multi-dimensional adjustment frames are adjusted so that the two panoramic cameras have the same optical axis and have overlapping fields of view. The two panoramic cameras are used to collect two panoramic images with parallax for three-dimensional reconstruction of a 360° ring scene. The computer processing system is used to drive the two panoramic cameras to collect panoramic images, calculate the calibration parameters of the two panoramic cameras, and complete the three-dimensional reconstruction of the 360° ring scene; The mapping relationship of the quadratic catadioptric panoramic lens adopts a plane-cylindrical projection model based on Taylor series expansion to project the 360° annular scene onto a two-dimensional plane. The optical system of the secondary catadioptric panoramic lens is composed of a panoramic annular lens, an aperture stop and a relay lens group arranged in sequence along the optical axis. The incident light is reflected and refracted twice in the panoramic annular lens in sequence, and then passes through the aperture stop and the relay lens group, and is finally received by the image sensor to generate a panoramic image. The image sensor outputs the panoramic image to a computer processing system for three-dimensional reconstruction of a 360° circular scene.

Images