A method for calibrating structural parameters of a dual-prism virtual binocular vision system

Abstract

The present application relates to a kind of structure parameter optimization calibration methods for double-prism virtual binocular vision system, steps are as follows: S1: adjusting the relative position of camera and double-prism;S2: obtaining the stereo image of multiple sets of calibration plate;S3: obtaining the internal participation of virtual binocular camera and structure parameter initial value;S4: obtaining stereo image sequence, establish the spatial measurement field consisting of distributed regular feature points;S5: using the internal parameter of virtual binocular camera obtained in step S3 to carry out distortion correction to the sub-image corresponding to left and right virtual camera;S6: obtaining feature point matching point pair;S7: the three-dimensional coordinates of feature point in measurement field;S8: construct the objective function based on spatial distance error and polar line constraint, nonlinear iterative optimization is carried out to structure parameter, and the three-dimensional coordinates of feature point in step S7 will be used to update each time iteration calculation obtained structure parameter.This application method can meet the comprehensive needs of double-prism virtual binocular system in three-dimensional measurement and two-dimensional epipolar geometry constraint.

Description

Technical Field

[0001] This invention belongs to the field of image digitization processing technology and relates to visual calibration methods, specifically a structural parameter optimization and calibration method for a dual-prism virtual binocular vision system. Background Technology

[0002] Virtual binocular vision systems based on the principle of biprism refraction are widely used in fields such as electric arc furnace topography measurement and stereo microscopy due to their compact structure and absence of binocular synchronous imaging errors. The stereo image refracted by the biprism and acquired by a single camera can be equivalent to a pair of binocular stereo images captured by two virtual cameras, hence the name virtual binocular vision system. The imaging principle of this system is as follows: Figure 1 As shown.

[0003] As is well known, high-precision calibration is crucial for 3D vision measurement. Previous research has shown that applying traditional binocular calibration methods to bi-prism virtual binocular systems is feasible and effective. Traditional binocular calibration methods typically involve two cameras acquiring images of the same target and extracting feature points. Initial values ​​for intrinsic and structural parameters are obtained through linear methods, and finally, error constraints are used to perform nonlinear optimization of the initial calibration values. Research in this area generally focuses on introducing different error constraints and employing different optimization objective functions to perform nonlinear optimization calibration of the parameters. Furthermore, the idea of ​​treating a bi-prism-based single-lens stereo vision system as a virtual binocular system is somewhat idealistic; for example, this virtual binocular system may lack a physically defined principal camera point. Therefore, directly applying traditional calibration methods to a virtual binocular system will correspondingly limit its measurement accuracy.

[0004] Compared to traditional binocular stereo vision systems, dual-prism virtual binocular vision systems are more suitable for measurement within a certain field of view, where the relative position of the object being measured and the vision system is relatively fixed. For example, a search revealed that Chinese patent CN108830906B provides an automatic camera parameter calibration method based on the principle of virtual binocular vision. The device includes a gimbal rotation bracket, a camera, an external trigger circuit, a PC, a calibration plate with a black and white checkerboard pattern, and two plane mirrors positioned at an angle. The two plane mirrors are located behind the gimbal rotation bracket; the calibration plate is placed on the gimbal rotation bracket; the camera is positioned in front of the gimbal rotation bracket with its image facing the two plane mirrors. When the camera captures an image of the calibration plate, the captured image includes two virtual images of the calibration plate formed in the two plane mirrors. Each time the calibration plate rotates by an angle, the external trigger circuit controls the camera to capture an image of the calibration plate. The PC processes the images of the calibration plate captured by the camera to obtain and calibrate the camera's internal parameters, distortion parameters, and external parameters in the world coordinate system.

[0005] Chinese patent CN114111637A discloses a method for 3D reconstruction based on virtual binocular striped structured light, including: constructing a virtual binocular striped structured light vision system based on dual prisms; calibrating the virtual binocular vision system to obtain the intrinsic and extrinsic parameters of the two cameras; projecting a three-frequency, four-step sinusoidal striped image onto the object; obtaining the unfolded phase corresponding to the left and right virtual cameras respectively using a multi-frequency heterodyne unwrapping method; performing epipolar correction on the unfolded phase based on the intrinsic and extrinsic parameters of the virtual cameras; performing stereo matching on the left and right unfolded phases to obtain the disparity of the matching points corresponding to the left and right unfolded phases; combining the intrinsic and extrinsic parameters of the virtual binoculars and using the disparity principle to convert the disparity into depth information; generating a dense point cloud map of the object based on its 3D spatial coordinates to complete the 3D reconstruction of the object. The two invention patents mentioned above aim to calibrate parameters by replacing traditional binocular systems with virtual binocular vision systems. However, the calibration methods are too cumbersome. For example, CN107121109A describes a structured light parameter calibration device and method based on a front-coated plane mirror, which includes: placing the front-coated plane mirror and a plane glass target in front of a camera, with the camera simultaneously capturing images of the plane glass target and its mirror image; establishing a virtual binocular measurement model; obtaining the optimized solution of the rotation matrix and translation vector from the coordinate system of the front-coated plane mirror to the coordinate system of the camera using a nonlinear optimization method; and using the least squares method to obtain the image ablation points of candidate feature points. A white printing paper is placed in front of the front-coated plane mirror, with the camera simultaneously capturing images of the actual light stripe and its mirror image. The center point of the light stripe image is extracted, and the three-dimensional coordinates of the matching point are calculated to solve the equation of the light plane. This invention improves the quality of light stripes, enhances the accuracy of light stripe center extraction, and provides feature points with micron-level positional accuracy. It obtains a greater number of calibration points, resulting in higher calibration accuracy and more stable calibration results. However, the calibration object is not a virtual binocular vision system with a biprism or similar structure, and the algorithmic logic involved in the calibration differs. Therefore, by inventing a biprism virtual binocular calibration method that combines the characteristics of a biprism virtual binocular system, it plays a crucial role in improving the three-dimensional measurement accuracy of this system. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a method for optimizing and calibrating the structural parameters of a dual-prism virtual binocular vision system, so as to improve the three-dimensional measurement accuracy of the system.

[0007] The technical problem solved by this invention is achieved through the following technical solution:

[0008] A method for optimizing and calibrating structural parameters for a dual-prism virtual binocular vision system.

[0009] S1: Adjust the relative position of the camera and the double prism so that the double prism virtual binocular vision system can acquire uniformly segmented left and right sub-images;

[0010] S2: Place the calibration board at different positions within the depth range of the dual-prism virtual binocular vision system to obtain multiple sets of stereo images of the calibration board;

[0011] S3: The original stereo image of the calibration board is uniformly divided to obtain two sub-images of the same size. Feature points are extracted from the left and right sub-images respectively. The dual-prism virtual binocular vision system is coarsely calibrated to obtain the initial values ​​of the intrinsic and structural parameters of the virtual binocular camera.

[0012] S4: Place the measuring board in front of the vision system, move the measuring board multiple times at a fixed distance and acquire a sequence of stereo images to establish a spatial measurement field composed of regularly distributed feature points;

[0013] S5: Uniformly divide several sets of stereo images in the spatial measurement field, and use the intrinsic parameters of the virtual binocular camera obtained in step S3 to perform distortion correction on the sub-images corresponding to the left and right virtual cameras.

[0014] S6: Extract the center feature points on the distortion-corrected measurement board image. Based on the order of the center feature point sequence, match the center feature points of the left and right sub-images to obtain feature point matching pairs.

[0015] S7: Combining the intrinsic structural parameters of the virtual binocular camera, the three-dimensional coordinates of feature points in the spatial measurement field are recovered using linear triangulation.

[0016] S8: Construct an objective function based on spatial distance error and epipolar constraints, perform nonlinear iterative optimization on the structural parameters, and use the structural parameters obtained in each iteration to update the three-dimensional coordinates of the feature points in step S7.

[0017] Moreover, the aforementioned dual-prism virtual binocular vision system, where the stereoscopic image refracted by the dual prisms and acquired by a single camera, can be equivalent to a pair of binocular stereoscopic vision images captured by two virtual cameras.

[0018] Moreover, the specific steps of S1 are as follows: using a precision displacement stage to adjust the position of the camera and the biprism so that the center line of the image in the vertical direction is close to coincide with the edge of the center of the biprism.

[0019] Furthermore, the biprism virtual binocular vision system was coarsely calibrated using Zhang Zhengyou's calibration method.

[0020] Moreover, the specific steps of S4 are as follows: a measuring plate with a regularly arranged circular target on its surface is fixed on the moving platform of a high-precision electric displacement stage. The spatial position of the measuring plate is relatively parallel to the imaging plane of the camera. The controller is operated to move the measuring plate multiple times and in a fixed direction by a fixed distance, while acquiring a three-dimensional image of the measuring plate at each displacement position.

[0021] Moreover, the specific steps of S8 are as follows:

[0022] S8.1: For any control point within the spatial measurement field volume, the distance between fixed points in six directions is measured. The error function based on spatial distance constraints is designed as follows:

[0023]

[0024] Lq (q = 1, 2, ..., 6) represents the measured length between the specified control point and the q-th feature point around it. The true value of its length is Dq (q = 1, 2, ..., 6), where i and j represent the displacement order of the measuring plate and the center order of the feature points on the measuring plate, respectively.

[0025] S8.2: Definition Given a set of homogeneous coordinates of matching points, the distance from each point to its corresponding epipolar line l is given. l With l r The distances are dis l ,dis r The error function based on epipolar constraints is constructed as follows:

[0026]

[0027] in, for The corresponding left pole line, for The corresponding right polar line, F represents the fundamental matrix of the virtual stereo system, i.e.: F = R·S, R = [r 11 ,r 12 ,r 13 ;r 21 ,r 22 ,r 23 ;r 31 ,r 32 ,r 33 [] represents the rotation matrix of the virtual stereoscopic view, and S represents the antisymmetric matrix of the virtual stereoscopic translation matrix T = [t1, t2, t3].

[0028]

[0029] S8.3: Combining the two error functions to form the objective function:

[0030] H obj (x)=arg min(E L +E P )

[0031] The input variables of the objective function are the structure parameters, i.e.: x = (r 11 ,r 12 ,r 13 ,r21 ,r 22 ,r 23 ,r 31 ,r 32 ,r 33 The structural parameters are iteratively solved using the SQP (Sequential Quadratic Programming) nonlinear optimization operator (t1, t2, t3). The iteration ends when the local minimum value of the objective function is output.

[0032] Moreover, the specific steps of S3 are as follows:

[0033] S3.1: Divide the original stereo image of the calibration board evenly along the vertical midline to obtain two calibration board sub-images I with the same pixel size. L I R The left and right sub-images are respectively regarded as left and right virtual cameras VC. L VC R Photos taken;

[0034] S3.2: Specify the world coordinates of feature points on the calibration plate. And extract the pixel coordinates corresponding to the feature point. Establish the mapping relationship between the 2D pixel coordinates and 3D world coordinates of feature points on each calibration plate image. Taking the left virtual camera as an example, obtain the transformation relationship between the world coordinates and pixel coordinates of feature points:

[0035]

[0036] Where i and j represent the captured calibration board sequence and the feature point sequence on the calibration board, respectively, and K L Let be the intrinsic parameter matrix of the left virtual camera. for Representing VC L The rotation and translation matrices of the optical center relative to the i-th calibration plate;

[0037] S3.3: Obtain VC using Zhang Zhengyou's calibration method L VC R The intrinsic parameter matrix, distortion coefficients, and VC corresponding to each calibration plate image L With VC R The coordinate transformation relationship between optical centers, i.e., the virtual binocular structure parameters, is the result of averaging multiple sets of virtual binocular structure parameters as the virtual binocular structure parameters calibrated by the system.

[0038] Furthermore, the structural parameters obtained in each iteration will be combined with the virtual camera intrinsic parameters obtained during coarse calibration to update the three-dimensional coordinates of the control points in the spatial measurement field. The new structural parameters and the three-dimensional coordinates of the feature point array will be used for the next iteration calculation.

[0039] Furthermore, to correct lens distortion, traditional radial and tangential distortions are introduced into the virtual binocular vision system. Taking the left virtual camera as an example, its distortion model is defined as follows:

[0040]

[0041] in With m nl (x lm ,y lm ) represent the coordinates of the distorted point and the undistorted point on the normalized plane, respectively, and k l(1-2) With p l(1-2) Each represents the radial and tangential distortion coefficients.

[0042] An electronic device, characterized in that it includes a processor and a memory, the processors being communicatively connected to each other, the memory storing computer instructions, and the processors executing the computer instructions to perform any of the methods described above.

[0043] The advantages and positive effects of this invention are:

[0044] This invention is applicable to the calibration of virtual binocular vision systems based on biprisms. Since a biprism virtual binocular system is a special type of binocular vision system, traditional binocular calibration methods have relatively limited effectiveness when applied to it. Furthermore, while binocular vision calibration methods based on objective function design can improve calibration accuracy, these methods also have significant drawbacks, such as: using complex constraints makes it difficult to determine the constraint types that significantly contribute to improving calibration accuracy; and too many optimization parameters can easily lead to coupled solutions. Therefore, this invention has the following advantages:

[0045] (1) The calibration method proposed in this invention fully combines the characteristics of the dual-prism virtual binocular vision system, and calibrates and optimizes the vision system within a limited space defined by human, thereby improving the calibration quality.

[0046] (2) The objective function design based on structural parameter optimization proposed in this invention is simpler. By combining spatial distance constraints with epipolar constraints, the comprehensive requirements of the dual-prism virtual binocular system in terms of three-dimensional measurement and two-dimensional epipolar geometric constraints are met. Attached Figure Description

[0047] Figure 1This is a schematic diagram of the imaging principle of a dual-prism virtual binocular system;

[0048] Figure 2 This is a flowchart illustrating the overall implementation of the dual-prism virtual dual-target positioning method in this embodiment of the invention.

[0049] Figure 3 This is a schematic diagram of constructing a spatial measurement field in an embodiment of the present invention;

[0050] Figure 4 This is a flowchart of the nonlinear optimization method for structural parameters in an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of spatial distance error measurement in an embodiment of the present invention;

[0052] Figure 6 These are schematic diagrams of epipolar constraint error in embodiments of the present invention. (a) is a schematic diagram of epipolar geometry, and (b) is a diagram defining epipolar distance. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer and easier to understand, the present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0054] This invention provides a method for optimizing and calibrating structural parameters of a dual-prism virtual binocular vision system. The overall calibration process is as follows: Figure 2 As shown. The overall process of this invention is as follows:

[0055] Step 1: Construct a virtual binocular stereo vision system based on biprism refraction. Adjust the relative positions of the camera and biprisms using a precision displacement stage, ensuring the biprisms are parallel to the camera's imaging plane and that the vision system acquires uniformly segmented left and right sub-images. The imaging principle of the biprism virtual binocular vision system is as follows: Figure 1 As shown, the acquired stereo image is uniformly divided along the image centerline to obtain two sub-images of the same pixel size. The left and right sub-images can be considered as two different left (VC) positions. L ), right (VC) R (Taken by a virtual camera)

[0056] Step 2: Randomly place the calibration plate with solid dots on its surface at different positions in the common field of view of the virtual binoculars to obtain stereo calibration plate images.

[0057] Step 3: Uniformly divide the calibration board image obtained in Step 2 along the midline of the image, and extract the three-dimensional coordinates of the center feature points on each calibration board. with two-dimensional pixel coordinates Taking the left virtual camera as an example, establish the mapping relationship between the world coordinates and pixel coordinates of the feature points on the calibration board:

[0058]

[0059] Where i and j represent the sequence of calibration board images and the sequence of feature points on the calibration board, respectively. K L Let be the intrinsic parameter matrix of the left virtual camera. for Representing VC L The rotation and translation matrices of the optical center relative to the i-th calibration plate. Furthermore, to correct lens distortion, this example introduces traditional radial and tangential distortion into the virtual binocular vision system. The distortion model is as follows:

[0060]

[0061] in Let m be the coordinates of the distorted point on the normalized plane. nl (x lm ,y lm ) represents the coordinates of the distortion-removed point on the normalized plane. k l(1-2) With p l(1-2) Each represents the radial and tangential distortion coefficients.

[0062] Step 4: Based on the virtual camera imaging model described in Step 3, perform coarse calibration on the left and right virtual cameras using the Zhang Zhengyou calibration method. The coarse calibration yields the intrinsic parameter matrices, distortion coefficients, and VC values ​​corresponding to each calibration board image for each virtual camera. L With VC R The coordinate transformation relationship between optical centers, i.e., the structural parameters of the virtual binoculars. This invention uses the average value of multiple sets of structural parameters as the initial values ​​for the structural parameters during system calibration.

[0063] Step 5: As Figure 3 As shown, a measuring plate with 10×21 surfaces arranged in a regular pattern is vertically fixed on a high-precision displacement stage. The displacement direction of the measuring plate is perpendicular to the imaging plane of the camera. The displacement stage is manipulated to move the measuring plate 1 mm each time, for a total of 51 movements. Each circular feature point on the measuring plate is considered a control point. Based on the linear displacement of the measuring plate, a spatial measuring field with a regular arrangement and known distances between adjacent control points is constructed in the common field of view of a virtual binocular camera.

[0064] Step 6: Uniformly segment the stereo images used to construct the spatial measurement field. Based on the intrinsic parameter results obtained in Step 4, perform distortion correction on the left and right view measurement board images and extract the pixel coordinates of the control points. Obtain the matching coordinates of the control points in the spatial control field based on the arrangement order of the control points on the measurement board and the displacement order of the measurement board in three-dimensional space. Then, construct the objective function to perform nonlinear optimization of the structural parameters. The specific process is as follows: Figure 4 As shown.

[0065] Step 7: Use linear trigonometry to recover the 3D coordinates of each control point with the optical center of the left camera as the coordinate system, and use this to establish a constraint function based on spatial distance error. The definition of spatial distance error is as follows: Figure 5 As shown, a control point in the measurement space is selected, and the spatial distances between this control point and six line segments on the plane of the measurement board—M1 (left-upward), M2 (right-upward), M3 (left-downward), M4 (right-downward), and M5 (forward) and M6 (backward) relative to the measurement board—are measured. The error function based on the spatial distance constraint is designed as follows:

[0066]

[0067] Lq (q = 1, 2, ..., 6) represents the measured length between a specified control point and its qth surrounding control point, and the true value of its length is Dq (q = 1, 2, ..., 6). i and j represent the displacement order of the measuring plate and the control point order on the measuring plate, respectively.

[0068] Step 7: Design constraint functions based on epipolar constraints. Figure 6 (a) illustrates the epipolar geometry model of a binocular vision system. VC L VC R The optical center is O cl With O cr Connect O cl With O cr The line segment is defined as the baseline of the binocular system, and its intersections with the imaging planes of the left and right virtual cameras are the left and right poles, respectively: e l e r . These are the homogeneous coordinates of the matching point pairs in the pixel coordinate system. The left and right epipolar lines corresponding to the matching points can be represented as: F represents the fundamental matrix of the virtual binocular system, i.e., F = R·S.

[0069] R = [r] 11 ,r 12 ,r 13 ;r 21 ,r 22 ,r 23 ;r 31,r 32 ,r 33 [] represents the virtual binocular rotation matrix, and S represents the antisymmetric matrix of the translation matrix T = [t1, t2, t3].

[0070]

[0071] Ideally, the matching point of a given point should lie on the epipolar line calculated from the pixel coordinates of that point, meaning the distance from the matching point to its corresponding epipolar line is 0. However, this is not the case in reality. There is always a certain vertical distance between the matching point and its corresponding epipolar line. The smaller this vertical distance, the better the structural parameters conform to the epipolar geometry constraints of the current stereo vision system. Figure 6 (b) illustrates the epipolar distance error, defining... to the polar line l l With l r The distances are dis l ,dis r The error function based on epipolar constraints is designed as follows:

[0072]

[0073] Where i and j represent the displacement order of the measuring plate and the arrangement order of the control point array on the measuring plate, respectively.

[0074] Step 8: Steps 6 and 7 above construct the error constraint function based on spatial distance error and epipolar constraints. Combining the two constraint functions and constructing the objective function is as follows:

[0075] H obj (x)=arg min(E L +E P ).

[0076] Here, there are 12 variables that need to be optimized, namely: x = (r 11 ,r 12 ,r 13 ,r 21 ,r 22 ,r 23 ,r 31 ,r 32 ,r 33(t1, t2, t3). The objective function is solved using the Sequential Quadratic Programming (SQP) method. The nonlinear optimization of the structural parameters is an iterative process. The error output from each iteration is used to determine the iteration cutoff condition. If the cutoff condition is not met, the structural parameters output from this iteration are used as the initial values ​​for the next iteration, and the 3D coordinates of the control points in the space measurement field are updated based on the current structural parameters. When the objective function output reaches the cutoff condition, the current structural parameters are considered to have reached a local optimum in the nonlinear optimization.

[0077] The present invention also provides an electronic device, comprising:

[0078] A memory and a processor, which are communicatively connected to each other, wherein the memory stores computer instructions, and the processor executes the computer instructions to perform any of the methods described above.

[0079] The present invention also provides a computer storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to perform any of the methods described above.

[0080] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0081] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for optimizing and calibrating structural parameters of a dual-prism virtual binocular vision system, characterized in that: S1: Adjust the relative position of the camera and the biprism so that the biprism virtual binocular vision system can acquire uniformly segmented left and right sub-images; S2: Place the calibration board at different positions within the depth range of the dual-prism virtual binocular vision system to obtain multiple sets of stereo images of the calibration board; S3: The original stereo image of the calibration board is uniformly divided to obtain two sub-images of the same size. Feature points are extracted from the left and right sub-images respectively. The dual-prism virtual binocular vision system is coarsely calibrated to obtain the initial values ​​of the intrinsic and structural parameters of the virtual binocular camera. S4: Place the measuring board in front of the vision system, move the measuring board multiple times at a fixed distance and acquire a sequence of stereo images to establish a spatial measurement field composed of regularly distributed feature points; S5: Uniformly segment several sets of stereo images in the spatial measurement field, and use the intrinsic parameters of the virtual binocular camera obtained in step S3 to perform distortion correction on the sub-images corresponding to the left and right virtual cameras. S6: Extract the center feature points on the distortion-corrected measurement board image. Based on the order of the center feature point sequence, match the center feature points of the left and right sub-images to obtain feature point matching pairs. S7: Combining the intrinsic structural parameters of the virtual binocular camera, the three-dimensional coordinates of feature points in the spatial measurement field are recovered using linear triangulation. S8: Construct an objective function based on spatial distance error and epipolar constraints, perform nonlinear iterative optimization on the structural parameters, and use the structural parameters obtained in each iteration to update the three-dimensional coordinates of the feature points in step S7.

2. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 2, characterized in that: The biprism virtual binocular vision system is a stereoscopic image obtained by a single camera after being refracted by a biprism, which can be equivalent to a pair of binocular stereoscopic vision images taken by two virtual cameras.

3. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 1, characterized in that: The specific steps of S1 are as follows: use a precision displacement stage to adjust the position of the camera and the biprism so that the center line of the image in the vertical direction is close to coincide with the edge of the center of the biprism.

4. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 1, characterized in that: The biprism virtual binocular vision system was coarsely calibrated using Zhang Zhengyou's calibration method.

5. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 1, characterized in that: The specific steps of S4 are as follows: a measuring plate with regularly arranged circular targets on its surface is fixed on the moving platform of a high-precision electric displacement stage. The spatial position of the measuring plate is relatively parallel to the imaging plane of the camera. The controller is operated to move the measuring plate multiple times and in a fixed direction by a fixed distance, while acquiring a three-dimensional image of the measuring plate at each displacement position.

6. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 1, characterized in that: The specific steps of S8 are as follows: S8.1: For any control point within the spatial measurement field volume, the distance between fixed points in six directions is measured. The error function based on spatial distance constraints is designed as follows: Lq (q = 1, 2, ..., 6) represents the measured length between the specified control point and the q-th feature point around it. The true value of its length is Dq (q = 1, 2, ..., 6), where i and j represent the displacement order of the measuring plate and the center order of the feature points on the measuring plate, respectively. S8.2: Definition Given a set of homogeneous coordinates of matching points, the distance from each point to its corresponding epipolar line l is given. l With l r The distances are dis l ,dis r The error function based on epipolar constraints is constructed as follows: in, for The corresponding left pole line, for The corresponding right polar line, F represents the fundamental matrix of the virtual stereo system, i.e.: F = R·S, R = [r 11 ,r 12 ,r 13 ;r 21 ,r 22 ,r 23 ;r 31 ,r 32 ,r 33 [] represents the rotation matrix of the virtual stereoscopic view, and S represents the antisymmetric matrix of the virtual stereoscopic translation matrix T = [t1, t2, t3]. S8.3: Combining the two error functions to form the objective function: H obj (x)=arg min(E L +E P ) The input variables of the objective function are the structure parameters, i.e.: x = (r 11 ,r 12 ,r 13 ,r 21 ,r 22 ,r 23 ,r 31 ,r 32 ,r 33 The structural parameters are iteratively solved using the SQP (Sequential Quadratic Programming) nonlinear optimization operator (t1, t2, t3). The iteration ends when the local minimum value of the objective function is output.

7. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 4, characterized in that: The specific steps of S3 are as follows: S3.1: Divide the original stereo image of the calibration board evenly along the vertical midline to obtain two calibration board sub-images I with the same pixel size. L I R The left and right sub-images are respectively regarded as left and right virtual cameras VC. L VC R Photos taken; S3.2: Specify the world coordinates of feature points on the calibration plate. And extract the pixel coordinates corresponding to the feature point. Establish the mapping relationship between the 2D pixel coordinates and 3D world coordinates of feature points on each calibration plate image. Taking the left virtual camera as an example, obtain the transformation relationship between the world coordinates and pixel coordinates of feature points: Where i and j represent the captured calibration board sequence and the feature point sequence on the calibration board, respectively, and K L The intrinsic parameter matrix of the left virtual camera. for Representing VC L The rotation and translation matrices of the optical center relative to the i-th calibration plate; S3.3: Obtain VC using Zhang Zhengyou's calibration method L VC R The intrinsic parameter matrix, distortion coefficients, and VC corresponding to each calibration plate image L With VC R The coordinate transformation relationship between optical centers, i.e., the structure parameters of the virtual binoculars, is the result of averaging multiple sets of virtual binocular structure parameters as the structure parameters of the virtual binoculars calibrated by the system.

8. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 6, characterized in that: The structural parameters obtained in each iteration will be combined with the virtual camera intrinsic parameters obtained during coarse calibration to update the three-dimensional coordinates of the control points in the spatial measurement field. The new structural parameters and the three-dimensional coordinates of the feature point array will be used for the next iteration calculation.

9. The structural parameter optimization and calibration method for a dual-prism virtual binocular vision system according to claim 7, characterized in that: To correct lens distortion, traditional radial and tangential distortions are introduced into the virtual binocular vision system. Taking the left virtual camera as an example, its distortion model is defined as follows: in With m nl (x lm ,y lm ) represent the coordinates of the distorted point and the undistorted point on the normalized plane, respectively, and k l(1-2) With p l(1-2) Each represents the radial and tangential distortion coefficients.

10. An electronic device, characterized in that: The method includes a processor and a memory, the processors being interconnected and the memory storing computer instructions, the processors executing the computer instructions to perform the method of any one of claims 1 to 9.

Images