A calibration method and system for dynamic binocular stereo vision using only a single feature point

Abstract

The application provides a kind of only using single feature point to realize dynamic binocular stereo vision's calibration method and system, it is related to binocular stereo vision field;Including: building non-zoom dynamic BSV model, pre-determining the initial relative position of the camera in model by Zhang method;Real-time through pose angle measuring instrument obtains the approximate value of rotation angle, and only uses single rectangular feature center point to iteratively solve the rotation angle of camera according to the homography between images;According to the characteristics of fundamental matrix, the corresponding rectangular center point pair obtained by the left and right cameras after rotation is corrected for pitch angle and yaw angle, and finally the determined rotation angle and the known initial relative position between left and right cameras can be used to calculate the extrinsic parameters of dynamic BSV after rotation.The application avoids using high-precision calibration objects with a certain size, and the rectangular feature is very common in daily life, the method has low cost, and is suitable for fast online calibration of dynamic BSV extrinsic parameters in occasions that do not require very high accuracy.

Description

Technical Field

[0001] This invention relates to the field of binocular stereo vision technology, and specifically to a calibration method and system for achieving dynamic binocular stereo vision using only a single feature point. Background Technology

[0002] With the rapid development of industrial cameras and image processing technology, stereo vision is widely used for target localization, ranging, and 3D structure measurement. Dynamic binocular stereo vision (BSV), thanks to the camera's free rotation capability, can expand the visual measurement range, thereby enabling the tracking and localization of moving targets.

[0003] Rotating cameras are a crucial component of dynamic binocular stereo vision, with rotation only in the horizontal and vertical directions used to adjust the field of view (FOV). Camera calibration is essential for accurately locating moving targets. Camera calibration methods mainly include traditional and self-calibration methods. Traditional methods require the use of known 3D coordinates of control points and corresponding pixel coordinates on the image plane. While offering high accuracy, they require a large number of control points, which must be kept within the FOV, making them undesirable in dynamic BSV scenarios where the common FOV changes frequently. Self-calibration methods are less robust than traditional methods but are suitable for calibrating rotating cameras. However, feature point mismatches can cause calibration results to deviate from the true values. High-precision results require robust algorithms and long computation times. To address feature point mismatch issues, existing technologies propose self-calibration methods based on structured light or different target types, but these algorithms are generally time-consuming. Therefore, a method is needed that can quickly perform online calibration of rotating cameras while simultaneously ensuring accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a calibration method and system for dynamic binocular stereo vision using only a single feature point. It proposes to use a single rectangular feature center point of unknown size to achieve online calibration of dynamic BSV extrinsic parameters, avoiding the use of high-precision calibration objects with specific dimensions. This method is low-cost and suitable for online calibration of dynamic BSV extrinsic parameters when extremely high precision is not required.

[0005] To achieve the above objectives, the present invention proposes the following technical solution:

[0006] Firstly, a calibration method for dynamic binocular stereo vision using only a single feature point is proposed, comprising the following steps:

[0007] A non-zoom dynamic BSV model is constructed. The intrinsic parameters and initial relative positions of the left and right cameras in the BSV model are determined in advance using Zhang's method. Each camera has horizontal and vertical rotation degrees of freedom under the condition that its position is fixed.

[0008] An attitude angle measuring instrument is used to obtain approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time. The rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction.

[0009] Based on the homography between images, the approximate value of the rotation angle of each camera is obtained as the initial value, and the rotation angle corresponding to each camera is solved iteratively by least squares using a single feature point.

[0010] Based on the rotation angles of the left and right cameras, calculate the rotation matrix and translation vector of the BSV model after rotation;

[0011] Based on visual geometry, a basic matrix is ​​established to associate corresponding image points, and an unconstrained nonlinear optimization algorithm is used to calculate the deviation of the rotation angles of the left and right cameras in the BSV model.

[0012] Based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the external parameters of the rotated BSV model are calculated, including the final rotation matrix and the final translation vector.

[0013] Furthermore, the coordinate systems of the left and right cameras before rotation in the BSV model are defined as O1 and O2 respectively. L -X l Y l Z l and O R -X r Y r Z r After rotation, the coordinate systems of the left and right cameras are O and O respectively. L -X L Y L Z L and O R -X R Y R Z R The transformation relationship between the left and right cameras before and after the BSV model rotation, and the transformation relationship between the left and right cameras relative to themselves before and after the BSV model rotation, are as follows:

[0014] The mapping relationship between the left and right camera coordinate systems before and after the BSV model rotation is as follows:

[0015] P r =R0P l +T0,P R =RP L +T;

[0016] Among them, P l P r and P L P RThe left and right camera coordinate systems before and after the BSV model rotation are represented respectively. R0 and R represent the rotation matrices of the BSV model before and after rotation, and T0 and T represent the translation vectors of the BSV model before and after rotation.

[0017] According to the rotation theorem, the rotation matrices of the left and right cameras of the BSV model relative to themselves before and after rotation are R0 and R1, respectively. l R r Then the rotation matrix R of the left camera l for:

[0018]

[0019] Right camera rotation matrix R r for:

[0020]

[0021] Where, p l y l R represents the pitch and yaw angles of the left camera, respectively. x (p l ) indicates that the left camera rotates around the x-axis. l The rotation matrix, R y (y l ) indicates that the left camera rotates around the y-axis. l rotation matrix of p; r y r R represents the pitch and yaw angles of the right camera, respectively. x (p r ) indicates that the right camera rotates about the x-axis. r The rotation matrix, R y (y r ) indicates that the right camera rotates about the y-axis. r The rotation matrix; C represents the trigonometric function cos, and S represents the trigonometric function sin.

[0022] Furthermore, based on image homography, the approximate rotation angle of each camera is obtained as the initial value. The process of iteratively solving for the rotation angle of each camera using a single feature point through the least squares method includes:

[0023] In the BSV model, the left camera coordinate system is considered the world coordinate system. The projection points of spatial point Q onto the image planes of the left and right camera coordinate systems of the BSV model before and after rotation are q0, respectively. l (u l ,v l ),q r (u r ,v r ) and q L (u L ,v L),q R (u R ,v R Then, according to the principle of geometric imaging, we can obtain:

[0024]

[0025] Among them, Z l Z L Let U and V represent the depths of spatial point Q in the left camera coordinate system before and after rotation of the BSV model; U0 and V0 represent the two-dimensional coordinates of spatial point Q on the image plane away from the principal point before rotation, and U and V represent the two-dimensional coordinates of spatial point Q on the image plane away from the principal point after rotation, and U0 = u l -c x V0 = v l -c y U = u L -c x V = v L -c y , (c x ,c y () represents the principal point coordinates of the left camera; K1 and K2 represent the intrinsic parameter matrices of the left and right cameras in the BSV model, respectively, f xL f yL These represent the focal lengths of the left camera in the horizontal and vertical directions, respectively. xR f yR These represent the focal lengths of the right camera in the horizontal and vertical directions, respectively.

[0026] In the BSV model, the homography matrices between the images before and after the left and right cameras rotate are defined as H1 and H2, respectively, where H1 = K1R. l K1 -1 H2 = K2R r K2 -1 Then, by associating the projection points of the spatial points Q of the BSV model before and after rotation onto the left camera image plane using the homography matrix H1, we can obtain:

[0027]

[0028] Where λ is the scaling factor, equal to Z L / Z l ;

[0029] Based on the correspondence between single feature points before and after rotation of the left and right cameras in the BSV model, the equation of the homography matrix H1 is:

[0030]

[0031] Among them, h mn(m,n=1,2,3) represents the value of the m-th row and n-th column of matrix H1;

[0032] The pitch angle p of the left camera is solved iteratively using the least squares method with a single feature point. l and yaw angle y l The equation is:

[0033]

[0034] Similarly, the pitch angle p of the right camera can be solved using the homography matrix H2. r and yaw angle y r .

[0035] Furthermore, based on the solved rotation angles of the left and right cameras, the process of calculating the rotation matrix and translation vector of the rotated BSV model includes:

[0036] The mapping relationship between the left and right camera coordinate systems before and after the BSV model rotation is adjusted as follows:

[0037] P R =R r R0R l -1 P L +R r T0;

[0038] After rotation, the rotation matrix R and translation vector T of the BSV model are respectively:

[0039] R = R r R0R l -1 T = R r T0.

[0040] Furthermore, the precise pitch and yaw angles of the left camera are defined as follows: The precise pitch and yaw angles of the right camera are... but:

[0041]

[0042] Where, Δp l Δy l Δp r and Δy r These represent the deviations in pitch and yaw angles of the left and right cameras, respectively.

[0043] In the BSV model, the left and right cameras have their self-corrected rotation matrices. and They are represented as follows:

[0044]

[0045] After associating the corresponding image points using the fundamental matrix F, the homogeneous coordinates of spatial point Q on the left and right camera imaging planes after rotation of the BSV model are U and U, respectively. l =[u L ,v L ,1] T U r =[u R ,v R ,1] T ,but:

[0046] U r T FU l =0,

[0047] in,

[0048] , (c′ x, c′ y ) represents the principal point coordinates of the right camera, t x t y t z These are the components of the translation vector T after rotation of the BSV model along each axis, i.e., T = [t] x ,t y ,t z ] T ;

[0049] Furthermore, the deviation Δp between the pitch and yaw angles of the left and right cameras l Δy l Δp r and Δy r The calculation is performed using an unconstrained nonlinear optimization method:

[0050] F(Δp l ,Δy l ,Δp r ,Δy r )=|U r T FU l | 2 →min.

[0051] Furthermore, based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the process of calculating the external parameters of the rotated BSV model is as follows:

[0052] The final rotation matrix of the BSV model after rotation is defined as follows: The final translation vector is but:

[0053]

[0054] Furthermore, the pitch angle measurement accuracy of the attitude angle measuring instrument is 0.2°, the yaw angle measurement accuracy is 1°, and the measurement range for all axes is 360°.

[0055] Secondly, a calibration system for dynamic binocular stereo vision using only a single feature point is proposed, including:

[0056] A determination module is built to construct a non-zoom dynamic BSV model. The intrinsic parameters and initial relative positions of the left and right cameras in the BSV model are determined in advance using Zhang's method. Each camera has horizontal and vertical rotational degrees of freedom under the condition of fixed position.

[0057] The acquisition module is used to acquire approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time using an attitude angle measuring instrument. The rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction.

[0058] The first calculation module is used to obtain the approximate value of the rotation angle of each camera based on the homography between images, and use the least squares method to iteratively solve the rotation angle corresponding to each camera using a single feature point.

[0059] The second calculation module is used to calculate the rotation matrix and translation vector of the BSV model after rotation based on the rotation angles of the left and right cameras.

[0060] The third calculation module is used to establish a basic matrix to associate corresponding image points based on visual geometry, and to calculate the deviation of the rotation angles of the left and right cameras in the BSV model using an unconstrained nonlinear optimization algorithm.

[0061] The fourth calculation module is used to calculate the external parameters of the rotated BSV model, including the final rotation matrix and the final translation vector, based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model.

[0062] Thirdly, an electronic device is proposed, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it enables the electronic device to implement the calibration method described above for achieving dynamic binocular stereo vision using only a single feature point.

[0063] Fourthly, a computer-readable storage medium is proposed for storing a computer program that, when run on a computer, causes the computer to perform the calibration method described above for achieving dynamic binocular stereo vision using only a single feature point.

[0064] As can be seen from the above technical solutions, the technical solutions of the present invention have achieved the following beneficial effects:

[0065] This invention discloses a calibration method and system for achieving dynamic binocular stereo vision using only a single feature point. The method includes: constructing a non-zoom dynamic BSV model; pre-determining the intrinsic parameters and initial relative positions of the left and right cameras in the BSV model using Zhang's method; each camera having horizontal and vertical rotation degrees of freedom under a fixed position; using an attitude angle measuring instrument to obtain approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time, the rotation angles including the camera's pitch angle in the vertical direction and yaw angle in the horizontal direction; based on image homography, using the approximate values ​​of the rotation angles of each camera as initial values, and using a single feature point to iteratively solve for the rotation angles corresponding to each camera using the least squares method. Based on the solved rotation angles of the left and right cameras, the rotation matrix and translation vector of the rotated BSV model are calculated. A basic matrix is ​​established based on visual geometry to associate corresponding image points, and an unconstrained nonlinear optimization algorithm is used to calculate the deviation of the rotation angles of the left and right cameras in the BSV model. Based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the extrinsic parameters of the rotated BSV model are calculated, including the final rotation matrix and the final translation vector. Compared with traditional calibration methods that use a large number of control points and time-consuming and algorithmically complex self-calibration methods, this invention uses only a single rectangular feature center point of unknown size to solve for the rotation angle of the dynamic BSV, achieving rapid online calibration of the extrinsic parameters of the dynamic BSV.

[0066] Specifically, compared to existing technologies, firstly, the present invention simplifies the mathematical model of binocular stereo vision, reducing camera extrinsic parameters to yaw and pitch angles, thus improving computational efficiency; secondly, the left and right camera calibration in the present invention avoids the use of high-precision calibration objects with specific dimensions, while rectangular features are very common in daily life, overcoming the inconvenience of calibration in complex environments; furthermore, the present invention is suitable for online calibration of dynamic BSV systems. In practical applications, the system can perform calibration adjustments in real time to adapt to changes in the environment and viewing angle, thereby improving adaptability and flexibility in dynamic scenes.

[0067] It should be understood that all combinations of the foregoing concepts and the additional concepts described in more detail below can be considered part of the inventive subject matter of this disclosure, provided that such concepts do not contradict each other.

[0068] The foregoing and other aspects, embodiments, and features of the teachings of the present invention will be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the invention, such as features and / or beneficial effects of exemplary embodiments, will become apparent from the following description or may be learned through practice of specific embodiments according to the teachings of the present invention. Attached Figure Description

[0069] The accompanying drawings are not drawn to scale. In the drawings, each identical or nearly identical component shown in the various figures may be denoted by the same reference numeral. For clarity, not every component is labeled in each figure. Embodiments of various aspects of the invention will now be described by way of example and with reference to the accompanying drawings, wherein:

[0070] Figure 1 The flowchart below shows the calibration method for implementing dynamic binocular stereo vision as disclosed in the embodiment.

[0071] Figure 2 This is a simplified non-zoom dynamic BSV model for this invention;

[0072] Figure 3 This is a block diagram of the calibration system for implementing dynamic binocular stereo vision disclosed in the embodiment;

[0073] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art to which this invention pertains.

[0075] The terms "first," "second," and similar words used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms of "an," "a," or "the," etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. Terms such as "comprising" or "including" mean that the element or object preceding "comprising" encompasses the features, integrals, steps, operations, elements, and / or components listed following "comprising" or "including," and do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0076] In existing technologies, when using traditional calibration methods or self-calibration methods to calibrate dynamic binocular stereo vision systems, robust algorithms and long-term computations are required if high-precision results are needed. This is time-consuming, labor-intensive, complex, and inconvenient. Therefore, this invention aims to propose a calibration method and system for dynamic binocular stereo vision that utilizes only a single feature point. This method not only enables rapid online calibration of dynamic binocular stereo vision but is also simple to operate.

[0077] The calibration method and system for achieving dynamic binocular stereo vision using only a single feature point, as disclosed in this invention, will be further described in detail below with reference to the accompanying drawings.

[0078] Combination Figure 1 As shown in the embodiment, the calibration method for dynamic binocular stereo vision using only a single feature point disclosed specifically includes the following steps:

[0079] Step S102: Construct a non-zoom dynamic BSV model. The intrinsic parameters and initial relative positions of the left and right cameras in the BSV model are pre-determined using Zhang's method. Each camera has horizontal and vertical rotation degrees of freedom under a fixed position. The left and right cameras can rotate horizontally and vertically. The intrinsic parameters and initial relative positions of the left and right cameras are pre-determined, such as... Figure 2 As shown;

[0080] In the BSV model, the coordinate systems of the left and right cameras before rotation are defined as O and O, respectively. L -X l Y l Z l and O R -X r Y r Z r After rotation, the coordinate systems of the left and right cameras are O and O respectively. L -X L Y L Z L and O r -X R Y R Z R The transformation relationship between the left and right cameras before and after rotation in the BSV model, and the transformation relationship between the left and right cameras relative to themselves before and after rotation in the BSV model are as follows:

[0081] The mapping relationship between the left and right camera coordinate systems before and after the BSV model rotation is as follows:

[0082] P r =R0P l +T0,P R =RP L +T; Equation (1)

[0083] Among them, Pl P r and P L P R The left and right camera coordinate systems before and after the BSV model rotation are represented respectively. R0 and R represent the rotation matrices of the BSV model before and after rotation, and T0 and T represent the translation vectors of the BSV model before and after rotation.

[0084] According to the rotation theorem, the rotation matrices of the left and right cameras of the BSV model relative to themselves before and after rotation are R0 and R1, respectively. l R r Then the rotation matrix R of the left camera l for:

[0085]

[0086] Right camera rotation matrix R r for:

[0087]

[0088] Where, p l y l R represents the pitch and yaw angles of the left camera, respectively. x (p l ) indicates that the left camera rotates around the x-axis. l The rotation matrix, R y (y l ) indicates that the left camera rotates around the y-axis. l rotation matrix of p; r y r R represents the pitch and yaw angles of the right camera, respectively. x (p r ) indicates that the right camera rotates about the x-axis. r The rotation matrix, R y (y r ) indicates that the right camera rotates about the y-axis. r The rotation matrix; C represents the trigonometric function cos, and S represents the trigonometric function sin.

[0089] Step S104: Use an attitude angle measuring instrument to obtain approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time. The rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction.

[0090] The attitude angle measurement instrument selected is the Ellipse-A attitude angle measurement instrument, which has a measurement accuracy of 0.2° for pitch angle, 1° for yaw angle, and a measurement range of 360° for all axes.

[0091] Step S106: Based on the homography of the images, the approximate value of the rotation angle of each camera is obtained as the initial value. The rotation angle corresponding to each camera is solved iteratively using the least squares method with a single feature point. The specific process is as follows:

[0092] In the BSV model, the left camera coordinate system is considered the world coordinate system. The projection points of spatial point Q onto the image planes of the left and right camera coordinate systems of the BSV model before and after rotation are q0, respectively. l (u l ,v l ),q r (u r ,v r ) and q L (u L ,v L ),q R (u R ,v R Then, according to the principle of geometric imaging, we can obtain:

[0093]

[0094] Among them, Z l Z L Let U and V represent the depths of spatial point Q in the left camera coordinate system before and after rotation of the BSV model; U0 and V0 represent the two-dimensional coordinates of spatial point Q on the image plane away from the principal point before rotation, and U and V represent the two-dimensional coordinates of spatial point Q on the image plane away from the principal point after rotation, and U0 = u l -c x V0 = v l -c y U = u L -c x V = v L -c y , (c x, c y () represents the principal point coordinates of the left camera; K1 and K2 represent the intrinsic parameter matrices of the left and right cameras in the BSV model, respectively, f xL f yL These represent the focal lengths of the left camera in the horizontal and vertical directions, respectively. xR f yR These represent the focal lengths of the right camera in the horizontal and vertical directions, respectively.

[0095] In the BSV model, the homography matrices between the images before and after the left and right cameras rotate are defined as H1 and H2, respectively, where H1 = K1R. l K1 -1 H2 = K2R r K2 -1Then, by associating the projection points of the spatial points Q of the BSV model before and after rotation onto the left camera image plane using the homography matrix H1, we can obtain:

[0096]

[0097] Where λ is the scaling factor, equal to Z L / Z l ;

[0098] Based on the correspondence between single feature points before and after rotation of the left and right cameras in the BSV model, the equation of the homography matrix H1 is:

[0099]

[0100] Among them, h mn (m,n=1,2,3) represents the value of the m-th row and n-th column of matrix H1;

[0101] The pitch angle p of the left camera is solved iteratively using the least squares method with a single feature point. l and yaw angle y l The equation is:

[0102]

[0103] Similarly, the pitch angle p of the right camera can be solved using the homography matrix H2. r and yaw angle y r .

[0104] Step S108: Based on the solved rotation angles of the left and right cameras, calculate the rotation matrix and translation vector of the rotated BSV model; the specific process is as follows:

[0105] The mapping relationship between the left and right camera coordinate systems before and after the BSV model rotation is adjusted as follows:

[0106] P R =R r R0R l -1 R L +R r T0; Equation (8)

[0107] After rotation, the rotation matrix R and translation vector T of the BSV model are respectively:

[0108] R = P r R0R l -1 T = R r T0.

[0109] Step S110: Establish a basic matrix to associate corresponding image points based on visual geometry, and use an unconstrained nonlinear optimization algorithm to calculate the deviation of the rotation angles of the left and right cameras in the BSV model;

[0110] To reduce the uncertainty of the rotation angle, the unconstrained nonlinear optimization algorithm (Nelder-Mead) is used to calculate the deviation of the rotation angle between the left and right cameras in the BSV model. The process is as follows:

[0111] Define the precise pitch and yaw angles of the left camera as follows: The precise pitch and yaw angles of the right camera are... but:

[0112]

[0113] Where, Δp l Δy l Δp r and Δy r These represent the deviations in pitch and yaw angles of the left and right cameras, respectively.

[0114] In the BSV model, the left and right cameras have their self-corrected rotation matrices. and They are represented as follows:

[0115]

[0116] After associating the corresponding image points using the fundamental matrix F, the homogeneous coordinates of spatial point Q on the left and right camera imaging planes after rotation of the BSV model are U and U, respectively. l =[u L ,v L ,1] T U r =[u R ,v R ,1] T ,but:

[0117] U r T FU l =0,

[0118] in, (c′ x, c′ y ) represents the principal point coordinates of the right camera, t x t y t z These are the components of the translation vector T after rotation of the BSV model along each axis, i.e., T = [t] x ,t y ,tz ] T ;

[0119] Furthermore, the deviation Δp between the pitch and yaw angles of the left and right cameras l Δy l Δp r and Δy r The calculation is performed using an unconstrained nonlinear optimization method:

[0120] F(Δp l ,Δy l ,Δp r ,Δy r )=|U r T FU l | 2 →min equation (10).

[0121] Step S112: Based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model, calculate the external parameters of the rotated BSV model, including the final rotation matrix and the final translation vector;

[0122] The process of calculating the extrinsic parameters of the rotated BSV model is as follows:

[0123] Define the final rotation matrix of the rotated BSV model as follows: The final translation vector is According to equation (8), we can obtain:

[0124]

[0125] The calibration method for dynamic binocular stereo vision disclosed in the above embodiments, which uses only a single feature point, does not use traditional calibration methods that require a large number of control points, nor does it use the proposed self-calibration methods that are relatively time-consuming, complex, and inconvenient. Instead, it is a new method that uses only the center point of a single rectangular feature of unknown size to calibrate the rotation angle of a dynamic BSV model online. Rectangular features are very common in daily life, so the method has a low cost and is suitable for rapid online calibration of the external parameters of dynamic BSV in situations where extremely high accuracy is not required.

[0126] Another type of embodiment of the present invention, such as Figure 3As shown, a calibration system for dynamic binocular stereo vision using only a single feature point is provided, comprising: a construction and determination module for constructing a non-zoom dynamic BSV model, pre-determining the intrinsic parameters and initial relative positions of the left and right cameras in the BSV model using Zhang's method, wherein each camera has horizontal and vertical rotation degrees of freedom under a fixed position; an acquisition module for acquiring approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time using an attitude angle measuring instrument, wherein the rotation angles include the camera's pitch angle in the vertical direction and yaw angle in the horizontal direction; and a first calculation module for calculating the acquired rotation angles of each camera based on image homography. The system uses an approximate value of degrees as an initial value and solves for the rotation angle of each camera using the least squares method iteratively with a single feature point. The second calculation module calculates the rotation matrix and translation vector of the rotated BSV model based on the solved rotation angles of the left and right cameras. The third calculation module establishes a basic matrix to associate corresponding image points based on visual geometry and uses an unconstrained nonlinear optimization algorithm to calculate the deviation of the rotation angles of the left and right cameras in the BSV model. The fourth calculation module calculates the external parameters of the rotated BSV model, including the final rotation matrix and the final translation vector, based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model.

[0127] Since the system is used to implement the steps of the above method, the calculation process and method for each module of the system to implement the corresponding function are the same, and therefore, those already described will not be repeated here.

[0128] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0129] Another embodiment of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it causes the electronic device to perform the method described in any embodiment of the present invention.

[0130] In one embodiment, the electronic device may be a server, and in this embodiment, the structure of the electronic device may be as follows: Figure 4 As shown, it includes a memory 1, a communication module 3, and one or more processors 2.

[0131] Memory 1 is used to store computer programs executed by processor 2. Memory 1 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required for running instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc. Memory 1 may be volatile memory, such as random-access memory (RAM); memory 1 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1 may be any other medium capable of carrying or storing a desired computer program in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1 may be a combination of the above-mentioned memories. Processor 2 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 2 is used to implement the above-mentioned audio data processing method when the computer program stored in memory 1 is invoked. Communication module 3 is used to communicate with terminal devices and other servers.

[0132] This application embodiment does not limit the specific connection medium between the memory 1, communication module 3, and processor 2. This application embodiment... Figure 4 The memory 1 and processor 2 are connected via bus 4, and bus 4 is in Figure 4 The connections between other components are illustrated with thick arrows and are for illustrative purposes only, not as limiting information. Bus 4 can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 4 The text uses only a thick arrow to describe it, but does not indicate that there is only one bus or one type of bus.

[0133] Based on the same inventive concept as the above method embodiments, embodiments of the present invention also provide a computer-readable storage medium for storing a computer program that, when executed on a computer, causes the electronic device to implement the methods in any of the above method embodiments. The computer-readable storage medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0134] Based on the same inventive concept as the above-described method embodiments, embodiments of the present invention also provide a computer program product, which includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of the control methods described above according to various exemplary embodiments of this application. The program product may take the form of any combination of one or more readable media. These computer program commands can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the commands executed by the processor of the computer or other programmable data processing device generate a process for implementing... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0135] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A calibration method for achieving dynamic binocular stereo vision using only a single feature point, wherein the single feature point is the center point of a rectangle, characterized in that, Includes the following steps: A non-zoom dynamic BSV model is constructed, and the intrinsic parameters and initial relative positions of the left and right cameras in the BSV model are determined by Zhang's method. Each camera has horizontal and vertical rotation degrees of freedom under the condition of fixed position. An attitude angle measuring instrument is used to obtain approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time. The rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction. Based on the homography between images, the approximate value of the rotation angle of each camera is obtained as the initial value, and the rotation angle corresponding to each camera is solved iteratively by least squares using a single feature point. Based on the rotation angles of the left and right cameras, calculate the rotation matrix and translation vector of the BSV model after rotation; Based on visual geometry, a basic matrix is ​​established to associate corresponding image points, and an unconstrained nonlinear optimization algorithm is used to calculate the deviation of the rotation angles of the left and right cameras in the BSV model. Based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the external parameters of the rotated BSV model are calculated, including the final rotation matrix and the final translation vector.

2. The calibration method for achieving dynamic binocular stereo vision using only a single feature point according to claim 1, characterized in that, The coordinate systems of the left and right cameras before rotation in the BSV model are defined as follows: and After rotation, the left and right camera coordinate systems are respectively and The transformation relationship between the left and right cameras before and after the BSV model rotation, and the transformation relationship between the left and right cameras relative to themselves before and after the BSV model rotation, are as follows: The mapping relationship between the left and right camera coordinate systems before and after the BSV model rotation is as follows: ; in, The coordinate systems of the left and right cameras before and after the BSV model rotation are shown in order. and This represents the rotation matrix of the BSV model before and after rotation. and Represents the translation vector of the BSV model before and after rotation; According to the rotation theorem, the rotation matrices of the left and right cameras of the BSV model relative to themselves before and after rotation are respectively... , Then the rotation matrix of the left camera for: ； Right camera rotation matrix for: ； in, , These represent the pitch and yaw angles of the left camera, respectively. Indicates left camera around x Axis rotation The rotation matrix, Indicates left camera around y Axis rotation The rotation matrix; , These represent the pitch and yaw angles of the right camera, respectively. Indicates right camera around x Axis rotation The rotation matrix, Indicates right camera around y Axis rotation The rotation matrix; C Representing trigonometric functions cos , S Representing trigonometric functions sin .

3. The calibration method for achieving dynamic binocular stereo vision using only a single feature point according to claim 2, characterized in that, Based on the homography between images, the approximate rotation angle of each camera is obtained as the initial value. The process of iteratively solving for the rotation angle of each camera using a single feature point through the least squares method includes: In the BSV model, the left camera coordinate system is considered the world coordinate system. The projection points of spatial point Q onto the image planes of the left and right camera coordinate systems of the BSV model before and after rotation are respectively: ( ), ( )and ( ), ( Then, according to the principle of geometric imaging, we can obtain: ； ， ； in, , These represent the depths of spatial point Q in the left camera coordinate system before and after rotation of the BSV model; Let Q represent the two-dimensional coordinates of the spatial point Q on the image plane before rotation, offset from the principal point. U and V Let represent the two-dimensional coordinates of the spatial point Q on the image plane after rotation, deviating from the principal point, and . = - = - U = - V = - , , () represents the principal point coordinates of the left camera; Let represent the intrinsic parameter matrices of the left and right cameras in the BSV model, respectively. 、 These represent the focal lengths of the left camera in the horizontal and vertical directions, respectively. 、 These represent the focal lengths of the right camera in the horizontal and vertical directions, respectively. Define the homography matrices of the images before and after the left and right cameras rotate in the BSV model as follows: , , , Then through the homography matrix By correlating the projections of the spatial point Q of the BSV model before and after rotation onto the left camera image plane, we can obtain: ； Where λ is the scaling factor, equal to / ; Based on the correspondence of single feature points in the BSV model before and after rotation of the left and right cameras, the homography matrix... The equation is: ； in, , representing a matrix No. m Line number n The value of the column; The pitch angle of the left camera is solved iteratively using the least squares method with a single feature point. and yaw angle The equation is: ； Similarly, according to the homography matrix Solve for the pitch angle of the right camera. and yaw angle .

4. The calibration method for achieving dynamic binocular stereo vision using only a single feature point according to claim 2, characterized in that, The process of calculating the rotation matrix and translation vector of the BSV model after rotation, based on the solved rotation angles of the left and right cameras, includes: The mapping relationship between the left and right camera coordinate systems before and after the BSV model rotation is adjusted as follows: ； The rotation matrix of the BSV model after rotation R The translation vector T are respectively: 。 5. The calibration method for achieving dynamic binocular stereo vision using only a single feature point according to claim 3, characterized in that, Define the precise pitch and yaw angles of the left camera as follows: , The precise pitch and yaw angles of the right camera are... , ,but: ； in, , , and These represent the deviations in pitch and yaw angles of the left and right cameras, respectively. In the BSV model, the left and right cameras have their self-corrected rotation matrices. and They are represented as follows: ； After associating the corresponding image points using the fundamental matrix F, the homogeneous coordinates of spatial point Q on the left and right camera imaging planes after rotation of the BSV model are respectively ,but: ； in, , ( () represents the principal point coordinates of the right camera. , , These are the components of the translation vector T after rotation of the BSV model along each axis. ; Furthermore, the deviations in pitch and yaw angles between the left and right cameras... , , and The calculation is performed using an unconstrained nonlinear optimization method: 。 6. The calibration method for achieving dynamic binocular stereo vision using only a single feature point according to claim 5, characterized in that, The process of calculating the external parameters of the rotated BSV model based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model is as follows: The final rotation matrix of the BSV model after rotation is defined as follows: The final translation vector is ,but: 。 7. The calibration method for achieving dynamic binocular stereo vision using only a single feature point according to claim 1, characterized in that, The attitude angle measuring instrument has a pitch angle measurement accuracy of 0.2°, a yaw angle measurement accuracy of 1°, and a measurement range of 360° for all axes.

8. A calibration system for achieving dynamic binocular stereo vision using only a single feature point, wherein the single feature point is the center point of a rectangle, characterized in that, include: A determination module is built to construct a non-zoom dynamic BSV model. The intrinsic parameters and initial relative positions of the left and right cameras in the BSV model are determined in advance using Zhang's method. Each camera has horizontal and vertical rotational degrees of freedom under the condition of fixed position. The acquisition module is used to acquire approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time using an attitude angle measuring instrument. The rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction. The first calculation module is used to obtain the approximate value of the rotation angle of each camera based on the homography between images, and use the least squares method to iteratively solve the rotation angle corresponding to each camera using a single feature point. The second calculation module is used to calculate the rotation matrix and translation vector of the BSV model after rotation based on the rotation angles of the left and right cameras. The third calculation module is used to establish a basic matrix to associate corresponding image points based on visual geometry, and to calculate the deviation of the rotation angles of the left and right cameras in the BSV model using an unconstrained nonlinear optimization algorithm. The fourth calculation module is used to calculate the external parameters of the rotated BSV model, including the final rotation matrix and the final translation vector, based on the calculated deviation of the rotation angles of the left and right cameras in the BSV model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it enables the electronic device to implement the calibration method for dynamic binocular stereo vision using only a single feature point as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer, causes the computer to perform a calibration method for achieving dynamic binocular stereo vision using only a single feature point as described in any one of claims 1 to 7.

Images