Calibration method and system for realizing dynamic binocular stereoscopic vision only by using single feature point

Abstract

The invention provides a calibration method and system for realizing dynamic binocular stereoscopic vision only by using a single feature point, and relates to the field of binocular stereoscopic vision. Comprising the following steps: building a non-zoom dynamic BSV model, and determining an internal reference and an initial relative position of a camera in the model by adopting a Zhang's method in advance; obtaining an approximate value of a rotation angle through an attitude angle measuring instrument in real time, and solving the rotation angle of the camera through iteration by only using a single rectangular feature center point according to homography between images; based on the characteristics of the basic matrix, the pitch angle and the yaw angle are corrected through corresponding rectangular center points obtained by the rotated left camera and the rotated right camera respectively, and finally the external parameters of the dynamic BSV after rotation can be calculated according to the determined rotation angle and the known initial relative position between the left camera and the right camera. According to the method, a high-precision calibration object with a certain size is prevented from being used, the rectangular characteristics are very common in daily life, the cost of the method is low, and the method is suitable for rapid online calibration of dynamic BSV external parameters in occasions without extremely high precision requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of binocular stereo vision, and in particular to a calibration method and system for realizing dynamic binocular stereo vision by using only a single feature point. Background Art

[0002] With the rapid development of industrial cameras and image processing technology, stereo vision is widely used in target positioning, distance measurement and 3D structure measurement. Dynamic binocular stereo vision (BSV) can expand the visual measurement range by virtue of the camera's free rotation ability, thereby achieving tracking and positioning of moving targets.

[0003] Rotating camera is an important part of dynamic binocular stereo vision. Only the rotation in the horizontal and vertical directions can be used to adjust the field of view (FOV). The calibration of rotating camera is crucial for accurately positioning moving targets. Camera calibration methods mainly include traditional calibration methods and self-calibration methods. Traditional methods must use the three-dimensional coordinates of known control points and the corresponding pixel coordinates on the image plane. Although the accuracy is high, a large number of control points are required, and it must be ensured to be within the FOV range, which is not desirable when the public FOV of dynamic BSV changes frequently. Self-calibration methods are not as robust as traditional methods, but are suitable for the calibration of rotating cameras. However, mismatching of feature points will cause the calibration results to deviate from the true value. If high-precision results are required, robust algorithms and long-term calculations must be used. In order to solve the problem of mismatching of feature points, the existing technology proposes self-calibration methods based on structured light or different types of targets, but these calibration algorithms are generally time-consuming. Therefore, it is necessary to study a method that can quickly complete the online calibration of rotating cameras while ensuring accuracy at the same time. Summary of the invention

[0004] The purpose of the present invention is to provide a calibration method and system for realizing dynamic binocular stereo vision using only a single feature point, and proposes to realize online calibration of dynamic BSV extrinsic parameters using a single rectangular feature center point of unknown size, thereby avoiding the use of high-precision calibration objects with specific sizes. The method has low cost and is suitable for online calibration of dynamic BSV extrinsic parameters without requiring extremely high precision.

[0005] To achieve the above object, the present invention proposes the following technical solutions:

[0006] In the first aspect, a calibration method for dynamic binocular stereo vision using only a single feature point is proposed, comprising the following steps:

[0007] A non-zooming dynamic BSV model is constructed, and the Zhang method is used in advance to determine the internal parameters and initial relative positions of the left and right cameras in the BSV model, and any of the cameras has the freedom of horizontal and vertical rotation under the condition of fixed position;

[0008] An attitude angle measuring instrument is used to obtain in real time the approximate values ​​of the rotation angles of the left and right cameras in the BSV model, wherein the rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction;

[0009] According to the homography between images, the approximate value of the rotation angle of each camera is taken as the initial value, and the rotation angle corresponding to each camera is iteratively solved by the least squares method using a single feature point;

[0010] According to the solved rotation angles of the left and right cameras, the rotation matrix and translation vector of the BSV model after rotation are calculated;

[0011] According to visual geometry, a basic matrix is ​​established to associate corresponding image points, and an unconstrained nonlinear optimization algorithm is used to calculate the deviation of the rotation angles of the left and right cameras in the BSV model;

[0012] According to the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the external parameters of the rotated BSV model are calculated, including a final rotation matrix and a final translation vector.

[0013] Further, the left and right camera coordinate systems before rotation in the BSV model are defined as L -X l Y l Z l and O R -X r Y r Z r , after rotation, the left and right camera coordinate systems are O L -X L Y L Z L and O R -X R Y R Z R , then the transformation relationship between the left and right cameras before and after the BSV model is rotated, and the transformation relationship between the left and right cameras relative to themselves before and after the BSV model is rotated are as follows:

[0014] The mapping relationship between the left and right camera coordinate systems before and after the BSV model is rotated is:

[0015] P r =R0P l +T0,P R =RP L +T;

[0016] Among them, P l , P r and P L , P RRepresent the left and right camera coordinate systems before and after the BSV model is rotated, R0 and R represent the rotation matrix before and after the BSV model is rotated, and T0 and T represent the translation vector before and after the BSV model is rotated;

[0017] According to the rotation theorem, the rotation matrices of the left and right cameras of the BSV model relative to themselves before and after rotation are R l , R r , then the rotation matrix R of the left camera l for:

[0018]

[0019] Right camera rotation matrix R r for:

[0020]

[0021] Among them, p l ,y l Respectively represent the pitch angle and yaw angle of the left camera, R x (p l ) indicates that the left camera rotates around the x-axis by p l The rotation matrix, R y (y l ) indicates that the left camera rotates around the y axis l The rotation matrix of r ,y r Respectively represent the pitch angle and yaw angle of the right camera, R x (p r ) indicates that the right camera rotates around the x-axis p r The rotation matrix, R y (y r ) indicates that the right camera rotates around the y axis r The rotation matrix; C represents the trigonometric function cosine, and S represents the trigonometric function sin.

[0022] Furthermore, according to the homography between images, the approximate value of the rotation angle of each camera is obtained as the initial value, and the process of iteratively solving the rotation angle corresponding to each camera by the least square method using a single feature point includes:

[0023] Define the left camera coordinate system in the BSV model as the world coordinate system. The projection points of the spatial point Q before and after the rotation of the image plane of the left and right camera coordinate systems of the BSV model are q l (u l ,v l ),q r (u r ,v r ) and q L (u L ,v L),q R (u R ,v R ), then according to the principle of geometric imaging, we can get:

[0024]

[0025] Among them, Z l , Z L Respectively represent the depth of the spatial point Q in the left camera coordinate system before and after the BSV model is rotated; U0 and V0 respectively represent the two-dimensional coordinates of the spatial point Q on the image plane that deviates from the principal point before rotation, and U and V respectively represent the two-dimensional coordinates of the spatial point Q on the image plane that deviates from the principal point after rotation, and U0 = u l -c x , V0=v l -c y , U=u L -c x , V = v L -c y , (c x ,c y ) represents the principal point coordinates of the left camera; K1 and K2 represent the intrinsic parameter matrices of the left and right cameras in the BSV model, respectively. xL 、f yL Respectively represent the focal length of the left camera in the horizontal and vertical directions, f xR 、f yR Respectively represent the focal length of the right camera in the horizontal and vertical directions;

[0026] Define the homography matrices between the images before and after the left and right camera rotation in the BSV model as H1 and H2, respectively, H1 = K1R l K1 -1 , H2=K2R r K2 -1 , then the projection points of the spatial point Q before and after the rotation of the BSV model on the left camera image plane are associated through the homography matrix H1 to obtain:

[0027]

[0028] Where λ is the scaling factor, equal to Z L / Z l ;

[0029] According to the correspondence between the single feature point before and after the left and right cameras rotate in the BSV model, the equation of the homography matrix H1 is:

[0030]

[0031] Among them, h mn(m,n=1,2,3) represents the value of the mth row and nth column of matrix H1;

[0032] The pitch angle p of the left camera is solved iteratively using the least squares method using a single feature point l and yaw angle y l The equation is:

[0033]

[0034] Similarly, the pitch angle p of the right camera is solved according to the homography matrix H2 r and yaw angle y r .

[0035] Furthermore, according to the solved rotation angles of the left and right cameras, the process of calculating the rotation matrix and the translation vector of the BSV model after rotation includes:

[0036] The mapping relationship between the left and right camera coordinate systems before and after the BSV model is rotated is adjusted to:

[0037] P R =R r R0R l -1 P L +R r T0;

[0038] After rotation, the rotation matrix R and translation vector T of the BSV model are:

[0039] R=R r R0R l -1 , T=R r T0.

[0040] Furthermore, the precise pitch and yaw angles of the left camera are defined as The precise pitch and yaw angles of the right camera are but:

[0041]

[0042] Among them, Δp l , Δy l , Δp r and Δy r Represents the deviation of pitch angle and yaw angle of left and right cameras respectively;

[0043] The rotation matrix of the left and right cameras in the BSV model after correction relative to themselves and Respectively expressed as:

[0044]

[0045] After the basic matrix F is used to associate the corresponding image points, when the spatial point Q is rotated in the BSV model, the homogeneous coordinates of the left and right camera imaging planes are U l =[u L ,v L ,1] T , U r =[u R ,v R ,1] T ,but:

[0046] U r T FU l =0,

[0047] in,

[0048] , (c′ x, c′ y ) is the principal point coordinate of the right camera, t x ,t y ,t z are the components of the translation vector T of the BSV model after rotation in each axis direction, that is, T = [t x ,t y ,t z ] T ;

[0049] In addition, the deviation of the pitch angle and yaw angle of the left and right cameras is Δp l , Δy l , Δp r and Δy r The unconstrained nonlinear optimization method is used to calculate:

[0050] F(Δp l ,Δy l ,Δp r ,Δy r )=|U r T FU l | 2 →min.

[0051] Furthermore, according to the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the process of calculating the external parameters of the rotated BSV model is as follows:

[0052] The final rotation matrix of the BSV model after rotation is defined as The final translation vector is but:

[0053]

[0054] Furthermore, the pitch angle measurement accuracy of the attitude angle measuring instrument is 0.2°, the yaw angle measurement accuracy is 1°, and the measurement range of all axes is 360°.

[0055] In the second aspect, a calibration system for dynamic binocular stereo vision using only a single feature point is proposed, including:

[0056] A determination module is constructed to construct a non-zoom dynamic BSV model, and Zhang's method is used to determine the internal parameters and initial relative positions of the left and right cameras in the BSV model in advance, and any of the cameras has the freedom of horizontal and vertical rotation under the condition of fixed position;

[0057] An acquisition module, used to acquire the approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time using an attitude angle measuring instrument, wherein the rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction;

[0058] The first calculation module is used to obtain the approximate value of the rotation angle of each camera as an initial value according to the homography between images, and use a single feature point to iteratively solve the rotation angle corresponding to each camera through the least square method;

[0059] A second calculation module is used to calculate the rotation matrix and translation vector of the BSV model after rotation according to the solved rotation angles of the left and right cameras;

[0060] A third calculation module is used to establish a basic matrix to associate corresponding image points according to visual geometry, and use an unconstrained nonlinear optimization algorithm to calculate the deviation of the rotation angles of the left and right cameras in the BSV model;

[0061] The fourth calculation module is used to calculate the external parameters of the rotated BSV model, including the final rotation matrix and the final translation vector, according to the calculated deviation of the rotation angles of the left and right cameras in the BSV model.

[0062] In the third aspect, an electronic device is proposed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the calibration method for dynamic binocular stereo vision using only a single feature point as described above.

[0063] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium is used to store a computer program. When the computer program is run on a computer, the computer executes the calibration method for realizing dynamic binocular stereo vision using only a single feature point as described above.

[0064] It can be seen from the above technical solutions that the technical solutions of the present invention have the following beneficial effects:

[0065] The present invention discloses a calibration method and system for realizing dynamic binocular stereo vision using only a single feature point, the method comprising: building a non-zoom dynamic BSV model, using Zhang's method to determine the internal parameters and initial relative positions of left and right cameras in the BSV model in advance, any of the cameras having horizontal and vertical rotation degrees of freedom under the condition of fixed position; using an attitude angle measuring instrument to obtain the approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time, the rotation angles including the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction; according to the homography between images, using the approximate values ​​of the rotation angles of each camera obtained as the initial values, using a single feature point to iteratively solve the rotation angles corresponding to each camera by the least squares method. ; According to the solved rotation angles of the left and right cameras, the rotation matrix and translation vector of the BSV model after rotation are calculated; According to visual geometry, a basic matrix is ​​established to associate corresponding image points, and an unconstrained nonlinear optimization algorithm is used to calculate the deviation of the rotation angles of the left and right cameras in the BSV model; According to the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the external parameters of the BSV model after rotation, including the final rotation matrix and the final translation vector, are calculated; Compared with the traditional calibration method using a large number of control points and the time-consuming and algorithmically complex self-calibration method, the present invention only uses a single rectangular feature center point of unknown size to solve the rotation angle of the dynamic BSV, thereby realizing the fast online calibration of the dynamic BSV external parameters.

[0066] Specifically, compared with the prior art, firstly, the scheme of the present invention simplifies the mathematical model of binocular stereo vision, and the camera extrinsic parameters are reduced to yaw angle and pitch angle, thereby improving the calculation efficiency; secondly, the left and right camera calibration in the scheme avoids the use of high-precision calibration objects with specific sizes, and rectangular features are very common in life, which overcomes the defect of inconvenient calibration in complex environments; in addition, this scheme is suitable for online calibration of dynamic BSV systems. In practical applications, the system can perform calibration adjustments in real time to adapt to changes in the environment and viewing angle, thereby improving the adaptability and flexibility in dynamic scenes.

[0067] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, may be considered to be part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent.

[0068] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings are not drawn to scale according to actual reference objects. In the accompanying drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For the sake of clarity, not every component is labeled in each figure. Now, embodiments of various aspects of the present invention will be described by way of example and with reference to the accompanying drawings, in which:

[0070] Figure 1 A flow chart of a calibration method for realizing dynamic binocular stereo vision disclosed in an embodiment;

[0071] Figure 2 It is a simplified non-zoom dynamic BSV model of the present invention;

[0072] Figure 3 The structure block diagram of the calibration system for realizing dynamic binocular stereo vision disclosed in the embodiment;

[0073] Figure 4 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meaning understood by people with general skills in the field to which the present invention belongs.

[0075] The words "first", "second" and similar words used in the patent application specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular form of "a", "an" or "the" and other similar words do not indicate a quantitative limitation, but indicate the existence of at least one. "Include" or "comprises" and other similar words mean that the elements or objects appearing before "include" or "comprises" include the features, wholes, steps, operations, elements and / or components listed after "include" or "comprises", and do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0076] In the prior art, when calibrating a dynamic binocular stereo vision system using traditional calibration methods or self-calibration methods, if high-precision results are required, a robust algorithm and long-term calculations must be used, which is time-consuming, labor-intensive, complex and inconvenient. Therefore, the present invention aims to propose a calibration method and system for dynamic binocular stereo vision using only a single feature point, which can not only quickly perform online calibration of dynamic binocular stereo vision but also is simple to operate.

[0077] The following is a further detailed introduction to the calibration method and system for realizing dynamic binocular stereo vision using only a single feature point disclosed in the present invention in conjunction with the accompanying drawings.

[0078] Combination Figure 1 As shown, the calibration method for realizing dynamic binocular stereo vision using only a single feature point disclosed in the embodiment specifically includes the following steps:

[0079] Step S102: Build a non-zoom dynamic BSV model, and use Zhang's method to determine the internal parameters and initial relative positions of the left and right cameras in the BSV model in advance. Any of the cameras has the freedom of horizontal and vertical rotation under the condition of fixed position; the left and right cameras can rotate horizontally and vertically, and the internal parameters and initial relative positions of the left and right cameras are determined in advance, such as Figure 2 As shown;

[0080] Define the left and right camera coordinate systems before rotation in the BSV model as L -X l Y l Z l and O R -X r Y r Z r , after rotation, the left and right camera coordinate systems are O L -X L Y L Z L and O r -X R Y R Z R , then the transformation relationship between the left and right cameras before and after the rotation in the BSV model, and the transformation relationship between the left and right cameras relative to themselves before and after the rotation of the BSV model are as follows:

[0081] The mapping relationship between the left and right camera coordinate systems before and after the BSV model is rotated is:

[0082] P r =R0P l +T0,P R =RP L +T; Formula (1)

[0083] Among them, Pl , P r and P L , P R Represent the left and right camera coordinate systems before and after the BSV model is rotated, R0 and R represent the rotation matrix before and after the BSV model is rotated, and T0 and T represent the translation vector before and after the BSV model is rotated;

[0084] According to the rotation theorem, the rotation matrices of the left and right cameras of the BSV model relative to themselves before and after rotation are R l , R r , then the rotation matrix R of the left camera l for:

[0085]

[0086] Right camera rotation matrix R r for:

[0087]

[0088] Among them, p l ,y l Respectively represent the pitch angle and yaw angle of the left camera, R x (p l ) indicates that the left camera rotates around the x-axis by p l The rotation matrix, R y (y l ) indicates that the left camera rotates around the y axis l The rotation matrix of r ,y r Respectively represent the pitch angle and yaw angle of the right camera, R x (p r ) indicates that the right camera rotates around the x-axis p r The rotation matrix, R y (y r ) indicates that the right camera rotates around the y axis r The rotation matrix; C represents the trigonometric function cosine, and S represents the trigonometric function sin.

[0089] Step S104: using an attitude angle measuring instrument to obtain in real time the approximate values ​​of the rotation angles of the left and right cameras in the BSV model, wherein the rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction;

[0090] The attitude angle measuring instrument selected is the Ellipse-A attitude angle measuring instrument, which has a measurement accuracy of 0.2° for pitch angle and 1° for yaw angle, and the measurement range of all axes is 360°.

[0091] Step S106: According to the homography between images, the approximate value of the rotation angle of each camera is obtained as the initial value, and the rotation angle corresponding to each camera is iteratively solved by the least square method using a single feature point; the specific process is as follows:

[0092] Define the left camera coordinate system in the BSV model as the world coordinate system. The projection points of the spatial point Q before and after the rotation of the image plane of the left and right camera coordinate systems of the BSV model are q l (u l ,v l ),q r (u r ,v r ) and q L (u L ,v L ),q R (u R ,v R ), then according to the principle of geometric imaging, we can get:

[0093]

[0094] Among them, Z l , Z L Respectively represent the depth of the spatial point Q in the left camera coordinate system before and after the BSV model is rotated; U0 and V0 respectively represent the two-dimensional coordinates of the spatial point Q on the image plane that deviates from the principal point before rotation, and U and V respectively represent the two-dimensional coordinates of the spatial point Q on the image plane that deviates from the principal point after rotation, and U0 = u l -c x , V0=v l -c y , U=u L -c x , V = v L -c y , (c x, c y ) represents the principal point coordinates of the left camera; K1 and K2 represent the intrinsic parameter matrices of the left and right cameras in the BSV model, respectively. xL 、f yL Respectively represent the focal length of the left camera in the horizontal and vertical directions, f xR 、f yR Respectively represent the focal length of the right camera in the horizontal and vertical directions;

[0095] Define the homography matrices between the images before and after the left and right camera rotation in the BSV model as H1 and H2, respectively, H1 = K1R l K1 -1 , H2=K2R r K2 -1, then the projection points of the spatial point Q before and after the rotation of the BSV model on the left camera image plane are associated through the homography matrix H1 to obtain:

[0096]

[0097] Where λ is the scaling factor, equal to Z L / Z l ;

[0098] According to the correspondence between the single feature point before and after the left and right cameras rotate in the BSV model, the equation of the homography matrix H1 is:

[0099]

[0100] Among them, h mn (m,n=1,2,3) represents the value of the mth row and nth column of matrix H1;

[0101] The pitch angle p of the left camera is solved iteratively using the least squares method using a single feature point l and yaw angle y l The equation is:

[0102]

[0103] Similarly, the pitch angle p of the right camera is solved according to the homography matrix H2 r and yaw angle y r .

[0104] Step S108: Calculate the rotation matrix and translation vector of the BSV model after rotation according to the solved rotation angles of the left and right cameras; the specific process is as follows:

[0105] The mapping relationship between the left and right camera coordinate systems before and after the BSV model is rotated is adjusted to:

[0106] P R =R r R0R l -1 R L +R r T0; Formula (8)

[0107] After rotation, the rotation matrix R and translation vector T of the BSV model are:

[0108] R=P r R0R l -1 , T=R r T0.

[0109] Step S110: establishing a basic matrix to associate corresponding image points according to visual geometry, and using an unconstrained nonlinear optimization algorithm to calculate the deviation of the rotation angles of the left and right cameras in the BSV model;

[0110] In order to reduce the uncertainty of the rotation angle, the unconstrained nonlinear optimization algorithm (Nelder-Mead) is used to calculate the deviation of the rotation angle of the left and right cameras in the BSV model. The process is as follows:

[0111] Define the precise pitch and yaw angles of the left camera as The precise pitch and yaw angles of the right camera are but:

[0112]

[0113] Among them, Δp l , Δy l , Δp r and Δy r Represents the deviation of pitch angle and yaw angle of left and right cameras respectively;

[0114] The rotation matrix of the left and right cameras in the BSV model after correction relative to themselves and Respectively expressed as:

[0115]

[0116] After the basic matrix F is used to associate the corresponding image points, when the spatial point Q is rotated in the BSV model, the homogeneous coordinates of the left and right camera imaging planes are U l =[u L ,v L ,1] T , U r =[u R ,v R ,1] T ,but:

[0117] U r T FU l =0,

[0118] in, (c′ x, c′ y ) is the principal point coordinate of the right camera, t x ,t y ,t z are the components of the translation vector T of the BSV model after rotation in each axis direction, that is, T = [t x ,t y ,tz ] T ;

[0119] In addition, the deviation of the pitch angle and yaw angle of the left and right cameras is Δp l , Δy l , Δp r and Δy r The unconstrained nonlinear optimization method is used to calculate:

[0120] F(Δp l ,Δy l ,Δp r ,Δy r )=|U r T FU l | 2 →min formula (10).

[0121] Step S112: Calculate the external parameters of the rotated BSV model, including a final rotation matrix and a final translation vector, according to the calculated deviation of the rotation angles of the left and right cameras in the BSV model;

[0122] The process of calculating the external parameters of the rotated BSV model is:

[0123] The final rotation matrix of the BSV model after rotation is defined as The final translation vector is According to formula (8), we can get:

[0124]

[0125] The calibration method for realizing dynamic binocular stereo vision using only a single feature point disclosed in the above embodiment does not use the traditional calibration method that requires a large number of control points, nor does it use the proposed self-calibration method that is time-consuming, complex and inconvenient. Instead, it is a new method that only uses a single rectangular feature center point of unknown size to online calibrate the rotation angle of the dynamic BSV model. Rectangular features are very common in daily life, so the method is low in cost and is suitable for fast online calibration of dynamic BSV external parameters in situations where extremely high precision is not required.

[0126] Another embodiment of the present invention is as follows Figure 3As shown, a calibration system for realizing dynamic binocular stereo vision using only a single feature point is provided, including: a determination module for building a non-zoom dynamic BSV model, and using Zhang's method to determine the internal parameters and initial relative positions of the left and right cameras in the BSV model in advance, and any of the cameras has the freedom of horizontal and vertical rotation under the condition of fixed position; an acquisition module for using an attitude angle measuring instrument to obtain the approximate value of the rotation angle of the left and right cameras in the BSV model in real time, and the rotation angle includes the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction; a first calculation module for respectively obtaining the rotation angle of each camera according to the homography between images; The approximate value of the degree is used as the initial value, and the rotation angle corresponding to each camera is iteratively solved by the least squares method using a single feature point; the second calculation module is used to calculate the rotation matrix and translation vector of the BSV model after rotation according to the solved rotation angles of the left and right cameras; the third calculation module is used to establish a basic matrix to associate corresponding image points according to visual geometry, and use an unconstrained nonlinear optimization algorithm to calculate the deviation of the rotation angles of the left and right cameras in the BSV model; the fourth calculation module is used to calculate the external parameters of the BSV model after rotation, including the final rotation matrix and the final translation vector, according to the calculated deviation of the rotation angles of the left and right cameras in the BSV model.

[0127] Since the system is used to implement the steps of the above method, the calculation process and method of each module of the system to implement the corresponding function are the same, so it has been explained and will not be repeated here.

[0128] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0129] Another embodiment of the present invention is an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements a method as described in any method embodiment of the present invention.

[0130] In one embodiment, the electronic device may be a server. In this embodiment, the structure of the electronic device may be as follows: Figure 4 As shown, it includes a memory 1, a communication module 3 and one or more processors 2.

[0131] Memory 1 is used to store computer programs executed by processor 2. Memory 1 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, and programs required for running instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc. Memory 1 may be a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); Memory 1 may also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk drive (hard disk drive, HDD) or a solid-state drive (solid-state drive, SSD); or Memory 1 is any other medium that can be used to carry or store a desired computer program in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. Memory 1 may be a combination of the above memories. Processor 2 may include one or more central processing units (central processing units, CPU) or a digital processing unit, etc. Processor 2 is used to implement the above audio data processing method when calling the computer program stored in memory 1. The communication module 3 is used to communicate with terminal devices and other servers.

[0132] The specific connection medium between the memory 1, the communication module 3 and the processor 2 is not limited in the present application embodiment. Figure 4 In the example, memory 1 and processor 2 are connected via bus 4. Figure 4 The connections between other components are described by thick arrows, which are only for schematic illustration and are not intended to be limiting. The bus 4 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 4 Only one thick arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0133] Based on the same inventive concept as the above method embodiment, an embodiment of the present invention further provides a computer-readable storage medium, the computer-readable storage medium is used to store a computer program, when the computer program is run on a computer, the electronic device implements the method in any of the above method embodiments. The computer-readable storage medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0134] Based on the same inventive concept as the above-mentioned method embodiment, an embodiment of the present invention further provides a computer program product, the computer program product including a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the control method according to various exemplary embodiments of the present application described above in this specification. The program product may adopt any combination of one or more readable media. These computer program commands may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the steps in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A calibration method for dynamic binocular stereo vision using only a single feature point, characterized in that: The steps include: A non-zoom dynamic BSV model is constructed, and Zhang's method is used to determine the intrinsic parameters and initial relative positions of the left and right cameras in the BSV model, where any of the cameras has horizontal and vertical rotation freedom under the condition of a fixed position; An attitude angle measuring instrument is used to obtain in real time the approximate values ​​of the rotation angles of the left and right cameras in the BSV model, wherein the rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction; According to the homography between images, the approximate value of the rotation angle of each camera is taken as the initial value, and the rotation angle corresponding to each camera is iteratively solved by the least squares method using a single feature point; According to the solved rotation angles of the left and right cameras, the rotation matrix and translation vector of the BSV model after rotation are calculated; According to visual geometry, a basic matrix is ​​established to associate corresponding image points, and an unconstrained nonlinear optimization algorithm is used to calculate the deviation of the rotation angles of the left and right cameras in the BSV model; According to the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the external parameters of the rotated BSV model are calculated, including a final rotation matrix and a final translation vector.

2. The calibration method for realizing dynamic binocular stereo vision using only a single feature point according to claim 1, characterized in that: Define the left and right camera coordinate systems before rotation in the BSV model as L -X l Y l Z l and O R -X r Y r Z r , after rotation, the left and right camera coordinate systems are O L -X L Y L Z L and O R -X R Y R Z R , then the transformation relationship between the left and right cameras before and after the BSV model is rotated, and the transformation relationship between the left and right cameras relative to themselves before and after the BSV model is rotated are as follows: The mapping relationship between the left and right camera coordinate systems before and after the BSV model is rotated is: P r =R0P l +T0,P R =RP L +T; Among them, P l , P r and P L , R R Represent the left and right camera coordinate systems before and after the BSV model is rotated, R0 and R represent the rotation matrix before and after the BSV model is rotated, and T0 and T represent the translation vector before and after the BSV model is rotated; According to the rotation theorem, the rotation matrices of the left and right cameras of the BSV model relative to themselves before and after rotation are R l , R r , then the rotation matrix R of the left camera l for: Right camera rotation matrix R r for: Among them, p l ,y l Respectively represent the pitch angle and yaw angle of the left camera, R x (p l ) indicates that the left camera rotates around the x-axis by p l The rotation matrix, R y (y l ) indicates that the left camera rotates around the y axis l The rotation matrix of r ,y r Respectively represent the pitch angle and yaw angle of the right camera, R x (p r ) indicates that the right camera rotates around the x-axis p r The rotation matrix, R y (y r ) indicates that the right camera rotates around the y axis r The rotation matrix; C represents the trigonometric function cosine, and S represents the trigonometric function sin.

3. The calibration method for realizing dynamic binocular stereo vision using only a single feature point according to claim 2, characterized in that: According to the homography between images, the approximate value of the rotation angle of each camera is taken as the initial value, and the process of iteratively solving the rotation angle corresponding to each camera by the least squares method using a single feature point includes: Define the left camera coordinate system in the BSV model as the world coordinate system. The projection points of the spatial point Q before and after the rotation of the image plane of the left and right camera coordinate systems of the BSV model are q l (u l ,v l ),q r (u r ,v r ) and q L (u L ,v L ),q R (u R ,v R ), then according to the principle of geometric imaging, we can get: Among them, Z l , Z L Respectively represent the depth of the spatial point Q in the left camera coordinate system before and after the BSV model is rotated; U0 and V0 respectively represent the two-dimensional coordinates of the spatial point Q on the image plane that deviates from the principal point before rotation, and U and V respectively represent the two-dimensional coordinates of the spatial point Q on the image plane that deviates from the principal point after rotation, and U0 = u l -c x , V0=v l -c y , U=u L -c x , V = v L -c y , (c x ,c y ) represents the principal point coordinates of the left camera; K1 and K2 represent the intrinsic parameter matrices of the left and right cameras in the BSV model, respectively. xL 、f yL Respectively represent the focal length of the left camera in the horizontal and vertical directions, f xR 、f yR Respectively represent the focal length of the right camera in the horizontal and vertical directions; Define the homography matrices between the images before and after the left and right camera rotation in the BSV model as H1 and H2, respectively, H1 = K1R l K1 -1 , H2=K2R r K2 -1 , then the projection points of the spatial point Q before and after the rotation of the BSV model on the left camera image plane are associated through the homography matrix H1 to obtain: Where λ is the scaling factor, equal to Z L / Z l ; According to the correspondence between the single feature point before and after the left and right cameras rotate in the BSV model, the equation of the homography matrix H1 is: Among them, h mn (m,n=1,2,3) represents the value of the mth row and nth column of matrix H1; The pitch angle p of the left camera is solved iteratively using the least squares method using a single feature point l and yaw angle y l The equation is: Similarly, the pitch angle p of the right camera is solved according to the homography matrix H2 r and yaw angle y r .

4. The calibration method for realizing dynamic binocular stereo vision using only a single feature point according to claim 2, characterized in that: According to the solved rotation angles of the left and right cameras, the process of calculating the rotation matrix and translation vector of the BSV model after rotation includes: The mapping relationship between the left and right camera coordinate systems before and after the BSV model is rotated is adjusted to: P R =R r R0R l -1 R L +R r T0; After rotation, the rotation matrix R and translation vector T of the BSV model are: R=R r R0R l -1 ,T=R r T0。 5. The calibration method for realizing dynamic binocular stereo vision using only a single feature point according to claim 3, characterized in that: Define the precise pitch and yaw angles of the left camera as The precise pitch and yaw angles of the right camera are but: Among them, Δp l , Δy l , Δp r and Δy r Represents the deviation of pitch angle and yaw angle of left and right cameras respectively; The rotation matrix of the left and right cameras in the BSV model after correction relative to themselves and Respectively expressed as: After the basic matrix F is used to associate the corresponding image points, when the spatial point Q is rotated in the BSV model, the homogeneous coordinates of the left and right camera imaging planes are U l =[u L ,v L ,1] T , U r =[u R ,v R ,1] T ,but: (c′ x ,c′ y ) is the principal point coordinate of the right camera, t x ,t y ,t z are the components of the translation vector T of the BSV model after rotation in each axis direction, that is, T = [t x ,t y ,t z ] T ; In addition, the deviation of the pitch angle and yaw angle of the left and right cameras is Δp l , Δy l , Δp r and Δy r The unconstrained nonlinear optimization method is used to calculate: F(Δp l ,Δy l ,Δp r ,Δy r )=|U r T FU l | 2 →min.

6. The calibration method for realizing dynamic binocular stereo vision using only a single feature point according to claim 4, characterized in that: According to the calculated deviation of the rotation angles of the left and right cameras in the BSV model, the process of calculating the external parameters of the rotated BSV model is as follows: The final rotation matrix of the BSV model after rotation is defined as The final translation vector is but:

7. The calibration method for realizing dynamic binocular stereo vision using only a single feature point according to claim 1, characterized in that: The pitch angle measurement accuracy of the attitude angle measuring instrument is 0.2°, the yaw angle measurement accuracy is 1°, and the measurement range of all axes is 360°.

8. A calibration system for realizing dynamic binocular stereo vision using only a single feature point, characterized in that: include: A determination module is constructed to construct a non-zoom dynamic BSV model, and Zhang's method is used to determine the internal parameters and initial relative positions of the left and right cameras in the BSV model in advance, and any of the cameras has the freedom of horizontal and vertical rotation under the condition of fixed position; An acquisition module, used to acquire the approximate values ​​of the rotation angles of the left and right cameras in the BSV model in real time using an attitude angle measuring instrument, wherein the rotation angles include the pitch angle of the camera in the vertical direction and the yaw angle in the horizontal direction; The first calculation module is used to obtain the approximate value of the rotation angle of each camera as an initial value according to the homography between images, and use a single feature point to iteratively solve the rotation angle corresponding to each camera through the least square method; A second calculation module is used to calculate the rotation matrix and translation vector of the BSV model after rotation according to the solved rotation angles of the left and right cameras; A third calculation module is used to establish a basic matrix to associate corresponding image points according to visual geometry, and use an unconstrained nonlinear optimization algorithm to calculate the deviation of the rotation angles of the left and right cameras in the BSV model; The fourth calculation module is used to calculate the external parameters of the rotated BSV model, including the final rotation matrix and the final translation vector, according to the calculated deviation of the rotation angles of the left and right cameras in the BSV model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the calibration method for realizing dynamic binocular stereo vision using only a single feature point as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program. When the computer program is executed on a computer, the computer executes the calibration method for realizing dynamic binocular stereo vision using only a single feature point as described in any one of claims 1 to 7.

Images