A method of solving a camera matrix for a surveillance camera within a three-dimensional model

Abstract

The application discloses a kind of methods for solving camera matrix of monitoring camera in three-dimensional model, comprising: obtaining data and preprocessing;Measure the same point: find at least 5 same points in monitoring camera image and three-dimensional real scene model, and record the image coordinate and object coordinate of same point;Object coordinate is transformed into camera coordinate system, based on the image coordinate of same point and object coordinate under camera coordinate system, and focus three-point collinear, construct equation group;After solving equation group, it is transformed into world coordinate system by coordinate, and output camera matrix.The application solves the problem that existing method is difficult to obtain camera internal parameter and external parameter, and the cost is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a method for calculating a camera matrix of a monitoring camera in a three-dimensional model. Background Art

[0002] When fusing surveillance camera images or videos with a 3D geographic model, a camera matrix is ​​required to relate the surveillance camera image coordinate system to the 3D geographic model coordinate system. The camera matrix is ​​the product of the camera intrinsic parameter matrix and the camera extrinsic parameter matrix. In traditional photogrammetry systems, the camera matrix is ​​calculated by converting the camera's intrinsic orientation elements (camera intrinsic parameters) obtained through camera calibration and the camera's extrinsic orientation elements (camera extrinsics) calculated through the back intersection solution. However, the camera intrinsic and extrinsic parameters of this method are difficult to obtain, and require a variety of external instruments and equipment, with high acquisition costs. It cannot be achieved only with monitoring cameras and three-dimensional models. For example, the mainstream Zhang Youding calibration method to obtain camera intrinsic parameters requires the use of a monitoring camera to take close-up, multi-angle photos of the chessboard. This is not easy to implement for installed monitoring cameras. The camera needs to be disassembled to obtain its intrinsic parameter data using camera calibration. For example, the image control points used to measure the camera extrinsic parameters require measuring instruments such as GNSS, which requires a large amount of field work. Moreover, when fusing a large amount of imaging data into the latest three-dimensional model, there is also the problem of difficulty in calibrating the camera intrinsic parameters. According to traditional photogrammetry methods, without accurate camera intrinsic parameters, it is impossible to calculate the camera extrinsic parameters through single-image resection, and thus it is impossible to calculate the transformation parameters required for fusion. Summary of the Invention

[0003] (1) Technical issues to be resolved

[0004] Based on the above problems, the present invention provides a method for solving the camera matrix of a surveillance camera in a three-dimensional model, which solves the problem that the existing methods are difficult to obtain camera intrinsic and extrinsic parameters and the acquisition cost is high.

[0005] (2) Technical solution

[0006] Based on the above technical problems, the present invention provides a method for calculating the camera matrix of a surveillance camera in a three-dimensional model, comprising:

[0007] S1. Data acquisition and preprocessing: Acquire the image of the monitoring camera C and mark the position of the monitoring camera C in the 3D real scene model W as its focus position W C ;

[0008] S2. Measure the same-name points: Find the same-name points T in the surveillance camera image and the 3D real-world model n And record the same-name point T i The image coordinate P i (u, v) and object coordinates W i (XW ,Y W ,Z W ), n≥5, 1≤i≤n;

[0009] S3, coordinate transformation to camera coordinates: by translating the object coordinate W in the world coordinate system i Convert to object coordinate W in camera coordinate system i ’ ;

[0010] S4. Constructing the equation system: based on the image coordinates P of the points with the same name i And the object coordinate W in the camera coordinate system i ’ , and the three focal points are collinear, constructing the equation system:

[0011]

[0012] Matrix M is the object coordinate W in the camera coordinate system i ’ Image coordinate P to pixel coordinate system i The transformation matrix of

[0013] S5. Solving the equation group: performing a direct linear transformation on the equation group to obtain the camera projection matrix R, and converting the projection matrix R into a corresponding transformation matrix M;

[0014] S6, coordinate transformation to world coordinate: transform the transformation matrix M from camera coordinate to world coordinate;

[0015] S7. Output camera matrix: The transformed change matrix M is the camera matrix result, which is used to obtain the corresponding photographic light of each pixel in the monitoring camera in the corresponding three-dimensional model.

[0016] Furthermore, in S1, the three-dimensional real scene model is any three-dimensional model containing real scene object information represented in three dimensions, including an oblique photography model, a BIM model, a manual model, a three-dimensional point cloud or any combination of the above models.

[0017] Furthermore, in S2, the points with the same name are obtained randomly.

[0018] Furthermore, the S3 includes: converting the object coordinate W i The origin of the corresponding coordinate system is translated to W C Place.

[0019] Furthermore, in S5, the method for obtaining the camera projection matrix R includes singular value decomposition.

[0020] Furthermore, in S6, the projection matrix R is R 9*1The column matrix of the transformation matrix M is M 3*3 phalanx.

[0021] Furthermore, in S6, converting the transformation matrix M from the camera coordinates to the world coordinates includes: converting M 3*3 Matrix transformation from the world coordinate system origin to the camera focus W C The translation transformation is combined to obtain the augmented matrix:

[0022]

[0023] in, is the transformed change matrix M, that is, the camera matrix result.

[0024] Furthermore, the step S1 may further include:

[0025] S0. Check data: Check whether the 3D real scene model contains the surveillance camera and its main monitoring area.

[0026] The present invention also discloses a system for calculating a camera matrix of a surveillance camera in a three-dimensional model, comprising:

[0027] at least one processor; and at least one memory communicatively coupled to the processor, wherein:

[0028] The memory stores program instructions that can be executed by the processor, and the processor can execute the method by calling the program instructions.

[0029] The present invention also discloses a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions enable the computer to execute the method.

[0030] (3) Beneficial effects

[0031] The above technical solution of the present invention has the following advantages:

[0032] (1) The present invention does not require the camera's internal parameter information, including focal length, CCD size, pixel size, lens distortion parameters, etc., and does not require the camera to be removed from its installed position for any operation. It also does not rely on other hardware devices. Instead, it is based on monitoring camera data and a three-dimensional real-scene model. After marking the spatial position of the monitoring camera in the three-dimensional real-scene model, only the image data of five points of the same name and the camera position information that can be easily observed are needed to calculate the camera matrix of the monitoring camera in this three-dimensional model. The data is easy to obtain and the acquisition cost is low. The calculation method is not complicated, and the camera matrix can be obtained more quickly.

[0033] (2) The present invention avoids the possibility of erroneous solutions and the difficulty in eliminating them in the traditional single-image resection algorithm by pre-marking the location of the monitoring camera, thereby greatly improving the accuracy and stability of the results;

[0034] (3) The data collection of the present invention is easy, does not require a variety of external instruments and equipment, and does not require the collection of a large amount of data, thus saving equipment costs, labor costs and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0036] Figure 1 Schematic diagram of a flow chart of a method for calculating a camera matrix of a surveillance camera within a three-dimensional model according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of three points on a collinear line according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0039] The embodiment of the present invention discloses a method for calculating the camera matrix of a surveillance camera in a three-dimensional model. Figure 1 As shown, the following steps are included:

[0040] S0. Check data: Check whether the 3D real scene model contains the surveillance camera and its main monitoring area;

[0041] S1. Data acquisition and preprocessing: Acquire the image of the monitoring camera C and mark the position of the monitoring camera C in the 3D real scene model W as its focus position W C ;

[0042] A 3D real scene model is any 3D model that contains real scene and object information in a 3D manner, including oblique photography models, BIM models, manual models, 3D point clouds, or any combination of the above.

[0043] Surveillance cameras generally refer to any fixed-position camera capable of capturing visible light or infrared images, including both rotatable and fixed cameras. For rotatable cameras, simply adjust the image coordinates of points with the same name based on angle and frame to calculate the camera matrix for the corresponding state using this method.

[0044] S2. Measure the same-name points: Find the same-name points T in the surveillance camera image and the 3D real-world model n And record the same-name point Ti The image coordinate P i (u, v) and object coordinates W i (X W ,Y W ,Z W ), n≥5, 1≤i≤n;

[0045] The image space coordinates refer to the image coordinates of the same-name points in pixel units on the two-dimensional image of the monitoring camera, and the object space coordinates refer to the three-dimensional space coordinates of the same-name points in the world coordinate system where the three-dimensional real scene model is located; the same-name points are obtained randomly.

[0046] S3, coordinate transformation to camera coordinates: by translating the object coordinate W in the world coordinate system i Convert to object coordinate W in camera coordinate system i ’ , that is: the object coordinate W i The origin of the corresponding world coordinate system is translated to W C Department;

[0047] The object coordinate W i Corresponding to the origin O of the world coordinate system w Pan to camera focus W C At, we get the origin of the camera coordinate system O c , obtain the object coordinate W in the corresponding camera coordinate system i ’ (X W ’ ,Y W ’ ,Z W ’ ), the world coordinates of the object point are converted to camera coordinates by translation.

[0048] S4. Constructing the equation system: based on the image coordinates P of the points with the same name i And the object coordinate W in the camera coordinate system i ’ , and the three focal points are collinear, constructing a system of equations;

[0049] Image coordinates P based on the same-name points i , object coordinate W in the camera coordinate system i ’ , and W i ’ The origin of the corresponding coordinate system is the three focal points, which are collinear. Figure 2 As shown; construct the corresponding equations:

[0050]

[0051] Where M is the object coordinate W in the camera coordinate systemi ’ Image coordinate P to pixel coordinate system i The transformation matrix.

[0052] S5. Solving the equation group: performing a direct linear transformation on the equation group to obtain the camera projection matrix R, and converting the projection matrix R into a corresponding transformation matrix M;

[0053] Solving the equations by direct linear transformation (DLT) yields:

[0054]

[0055] R=[r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T

[0056] A 2n×9 R 9×1 =0, n≥5

[0057] DLT (Direct Linear Transform) is a method based on epipolar geometry for estimating the camera's projection matrix. Using known 3D points and their corresponding points in the image, DLT establishes a system of linear equations and solves them to obtain the camera's projection matrix. However, DLT can only determine the camera's projection matrix and cannot directly determine the camera's position and attitude.

[0058] There are many ways to solve the DLT algorithm, which can be solved by using singular value decomposition SVD or other methods.

[0059] The projection matrix R is R 9*1 The column matrix of the transformation matrix M is M 3*3 The projection matrix R is a transformation matrix of the transformation matrix M, and can be directly converted into the corresponding transformation matrix M through the projection matrix R.

[0060] S6, coordinate transformation to world coordinate: transform the transformation matrix M from camera coordinate to world coordinate;

[0061] M 3*3 The matrix is ​​from the camera coordinate O c Convert to world coordinate O w , that is: M 3*3 Matrix transformation and the world coordinate system origin O wTo camera focus W C The translation transformation of is combined to obtain the augmented matrix:

[0062]

[0063] S3 converts the world coordinates into camera coordinates, and the three-dimensional coordinates are displayed based on the world coordinates, so it is necessary to translate the entire result to the world coordinate system based on the coordinates of the camera focus in the world coordinate system, that is, M 3*3 Matrix transformation from the world coordinate system origin to the camera focus W C The translation transformation is merged.

[0064] S7, output camera matrix: the transformed change matrix M, that is is the camera matrix result, which is used to obtain the photographic light corresponding to each pixel in the monitoring camera in the corresponding three-dimensional model.

[0065] According to the camera matrix calculated based on the 3D model, the corresponding photographic light of each pixel in the surveillance camera in the corresponding 3D model world can be restored. This is the basic parameter required to realize the projection and fusion of the image or video from the surveillance camera into the 3D model.

[0066] Finally, it should be noted that the above method can be converted into software program instructions, which can be implemented by running a system including a processor and a memory, or by computer instructions stored in a non-transitory computer-readable storage medium. The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0067] In summary, the above method for calculating the camera matrix of a surveillance camera within a three-dimensional model has the following beneficial effects:

[0068] (1) The present invention does not require the camera's internal parameter information, including focal length, CCD size, pixel size, lens distortion parameters, etc., and does not require the camera to be removed from its installed position for any operation. It also does not rely on other hardware devices. Instead, it is based on monitoring camera data and a three-dimensional real-scene model. After marking the spatial position of the monitoring camera in the three-dimensional real-scene model, only the image data of five points of the same name and the camera position information that can be easily observed are needed to calculate the camera matrix of the monitoring camera in this three-dimensional model. The data is easy to obtain and the acquisition cost is low. The calculation method is not complicated and the camera matrix can be obtained quickly.

[0069] (2) The present invention avoids the possibility of erroneous solutions and the difficulty in eliminating them in the traditional single-image resection algorithm by pre-marking the location of the monitoring camera, thereby greatly improving the accuracy and stability of the results;

[0070] (3) The data collection of the present invention is easy, does not require a variety of external instruments and equipment, and does not require the collection of a large amount of data, thus saving equipment costs, labor costs and time costs.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall fall within the scope defined by the appended claims.

Claims

1. A method for calculating the camera matrix of a surveillance camera in a three-dimensional model, characterized in that: include: S1. Data acquisition and preprocessing: Acquire the image of the monitoring camera C and mark the position of the monitoring camera C in the 3D real scene model W as its focus position W C ; S2. Measure the same-name points: Find the same-name points T in the surveillance camera image and the 3D real-world model n And record the same-name point T i The image coordinate P i (u, v) and object coordinates W i (X W ,Y W ,Z W ), n≥5, 1≤i≤n; S3, coordinate transformation to camera coordinates: by translating the object coordinate W in the world coordinate system i Convert to object coordinate W in camera coordinate system i '; S4. Constructing the equation system: based on the image coordinates P of the points with the same name i And the object coordinate W in the camera coordinate system i ', and the three focal points are collinear, constructing the equation system: Matrix M is the object coordinate W in the camera coordinate system i 'Image coordinate P to pixel coordinate system i The transformation matrix of S5. Solve the equation group: Perform a direct linear transformation on the equation group to obtain the camera projection matrix R, and convert the projection matrix R into the corresponding transformation matrix M. The transformation matrix M is M 3*3 phalanx; Solve the equations by direct linear transformation DLT and get: R＝[r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T A 2n×9 R 9×1 =0,n≥5 The projection matrix R is R 9*1 The column matrix of the transformation matrix M is M 3*3 The projection matrix R is a transformation matrix of the transformation matrix M, which is directly converted into the corresponding transformation matrix M through the projection matrix R. S6, coordinate transformation to world coordinate: transform the transformation matrix M from camera coordinate to world coordinate; The method of converting the transformation matrix M from the camera coordinate to the world coordinate includes: converting M 3*3 Matrix transformation from the world coordinate system origin to the camera focus W C The translation transformation is combined to obtain the augmented matrix: in, is the transformed change matrix M, i.e. the camera matrix result; S7. Output camera matrix: The transformed change matrix M is the camera matrix result, which is used to obtain the corresponding photographic light of each pixel in the monitoring camera in the corresponding three-dimensional model.

2. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, wherein: In S1, the three-dimensional real scene model is any three-dimensional model containing real scene object information represented in three dimensions, including an oblique photography model, a BIM model, a manual model, a three-dimensional point cloud or any combination of the above models.

3. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, wherein: In S2, the points with the same name are obtained randomly.

4. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, wherein: The S3 includes: i The origin of the corresponding coordinate system is translated to W C Place.

5. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, wherein: In S5, the method for obtaining the projection matrix R of the camera includes singular value decomposition.

6. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, wherein: In S6, the projection matrix R is R 9*1 The column matrix of the transformation matrix M is M 3*3 phalanx.

7. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 6, wherein: In S6, the transformation matrix M is converted from the camera coordinate to the world coordinate, including: 3*3 Matrix transformation from the world coordinate system origin to the camera focus W C The translation transformation is combined to obtain the augmented matrix: in, is the transformed change matrix M, that is, the camera matrix result.

8. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, wherein: The S1 also includes: S0. Check data: Check whether the 3D real scene model contains the surveillance camera and its main monitoring area.

9. A system for calculating the camera matrix of a surveillance camera in a three-dimensional model, characterized in that: include: at least one processor; and at least one memory communicatively connected to the processor, wherein: The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method according to any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions, which cause the computer to execute the method according to any one of claims 1 to 8.

Images