Method for resolving camera matrix of monitoring camera in three-dimensional model

Abstract

The invention discloses a method for resolving a camera matrix of a monitoring camera in a three-dimensional model. The method comprises the steps that data are acquired and preprocessed; measuring homonymy points: finding out at least five homonymy points in the monitoring camera image and the three-dimensional real scene model, and recording image space coordinates and object space coordinates of the homonymy points; converting the object space coordinates into image space coordinates based on homonymy points and object space coordinates in a camera coordinate system under the camera coordinate system, and constructing an equation set according to three-point collinearity of a focus; and after solving the equation set, converting coordinates into a world coordinate system, and outputting a camera matrix. The method solves the problems that an existing method is difficult to obtain internal parameters and external parameters of the camera, and the obtaining cost is high.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a method for calculating a camera matrix of a monitoring camera in a three-dimensional model. Background Art

[0002] When the surveillance camera image or video is integrated with the 3D geographic model, the camera matrix is ​​needed to associate the surveillance camera image coordinate system with the 3D geographic model coordinate system. The camera matrix is ​​the product of the camera intrinsic parameter matrix and the camera extrinsic parameter matrix. In the traditional photogrammetry system, the camera matrix is ​​calculated by converting the camera internal orientation elements (camera intrinsic parameters) obtained by camera calibration and the camera external orientation elements (camera extrinsic parameters) calculated by the back intersection solution. However, the camera intrinsics and extrinsics of this method are difficult to obtain and require a variety of external instruments and equipment. The acquisition cost is high and cannot be achieved only with monitoring cameras and 3D models. For example, the mainstream Zhang Youding calibration method requires the use of a monitoring camera to take close-up and multi-angle photos of the chessboard, which is not easy to achieve for installed monitoring cameras. The camera needs to be disassembled to obtain its intrinsic parameter data using camera calibration. For example, the image control points used to measure the camera extrinsics require measuring instruments such as GNSS, which requires a large amount of field work. Moreover, when integrating a large amount of imaging data into the latest 3D model, it is also difficult to calibrate the camera intrinsics. According to traditional photogrammetry methods, without accurate camera intrinsics, it is impossible to calculate the camera extrinsics through single-image resection, and thus it is impossible to calculate the transformation parameters required for fusion. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] Based on the above problems, the present invention provides a method for solving the camera matrix of a monitoring camera in a three-dimensional model, which solves the problem that the existing methods are difficult to obtain camera intrinsic and extrinsic parameters and the acquisition cost is high.

[0005] (II) Technical solution

[0006] Based on the above technical problems, the present invention provides a method for calculating the camera matrix of a surveillance camera in a three-dimensional model, comprising:

[0007] S1. Data acquisition and preprocessing: Acquire the image of the monitoring camera C, and mark the position of the monitoring camera C in the 3D real scene model W as its focus position W C ;

[0008] S2. Measure the same-name points: Find the same-name points T in the surveillance camera image and the 3D real-life model n And record the same-name point T i The image coordinates P i (u, v) and object coordinates W i (XW ,Y W ,Z W ), n≥5, 1≤i≤n;

[0009] S3, coordinate transformation to camera coordinates: transform the object coordinate W in the world coordinate system by translation i Convert to object coordinates W in the camera coordinate system i ’ ;

[0010] S4. Constructing the equation system: Image coordinates P based on the same-name points i And the object coordinate W in the camera coordinate system i ’ , and the three focal points are collinear, constructing the equation system:

[0011]

[0012] The matrix M is the object coordinate W in the camera coordinate system i ’ Image coordinates P to pixel coordinate system i The transformation matrix of

[0013] S5, solving the equation group: performing direct linear transformation on the equation group to obtain the projection matrix R of the camera, and converting the projection matrix R into the corresponding transformation matrix M;

[0014] S6, coordinate transformation to world coordinate: transform the transformation matrix M from camera coordinate to world coordinate;

[0015] S7. Output camera matrix: The transformed change matrix M is the camera matrix result, which is used to obtain the corresponding photographic light of each pixel in the monitoring camera in the corresponding three-dimensional model.

[0016] Furthermore, in S1, the three-dimensional real scene model is any three-dimensional model containing real scene object information represented in three dimensions, including an oblique photography model, a BIM model, a manual model, a three-dimensional point cloud or any combination of the above models.

[0017] Furthermore, in S2, the points with the same name are obtained randomly.

[0018] Furthermore, the S3 includes: converting the object coordinate W i The origin of the corresponding coordinate system is translated to W C Place.

[0019] Furthermore, in S5, the method for solving the projection matrix R of the camera includes singular value decomposition.

[0020] Further, in S6, the projection matrix R is R 9*1The column matrix of the transformation matrix M is M 3*3 phalanx.

[0021] Further, in S6, converting the transformation matrix M from the camera coordinates to the world coordinates includes: converting M 3*3 Matrix transformation from the world coordinate system origin to the camera focus W C The translation transformation is combined to obtain the augmented matrix:

[0022]

[0023] in, is the transformed change matrix M, that is, the camera matrix result.

[0024] Furthermore, before S1, the step further includes:

[0025] S0. Check data: Check whether the 3D real scene model contains the surveillance camera and its main surveillance area.

[0026] The present invention also discloses a system for calculating a camera matrix of a surveillance camera in a three-dimensional model, comprising:

[0027] at least one processor; and at least one memory in communication with the processor, wherein:

[0028] The memory stores program instructions that can be executed by the processor, and the processor can execute the method by calling the program instructions.

[0029] The present invention also discloses a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the method.

[0030] (III) Beneficial effects

[0031] The above technical solution of the present invention has the following advantages:

[0032] (1) The present invention does not require the camera's internal parameter information including focal length, CCD size, pixel size, lens distortion parameters, etc., does not require the camera to be removed from an installed position for any operation, and does not rely on other hardware devices. Instead, the present invention is based on monitoring camera data and a three-dimensional real scene model. After marking the spatial position of the monitoring camera in the three-dimensional real scene model, only the image data of five points of the same name and the camera position information that can be easily observed are needed to calculate the camera matrix of the monitoring camera in the three-dimensional model. The data is easy to obtain and the acquisition cost is low. The calculation method is not complicated, and the camera matrix can be obtained more quickly.

[0033] (2) The present invention avoids the possibility of erroneous solutions and the difficulty in eliminating them in the traditional single-image resection algorithm by pre-marking the position of the monitoring camera, thereby greatly improving the accuracy and stability of the results;

[0034] (3) The data collection of the present invention is easy, does not require a variety of external instruments and equipment, and does not require the collection of a large amount of data, thus saving equipment costs, labor costs and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0036] Figure 1 A schematic flow chart of a method for calculating a camera matrix of a surveillance camera in a three-dimensional model according to an embodiment of the present invention; Figure 2 Schematic diagram of three points on a collinear line according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0038] The embodiment of the present invention discloses a method for calculating the camera matrix of a surveillance camera in a three-dimensional model. Figure 1 As shown, the following steps are included:

[0039] S0. Check data: Check whether the 3D real scene model contains the surveillance camera and its main surveillance area;

[0040] S1. Data acquisition and preprocessing: Acquire the image of the monitoring camera C, and mark the position of the monitoring camera C in the 3D real scene model W as its focus position W C ;

[0041] The three-dimensional real scene model is any three-dimensional model that contains real scene information represented in three dimensions, including oblique photography models, BIM models, manual models, three-dimensional point clouds, or any combination of the above models.

[0042] Surveillance cameras refer to all fixed cameras with visible light or infrared light recording functions, including cameras that can rotate and cameras that cannot rotate. For cameras that can rotate, you only need to adjust the image coordinates of the same-name points according to the angle and frame, and then refer to this method to calculate the camera matrix of the corresponding state.

[0043] S2. Measure the same-name points: Find the same-name points T in the surveillance camera image and the 3D real-life model n And record the same-name point T iThe image coordinates P i (u, v) and object coordinates W i (X W ,Y W ,Z W ), n≥5, 1≤i≤n;

[0044] The image coordinates refer to the image coordinates of the same-name points in pixel units on the two-dimensional image of the monitoring camera, and the object coordinates refer to the three-dimensional space coordinates of the same-name points in the world coordinate system where the three-dimensional real scene model is located; the same-name points are obtained randomly.

[0045] S3, coordinate transformation to camera coordinates: transform the object coordinate W in the world coordinate system by translation i Convert to object coordinates W in the camera coordinate system i ’ , that is: the object coordinate W i The origin of the corresponding world coordinate system is translated to W C Department;

[0046] The object coordinate W i Corresponding to the origin O of the world coordinate system w Pan to camera focus W C At the origin of the camera coordinate system, O c , get the object coordinate W in the corresponding camera coordinate system i ’ (X W ’ ,Y W ’ ,Z W ’ ), the world coordinates of the object point are converted to camera coordinates by translation.

[0047] S4. Constructing the equation system: Image coordinates P based on the same-name points i And the object coordinate W in the camera coordinate system i ’ , and the three focal points are collinear, constructing a system of equations;

[0048] Image coordinates P based on the same-name points i , object coordinate W in the camera coordinate system i ’ , and W i ’ The origin of the corresponding coordinate system is the three focal points, which are collinear. Figure 2 As shown; construct the corresponding equation system:

[0049]

[0050] Where M is the object coordinate W in the camera coordinate system i’ Image coordinates P to pixel coordinate system i The transformation matrix of .

[0051] S5, solving the equation group: performing direct linear transformation on the equation group to obtain the projection matrix R of the camera, and converting the projection matrix R into the corresponding transformation matrix M;

[0052] Solving the equations by direct linear transformation (DLT) yields:

[0053]

[0054] R=[r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T

[0055] A 2n×9 R 9×1 =0, n≥5

[0056] DLT (Direct Linear Transform) is a method based on epipolar geometry to estimate the camera's projection matrix. The DLT method establishes a linear equation system through known 3D points and their corresponding points in the image, and obtains the camera's projection matrix by solving the equation system. The DLT method can only solve the camera's projection matrix, but cannot directly obtain the camera's position and posture.

[0057] There are many ways to solve the DLT algorithm, which can be solved by singular value decomposition SVD or other methods.

[0058] The projection matrix R is R 9*1 The column matrix of the transformation matrix M is M 3*3 The projection matrix R is a transformed form of the transformation matrix M, and can be directly converted into the corresponding transformation matrix M through the projection matrix R.

[0059] S6, coordinate transformation to world coordinate: transform the transformation matrix M from camera coordinate to world coordinate;

[0060] M 3*3 The matrix is ​​from the camera coordinate O c Convert to world coordinates O w , that is: M 3*3 Matrix transformation and the origin of the world coordinate system O w To camera focus WC The translation transformation of is combined to obtain the augmented matrix:

[0061]

[0062] S3 converts world coordinates into camera coordinates, and the 3D coordinates are displayed based on the world coordinates, so it is necessary to translate the entire result to the world coordinate system based on the coordinates of the camera focus in the world coordinate system, that is, 3*3 Matrix transformation from the world coordinate system origin to the camera focus W C The translation transformation is merged.

[0063] S7, output camera matrix: the transformed change matrix M, that is, It is the camera matrix result, which is used to obtain the photographic light corresponding to each pixel in the monitoring camera in the corresponding three-dimensional model.

[0064] According to the camera matrix calculated based on the three-dimensional model, the corresponding photographic light of each pixel in the surveillance camera in the corresponding three-dimensional model world can be restored. This is the basic parameter required to realize the projection and fusion of the image or video in the surveillance camera into the three-dimensional model.

[0065] Finally, it should be noted that the above method can be converted into software program instructions, which can be implemented by using a system including a processor and a memory, or by computer instructions stored in a non-transitory computer-readable storage medium. The above integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above software functional unit is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0066] In summary, the above method for calculating the camera matrix of a surveillance camera in a three-dimensional model has the following beneficial effects:

[0067] (1) The present invention does not require the camera's internal parameter information including focal length, CCD size, pixel size, lens distortion parameters, etc., does not require the camera to be removed from an installed position for any operation, and does not rely on other hardware devices. Instead, the present invention is based on monitoring camera data and a three-dimensional real scene model. After marking the spatial position of the monitoring camera in the three-dimensional real scene model, only the image data of five points of the same name and the camera position information that can be easily observed are needed to calculate the camera matrix of the monitoring camera in the three-dimensional model. The data is easy to obtain and the acquisition cost is low. The calculation method is not complicated and the camera matrix can be quickly obtained.

[0068] (2) The present invention avoids the possibility of erroneous solutions and the difficulty in eliminating them in the traditional single-image resection algorithm by pre-marking the position of the monitoring camera, thereby greatly improving the accuracy and stability of the results;

[0069] (3) The data collection of the present invention is easy, does not require a variety of external instruments and equipment, and does not require the collection of a large amount of data, thus saving equipment costs, labor costs and time costs.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the embodiments of the present invention are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations shall fall within the scope defined by the appended claims.

Claims

1. A method for calculating the camera matrix of a surveillance camera in a three-dimensional model, characterized in that: include: S1. Data acquisition and preprocessing: Acquire the image of the monitoring camera C, and mark the position of the monitoring camera C in the 3D real scene model W as its focus position W C ; S2. Measure the same-name points: Find the same-name points T in the surveillance camera image and the 3D real-life model n And record the same-name point T i The image coordinates P i (u, v) and object coordinates W i (X W ,Y W ,Z W ), n≥5, 1≤i≤n; S3, coordinate transformation to camera coordinates: transform the object coordinate W in the world coordinate system by translation i Convert to object coordinates W in the camera coordinate system i ’ ; S4. Constructing the equation system: Image coordinates P based on the same-name points i And the object coordinate W in the camera coordinate system i ’ , and the three focal points are collinear, constructing the equation system: The matrix M is the object coordinate W in the camera coordinate system i ’ Image coordinates P to pixel coordinate system i The transformation matrix of S5, solving the equation group: performing direct linear transformation on the equation group to obtain the projection matrix R of the camera, and converting the projection matrix R into the corresponding transformation matrix M; S6, coordinate transformation to world coordinate: transform the transformation matrix M from camera coordinate to world coordinate; S7. Output camera matrix: The transformed change matrix M is the camera matrix result, which is used to obtain the corresponding photographic light of each pixel in the monitoring camera in the corresponding three-dimensional model.

2. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, characterized in that: In S1, the three-dimensional real scene model is any three-dimensional model containing real scene object information represented in three dimensions, including an oblique photography model, a BIM model, a manual model, a three-dimensional point cloud or any combination of the above models.

3. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, characterized in that: In S2, the points with the same name are obtained randomly.

4. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, characterized in that: The S3 includes: i The origin of the corresponding coordinate system is translated to W C Place.

5. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, characterized in that: In S5, the method for solving the projection matrix R of the camera includes singular value decomposition.

6. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, characterized in that: In S6, the projection matrix R is R 9*1 The column matrix of the transformation matrix M is M 3*3 phalanx.

7. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 6, characterized in that: In S6, converting the transformation matrix M from the camera coordinates to the world coordinates includes: 3*3 Matrix transformation from the world coordinate system origin to the camera focus W C The translation transformation is combined to obtain the augmented matrix: in, is the transformed change matrix M, that is, the camera matrix result.

8. The method for calculating the camera matrix of a surveillance camera in a three-dimensional model according to claim 1, characterized in that: The S1 also includes: S0. Check data: Check whether the 3D real scene model contains the surveillance camera and its main surveillance area.

9. A system for calculating the camera matrix of a surveillance camera in a three-dimensional model, characterized in that: include: at least one processor; and at least one memory in communication with the processor, wherein: The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 8 by calling the program instructions.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions that cause the computer to execute the method according to any one of claims 1 to 8.

Images