Camera calibration method, device, electronic device and readable storage medium

Abstract

The embodiments of the present disclosure disclose a camera calibration method, apparatus, electronic device, and readable storage medium. The method includes: for each image target point in an image obtained by photographing a target object by a camera, obtaining a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, where the object target point is a point on the target object corresponding to the image target point; and determining camera parameters of the camera based on the first coordinates of multiple image target points and the second coordinates of multiple corresponding object target points.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a camera calibration method, device, electronic device, and readable storage medium. Background Art

[0002] In real-world urban environments, surveillance cameras, as inexpensive, intuitive, and efficient visual sensors, are widely used in various industries, including municipal administration and transportation. The information extracted from surveillance cameras can effectively assist in urban governance. However, there are differences between the two-dimensional properties of surveillance images and the three-dimensional properties of space. Mapping this two-dimensional information to the three-dimensional information of real-world scenes is crucial. The core technology for this is camera calibration, which uses the calibrated camera parameters to back-project the two-dimensional image information into three-dimensional space. Summary of the Invention

[0003] In order to solve the problems in the related art, the embodiments of the present disclosure provide a camera calibration method, device, electronic device and readable storage medium.

[0004] In a first aspect, an embodiment of the present disclosure provides a camera calibration method.

[0005] Specifically, the camera calibration method includes:

[0006] For each image target point in an image obtained by photographing a target object with a camera, obtaining a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, where the object target point is a point on the target object corresponding to the image target point;

[0007] The camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0008] In combination with the first aspect, in a first implementation of the first aspect of the present disclosure, the method further includes:

[0009] Obtaining a texture map of the target object;

[0010] Extracting and matching feature points on the image of the target object and the texture map to obtain the plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points;

[0011] Determining a plurality of model target points on a target object model constructed based on the texture map corresponding to the plurality of texture map target points;

[0012] The second coordinates of the plurality of object target points are acquired according to the plurality of model target points.

[0013] In combination with the first aspect, the present disclosure discloses a second implementation of the first aspect, wherein:

[0014] The camera is a monocular camera; and / or

[0015] The first coordinate system is the image coordinate system of the camera, and the second coordinate system is the world coordinate system in the space where the target object is located or the two-dimensional coordinate system on the plane where the target object is located; and / or

[0016] The camera parameters include camera intrinsic parameters and extrinsic parameters.

[0017] In combination with the first aspect, in a third implementation manner of the first aspect of the present disclosure, determining the camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points includes:

[0018] determining a projection matrix of the camera according to first coordinates of a plurality of the image target points and second coordinates of a corresponding plurality of the object target points;

[0019] The camera parameters of the camera are calculated according to the projection matrix.

[0020] In combination with the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect of the present disclosure, the step of calculating the camera parameters of the camera according to the projection matrix includes:

[0021] Perform QR decomposition on the 3*3 matrix consisting of the first three rows and first three columns of the projection matrix to obtain the camera intrinsic parameter matrix K and the rotation matrix R from the second coordinate system to the camera coordinate system under the camera perspective;

[0022] According to the projection matrix, the camera intrinsic parameter matrix K and the rotation matrix R, a translation vector t from the second coordinate system to the camera coordinate system under the camera perspective is determined.

[0023] In combination with the first aspect, in a fifth implementation of the first aspect of the present disclosure, the method further includes:

[0024] Determine a third coordinate of the object target point in a third coordinate system according to the second coordinate of the object target point and the extrinsic parameter of the camera;

[0025] Determining, according to the third coordinate of the object target point and the intrinsic parameter of the camera, a fourth coordinate of a point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system;

[0026] determining an error function according to the first coordinate and the fourth coordinate of the image target point corresponding to the object target point;

[0027] The camera parameters are optimized with the goal of minimizing the error function.

[0028] In combination with the fifth implementation manner of the first aspect, in a sixth implementation manner of the first aspect of the present disclosure, the third coordinate system is a camera coordinate system of the camera.

[0029] In a second aspect, an embodiment of the present disclosure provides a camera calibration method, including:

[0030] receiving a camera calibration request, the camera calibration request including image identification information, the image identification information being used to identify an image obtained by photographing a target object with a camera;

[0031] Acquire an image of the target object photographed by a camera according to the image identification information;

[0032] Acquire each image target point in the image, obtain a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, wherein the object target point is a point on the target object corresponding to the image target point;

[0033] The camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0034] In a third aspect, an embodiment of the present disclosure provides a camera calibration device.

[0035] Specifically, the camera calibration device includes:

[0036] a first acquisition module configured to acquire, for each image target point in an image obtained by photographing a target object with a camera, a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, wherein the object target point is a point on the target object corresponding to the image target point;

[0037] The first determining module is configured to determine camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0038] In conjunction with the third aspect, in a first implementation of the third aspect of the present disclosure, the apparatus further includes:

[0039] A second acquisition module is configured to acquire a texture map of the target object;

[0040] a third acquisition module, configured to extract and match feature points on the image of the target object and the texture map to obtain the plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points;

[0041] A second determining module is configured to determine a plurality of model target points corresponding to the plurality of texture map target points on a target object model constructed based on the texture map;

[0042] The fourth acquisition module is configured to acquire second coordinates of the plurality of object target points according to the plurality of model target points.

[0043] In conjunction with the third aspect, the present disclosure provides a second implementation of the third aspect, wherein:

[0044] The camera is a monocular camera; and / or

[0045] The first coordinate system is the image coordinate system of the camera, and the second coordinate system is the world coordinate system in the space where the target object is located or the two-dimensional coordinate system on the plane where the target object is located; and / or

[0046] The camera parameters include camera intrinsic parameters and extrinsic parameters.

[0047] In conjunction with the third aspect, in a third implementation of the third aspect of the present disclosure, determining the camera parameters of the camera based on the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points includes:

[0048] determining a projection matrix of the camera according to first coordinates of a plurality of the image target points and second coordinates of a corresponding plurality of the object target points;

[0049] The camera parameters of the camera are calculated according to the projection matrix.

[0050] In combination with the third implementation manner of the third aspect, in a fourth implementation manner of the third aspect of the present disclosure, the step of calculating the camera parameters of the camera according to the projection matrix includes:

[0051] Perform QR decomposition on the 3*3 matrix consisting of the first three rows and first three columns of the projection matrix to obtain the camera intrinsic parameter matrix K and the rotation matrix R from the second coordinate system to the camera coordinate system under the camera perspective;

[0052] According to the projection matrix, the camera intrinsic parameter matrix K and the rotation matrix R, a translation vector t from the second coordinate system to the camera coordinate system under the camera perspective is determined.

[0053] In combination with the third aspect, in a fifth implementation of the third aspect of the present disclosure, the apparatus further includes:

[0054] a third determining module, configured to determine a third coordinate of the object target point in a third coordinate system according to the second coordinate of the object target point and an extrinsic parameter of the camera;

[0055] a fourth determining module configured to determine, based on the third coordinate of the object target point and the intrinsic parameter of the camera, a fourth coordinate of a point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system;

[0056] a fifth determining module configured to determine an error function according to the first coordinate of the image target point corresponding to the object target point and the fourth coordinate;

[0057] The optimization module is configured to optimize the camera parameters with the goal of minimizing the error function.

[0058] In combination with the fifth implementation manner of the third aspect, in a sixth implementation manner of the third aspect of the present disclosure, the third coordinate system is a camera coordinate system of the camera.

[0059] In a fourth aspect, an embodiment of the present disclosure provides a camera calibration device, including:

[0060] a receiving module configured to receive a camera calibration request, wherein the camera calibration request includes image identification information, and the image identification information is used to identify an image obtained by photographing a target object with a camera;

[0061] A fifth acquisition module is configured to acquire an image of the target object photographed by the camera according to the image identification information;

[0062] a first acquisition module configured to acquire each image target point in the image, acquire a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, wherein the object target point is a point on the target object corresponding to the image target point;

[0063] The first determining module is configured to determine camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0064] In a fifth aspect, an embodiment of the present disclosure provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first to second aspects.

[0065] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the methods described in the first to second aspects are implemented.

[0066] According to the technical solution provided by the embodiment of the present disclosure, for each image target point in the image obtained by photographing the target object by the camera, the first coordinate of the image target point in the first coordinate system and the second coordinate of the corresponding object target point in the second coordinate system are obtained, and the object target point is the point on the target object corresponding to the image target point; the camera parameters of the camera are determined according to the first coordinates of the multiple image target points and the second coordinates of the multiple corresponding object target points. According to the embodiment of the present disclosure, the camera can be calibrated directly from a single image obtained by photographing the target object by the camera, and the camera parameters can be solved without the need for additional calibration objects. For monocular cameras used as surveillance cameras, the installation position is usually high and fixed, and it is impossible to move the surveillance camera or take multiple images from different perspectives. The method according to the embodiment of the present disclosure can be used to conveniently calibrate such monocular cameras.

[0067] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0069] Figure 1 A schematic diagram of the overall process of a camera calibration method according to an embodiment of the present disclosure is shown.

[0070] Figure 2A A flowchart of a camera calibration method according to an embodiment of the present disclosure is shown.

[0071] Figure 2B A flowchart of a camera calibration method according to an embodiment of the present disclosure is shown.

[0072] Figure 3A A structural block diagram of a camera calibration device according to an embodiment of the present disclosure is shown.

[0073] Figure 3B A structural block diagram of a camera calibration device according to an embodiment of the present disclosure is shown.

[0074] Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0075] Figure 5 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0076] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0077] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof exist or are added.

[0078] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0079] In this disclosure, the acquisition of user information or user data is an operation authorized and confirmed by the user, or actively selected by the user.

[0080] As mentioned above, in real urban environments, surveillance cameras, as inexpensive, intuitive, and efficient visual sensors, are widely used in various industries, including municipal administration and transportation. The information extracted from surveillance cameras can effectively assist in urban governance. However, there are differences between the two-dimensional properties of surveillance images and the three-dimensional properties of space. Mapping this two-dimensional information to the three-dimensional information of real scenes is a crucial step. The core technology for this is camera calibration, which uses the calibrated camera parameters to back-project the two-dimensional image information into three-dimensional space.

[0081] In applications such as image measurement and machine vision, to determine the relative transformation between the 3D geometric position of a point on a spatial object's surface and its corresponding point in the image, a geometric model of camera imaging must be established. These geometric model parameters include camera intrinsic parameters and camera extrinsic parameters, collectively referred to as camera parameters. The process of determining these parameters is called camera calibration.

[0082] According to an embodiment of the present disclosure, the camera intrinsic parameters include any one or more of the following parameters: camera focal length, principal point position, skew coefficient, and distortion parameters, where the distortion parameters include radial distortion parameters and tangential distortion parameters. According to an embodiment of the present disclosure, the camera extrinsic parameters include the rotation matrix and translation vector from the world coordinate system to the camera coordinate system under the camera's perspective.

[0083] In order to facilitate understanding of the technical solution of the present disclosure, the world coordinate system, camera coordinate system, imaging plane coordinate system, and image coordinate system are first explained.

[0084] According to embodiments of the present disclosure, a reference coordinate system can be established in the space where the camera and the target object being photographed are located to describe the positions of the camera and the target object. This reference coordinate system is called the world coordinate system. The relationship between the camera coordinate system and the world coordinate system can be described using a rotation matrix R and a translation vector t. According to embodiments of the present disclosure, the coordinates of an object in the world coordinate system can be converted using the latitude, longitude, and altitude of the object.

[0085] According to an embodiment of the present disclosure, the imaging plane coordinate system is a two-dimensional coordinate system expressed in physical units (e.g., centimeters). The origin of the imaging plane coordinate system is defined at the intersection of the camera optical axis and the imaging plane. This intersection is called the principal point of the image. The x-axis of the imaging plane coordinate system is the width direction of the photosensitive surface of the camera's photosensitive element (e.g., a charge-coupled device (CCD)). The y-axis of the imaging plane coordinate system is the height direction of the photosensitive surface of the camera's photosensitive element. According to an embodiment of the present disclosure, the imaging plane of the camera is the plane where the photosensitive surface of the photosensitive element is located.

[0086] According to an embodiment of the present disclosure, the origin of the camera coordinate system is the center of projection of the camera, the x-axis and the y-axis are parallel to the x-axis and y-axis of the imaging plane coordinate system respectively, and the z-axis is the optical axis of the camera, which is perpendicular to the imaging plane. The spatial rectangular coordinate system formed in this way is called the camera coordinate system, which is a three-dimensional coordinate system.

[0087] According to an embodiment of the present disclosure, the image coordinate system is a rectangular coordinate system defined on an image, and the coordinates of a point on the image coordinate system are the column number and row number of the point in the pixel matrix of the image.

[0088] As mentioned above, camera calibration is a key step in mapping a 2D image to a 3D object. Existing camera calibration algorithms include traditional camera calibration algorithms, camera self-calibration methods, and active vision camera calibration algorithms.

[0089] (1) Traditional camera calibration algorithm: It requires the use of a three-dimensional calibration object or a flat calibration object with known real size. By establishing a correspondence between the points on the calibration object with known coordinates and its image points, an optimization algorithm is used to solve the internal and external parameters of the camera model. A three-dimensional calibration object can be calibrated with a single image, and the calibration accuracy is high, but the processing and maintenance of high-precision three-dimensional calibration objects are more difficult. Flat calibration objects are simpler to produce than three-dimensional calibration objects, and the accuracy is easier to ensure, but two or more images must be used for calibration.

[0090] (2) Camera self-calibration method: The camera internal and external parameters are calibrated using some parallel or orthogonal constraints in the scene. The intersection of spatial parallel lines on the camera image plane is called the vanishing point. Due to the low accuracy of the vanishing point solution, the camera parameter estimation error is large. In addition, for cameras with distortion, the distortion of spatial parallel lines on the imaging plane will also lead to a decrease in the accuracy of the vanishing point solution, which directly affects the robustness of the calibration.

[0091] (3) Active visual camera calibration algorithm: Using structure-from-motion technology, the camera's motion is analyzed to restore the scene's 3D geometry while optimizing the camera's internal and external parameters. This method does not require a known calibration object; it only requires moving the camera and capturing images in the same scene. However, it is important to ensure that the baseline distance between adjacent images is large enough.

[0092] The common disadvantage of the three types of calibration algorithms is that they cannot solve the external parameters of the camera relative to the geographic coordinate system. Because these algorithms all implement camera calibration based on local three-dimensional information, they naturally cannot achieve the positioning of objects in the image in real space.

[0093] Reference below Figure 1 The principle of the camera calibration method according to the embodiment of the present disclosure is described. In the following formulas, unless otherwise specified, the coordinate transformations around the equal sign default to perspective division, that is, normalizing the coordinates of the last dimension.

[0094] Figure 1 A schematic diagram of the overall process of a camera calibration method according to an embodiment of the present disclosure is shown.

[0095] like Figure 1 As shown, first, a texture map of the target object can be obtained through drone flight images or lidar scanning. For example, in the application scenario of a digital city, a texture map of the urban scene can be obtained using drone flight images or vehicle-mounted lidar scanning. Based on this texture map, a three-dimensional model of the urban scene can be reconstructed. The reconstructed three-dimensional model is typically located in a local coordinate system and lacks true location information and scale. By obtaining the global positioning system (GPS) information of the data acquisition device, a similarity transformation and positioning can be performed on the reconstructed three-dimensional model, and it can be aligned to a geographic coordinate system. In this way, the reconstructed three-dimensional model has the corresponding actual object's location information, such as latitude, longitude, and altitude.

[0096] Feature points are extracted from a single image captured by a camera of a target object in space. These feature points may include any one or more of the following: corner points, SIFT (Scale-Invariant Feature Transform) feature points, ORB (Oriented Fast and Rotated BRIEF) feature points, etc. Feature points of the same type are extracted from the texture map of the target object. A nearest neighbor algorithm is used to match these feature points to determine multiple matching two-dimensional feature points. The matched feature points in the image are referred to as image target points, and the matched feature points in the texture map are referred to as texture map target points. Then, corresponding 3D model target points on the reconstructed 3D model are obtained based on the texture map target points. Because the reconstructed 3D model contains the corresponding position information of the actual target object, the coordinates of the model target points in the world coordinate system can be determined. These coordinates are also the coordinates of the object target points on the target object corresponding to the model target points (and, therefore, the image target points).

[0097] Assume that the coordinates of the image target point x in the image coordinate system are (x1, x2), the image target point x corresponds to the object target point X, and the coordinates of the object target point X in the world coordinate system are (X1, X2, X3). Based on the pinhole imaging model, the camera projection matrix is ​​expressed as a 3*4 matrix P, which satisfies the following relationship in the homogeneous coordinate system:

[0098]

[0099] According to the above relationship, the linear equations are constructed as follows:

[0100]

[0101]

[0102] Substituting the matched image target point coordinates and the corresponding object target point coordinates into equation group (1), the camera projection matrix P can be solved.

[0103] After solving the camera projection matrix P, P can be decomposed to obtain the camera intrinsic parameter matrix K and external parameters R, t, where the intrinsic parameter matrix K is:

[0104]

[0105] Among them, f x is the focal length of the camera in the x-axis direction of the camera coordinate system, f y is the focal length of the camera in the y-axis direction of the camera coordinate system, s is the camera's skew coefficient (usually 0), c xis the x coordinate of the camera principal point in the camera coordinate system, c y is the y coordinate of the camera's principal point in the camera coordinate system. R is the rotation matrix from the world coordinate system to the camera coordinate system under the camera's perspective, and t is the translation vector from the world coordinate system to the camera coordinate system under the camera's perspective.

[0106] According to an embodiment of the present disclosure, let P = (KR | -KRC). The matrix P is a 3*4 matrix. Since it contains the intrinsic and extrinsic parameters of the camera that maps a 3D point in the world coordinate system to a 2D point in the image coordinate system, the matrix P is called the projection matrix of the camera. If the matrix P is known, the intrinsic and extrinsic parameters of the camera can be decomposed from it. The first three rows and three columns of the projection matrix P are composed of KR, and its inverse is R. T K -1 , where the rotation matrix R is an orthogonal matrix, and the matrix K -1 It is a non-singular upper triangular matrix. According to the knowledge of linear algebra, any 3*3 matrix can be uniquely decomposed into the product of an orthogonal matrix and a non-singular upper triangular matrix through QR decomposition. Therefore, the matrices K and R can be calculated respectively, and then the camera center C can be calculated by combining the fourth column of the projection matrix. Camera center C = -R T *t, therefore, after obtaining C and R, t can be obtained.

[0107] According to an embodiment of the present disclosure, if the target points of the object are located on the same plane, for example, when the target object is the ground or wall in the room, the projection matrix P is simplified to a 3*3 homography matrix H:

[0108]

[0109] Since R is a rotation matrix, we get:

[0110]

[0111]

[0112] r 11 r 12 +r 21 r 22 +r 31 r 32 =0 (5)

[0113] Let h ij Representing the (i, j)th component of the matrix H, we can obtain from formulas (2) and (5):

[0114]

[0115] From formulas (3) and (4), we can get:

[0116]

[0117]

[0118] Eliminate λ 2 ,get:

[0119]

[0120] Let f x =1 / α u , f y =1 / α v , from formulas (6) and (9), we can get:

[0121]

[0122]

[0123] in,

[0124]

[0125] In the calculation of α u and α v After that, we can use formula (7) or (8) to calculate λ. So we can get:

[0126] r 11 =λh 11 / α u ,r 21 =λh 21 / α v ,r 31 =λh 31

[0127] r 12 =λh 12 / α u ,r 22 =λh 22 / α v ,r 32 =λh 32

[0128] t1=λh 13 / α u ,t2=λh 23 / α v ,t3=λh 33

[0129] Using the orthogonality of the rotation matrix, r can be easily calculated. i3 (i=1,...,3).

[0130] After calculating the components r of the rotation matrix Rij (i=1,…,3,j=1,…,3), we can get the rotation matrix R. Then, according to formula (2), we can get the three components t of the translation vector t i (i=1,…,3).

[0131] In addition, f x is the focal length of the camera in the x-axis direction of the camera coordinate system, f y is the focal length of the camera in the y-axis direction of the camera coordinate system, the principal point position is the intersection of the camera optical axis and the camera imaging plane, and the skew coefficient defaults to 0, so the camera intrinsic parameter matrix K can be obtained.

[0132] In this way, when the target object is a two-dimensional object, the camera can also be calibrated, which is suitable for the case where the camera is installed indoors, for example.

[0133] According to the above method, the initial values ​​of the camera intrinsic parameter matrix K, the rotation matrix R and the translation vector t can be obtained. Then, the camera distortion is modeled as follows. The general radial distortion and tangential distortion models are used to calculate the distortion in the camera coordinate system. Assume that the coordinates of the object target point X after transformation to the camera coordinate system are x c , after distortion calculation, it is Satisfies the following relationship:

[0134]

[0135]

[0136] in, k1 and k2 represent radial distortion parameters, p1 and p2 represent tangential distortion parameters, Projecting onto the imaging plane yields the estimated pixel coordinates in the image coordinate system: Based on this, the error function is constructed Among them, x i is the target point X of the i-th object i The corresponding image target point, is the target point X i Obtained from the above calculation, N is the total number of target points of the object.

[0137] Using the initial values ​​of the camera intrinsic parameter matrix K, the rotation matrix R, and the translation vector t, the initial values ​​of the distortion parameters are all set to 0. The camera intrinsic parameter matrix K, the rotation matrix R, the translation vector t, and the distortion parameters are optimized by, for example, the Levenberg-Marquardt method or the gradient descent method with the goal of minimizing the error function, thereby obtaining the optimized camera parameters.

[0138] According to the embodiments of the present disclosure, a camera can be calibrated and its parameters determined directly from a single image of a target object, without the need for additional calibration objects. Monocular cameras used as surveillance cameras are typically mounted high and fixed, making it difficult to move the camera or capture multiple images from different perspectives. The methods according to the embodiments of the present disclosure facilitate the calibration of such monocular cameras.

[0139] According to the embodiments of the present disclosure, unlike traditional camera calibration in a local coordinate system, a target object model located in a geographic coordinate system provides 3D feature points with real geographic information to assist camera calibration. The calibration results can be directly applied to pedestrian and vehicle speed estimation in real-world scenarios, augmented reality, and other applications.

[0140] According to the embodiments of the present disclosure, for digital city scenes, this solution proposes a universal surveillance camera calibration algorithm that can realize the positioning of surveillance cameras in virtual city scenes with only a single surveillance image. The positioning position is the real position in the geographic coordinate system.

[0141] According to the embodiments of the present disclosure, this solution only requires a small number of two- and three-dimensional corresponding points to accurately calibrate camera parameters.

[0142] The optimization strategy according to the embodiments of the present disclosure can be extended to arbitrary imaging models and distortion models.

[0143] Figure 2A FIG. 1 is a flow chart showing a camera calibration method according to an embodiment of the present disclosure. Figure 2A As shown, the camera calibration method includes the following steps S101-S102:

[0144] In step S101, for each image target point in an image obtained by photographing a target object with a camera, a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system are obtained, where the object target point is a point on the target object corresponding to the image target point;

[0145] In step S102 , camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0146] Figure 2B A flowchart of a camera calibration method according to an embodiment of the present disclosure is shown. Figure 2B The method flow shown can be implemented by a server, for example. Figure 2B As shown, the camera calibration method includes steps S103 and S104 in addition to the above steps S101-S102:

[0147] In step S103, a camera calibration request is received, where the camera calibration request includes image identification information, where the image identification information is used to identify an image obtained by photographing a target object with a camera;

[0148] In step S104, an image of the target object photographed by the camera is acquired according to the image identification information;

[0149] In step S101, for each image target point in an image obtained by photographing a target object with a camera, a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system are obtained, where the object target point is a point on the target object corresponding to the image target point;

[0150] In step S102 , camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0151] According to an embodiment of the present disclosure, receiving the camera calibration request in step S203 may be, for example, receiving the camera calibration request from a client.

[0152] According to an embodiment of the present disclosure, the camera is a monocular camera, the first coordinate system is the image coordinate system of the camera, the second coordinate system is the world coordinate system in the space where the target object is located or the two-dimensional coordinate system on the plane where the target object is located, and the camera parameters include camera intrinsic parameters and extrinsic parameters. According to an embodiment of the present disclosure, the first coordinate system and the second coordinate system can be arbitrary coordinate systems.

[0153] As mentioned above Figure 1 As described, according to an embodiment of the present disclosure, the image target point is, for example, the image target point x, the first coordinate system is, for example, the image coordinate system, the first coordinate is, for example, (x1, x2), the object target point is, for example, the object target point X, the second coordinate system is, for example, the world coordinate system, the second coordinate is, for example, (X1, X2, X3). When the target object is a two-dimensional plane object, such as the ground or a wall, the second coordinate system can be a two-dimensional coordinate system on the plane where the target object is located. According to the first coordinate of the image target point and the second coordinate of the corresponding object target point, the projection matrix P of the camera can be obtained, and then the camera parameters can be obtained, such as the intrinsic parameter matrix K, the rotation matrix R and the translation vector t.

[0154] According to the embodiments of the present disclosure, a camera can be calibrated and its parameters determined directly from a single image of a target object, without the need for additional calibration objects. Monocular cameras used as surveillance cameras are typically mounted high and fixed, making it difficult to move the camera or capture multiple images from different perspectives. The methods according to the embodiments of the present disclosure facilitate the calibration of such monocular cameras.

[0155] According to an embodiment of the present disclosure, the camera calibration method further includes: obtaining a texture map of the target object; obtaining the multiple image target points and multiple texture map target points on the texture map corresponding to the multiple image target points by performing feature point extraction and feature point matching on the image of the target object and the texture map; determining multiple model target points on a target object model constructed based on the texture map corresponding to the multiple texture map target points; and obtaining second coordinates of the multiple object target points based on the multiple model target points.

[0156] For example, reference Figure 1 First, a texture map of the target object can be obtained through drone flight images or lidar scanning. For example, in the application scenario of a digital city, a texture map of the urban scene can be obtained using drone flight images or vehicle-mounted lidar scanning, and a three-dimensional model of the urban scene can be reconstructed based on the texture map of the urban scene. The reconstructed three-dimensional model is usually located in a local coordinate system and lacks real-world location information and scale. By obtaining the global positioning system (GPS) information of the data acquisition device, a similarity transformation and positioning can be performed on the reconstructed three-dimensional model, and it can be aligned to the geographic coordinate system. In this way, the reconstructed three-dimensional model has the corresponding actual object's location information, such as latitude, longitude, and altitude.

[0157] Feature points are extracted from a single image captured by a camera of a target object in space. These feature points may include any one or more of the following: corner points, SIFT (Scale-Invariant Feature Transform) feature points, ORB (Oriented Fast and Rotated BRIEF) feature points, etc. Feature points of the same type are extracted from the texture map of the target object. A nearest neighbor algorithm is used to match these feature points to determine multiple matching feature points. The matched feature points in the image are referred to as image target points, and the matched feature points in the texture map are referred to as texture map target points. Then, corresponding model target points on the reconstructed 3D model are obtained based on the texture map target points. Because the reconstructed 3D model contains the corresponding position information of the actual target object, the coordinates of the model target points in the world coordinate system can be determined. These coordinates are also the coordinates of the object target points on the target object corresponding to the model target points (and, therefore, the image target points).

[0158] According to the embodiments of the present disclosure, unlike traditional camera calibration in a local coordinate system, a target object model located in a geographic coordinate system provides 3D feature points with real geographic information to assist camera calibration. The calibration results can be directly applied to pedestrian and vehicle speed estimation in real-world scenarios, augmented reality, and other applications.

[0159] According to the disclosed embodiments, a universal surveillance camera calibration algorithm is proposed for digital city scenarios. Using only a single surveillance image, a surveillance camera can be positioned within a virtual city. The positioning is the actual position in the geographic coordinate system. Specifically, by determining the camera's rotation matrix R and translation vector t, the position of the camera coordinate system relative to the world coordinate system can be determined, thereby determining the camera's position.

[0160] According to an embodiment of the present disclosure, determining the camera parameters of the camera based on the first coordinates of the plurality of image target points and the second coordinates of the corresponding plurality of object target points includes: determining the projection matrix of the camera based on the first coordinates of the plurality of image target points and the second coordinates of the corresponding plurality of object target points; and calculating the camera parameters of the camera based on the projection matrix.

[0161] For example, reference Figure 1 , assuming that the coordinates of the image target point x in the image coordinate system are (x1, x2), the image target point x corresponds to the object target point X, and the coordinates of the object target point X in the world coordinate system are (X1, X2, X3), then according to the pinhole imaging model, the camera projection matrix is ​​expressed as a 3*4 matrix P, then the following relationship is satisfied in the homogeneous coordinate system:

[0162]

[0163] According to the above relationship, a linear equation group (1) is constructed. The matched image target point coordinates and the corresponding object target point coordinates are substituted into the equation group (1) to solve the camera projection matrix P.

[0164] According to an embodiment of the present disclosure, the camera parameters of the camera are calculated based on the projection matrix, including: performing QR decomposition on a 3*3 matrix consisting of the first three rows and first three columns of the projection matrix to obtain a camera intrinsic parameter matrix K and a rotation matrix R from the second coordinate system to the camera coordinate system under the camera perspective; and determining a translation vector t from the second coordinate system to the camera coordinate system under the camera perspective based on the projection matrix, the camera intrinsic parameter matrix K, and the rotation matrix R.

[0165] For example, reference Figure 1 After obtaining the camera projection matrix P, P can be decomposed to obtain the camera intrinsic parameter matrix K and external parameters R, t.

[0166] According to an embodiment of the present disclosure, the camera calibration method also includes: determining the third coordinate of the object target point in a third coordinate system based on the second coordinate of the object target point and the external parameters of the camera; determining the fourth coordinate of the point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system based on the third coordinate of the object target point and the internal parameters of the camera; determining an error function based on the first coordinate and the fourth coordinate of the image target point corresponding to the object target point; and optimizing the camera parameters with the goal of minimizing the error function.

[0167] According to an embodiment of the present disclosure, the third coordinate system is a camera coordinate system of the camera.

[0168] For example, reference Figure 1 , the third coordinate of the target point X in the third coordinate system is x c The fourth coordinate of the point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system is The error function is Among them, x i is the target point X of the i-th object i The corresponding image target point, is the target point X i Obtained from the above calculation, N is the total number of target points of the object.

[0169] Using the initial values ​​of the camera intrinsic parameter matrix K, the rotation matrix R, and the translation vector t, the initial values ​​of the distortion parameters are all set to 0. The camera intrinsic parameter matrix K, the rotation matrix R, the translation vector t, and the distortion parameters are optimized by, for example, the Levenberg-Marquardt method or the gradient descent method with the goal of minimizing the error function, thereby obtaining the optimized camera parameters.

[0170] Figure 3A The following is a block diagram of a camera calibration device according to an embodiment of the present disclosure, wherein the device can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0171] like Figure 3A As shown, the camera calibration device 300 includes a first acquisition module 301 and a first determination module 302 .

[0172] The first acquisition module 301 is configured to acquire, for each image target point in an image obtained by photographing a target object with a camera, a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, where the object target point is a point on the target object corresponding to the image target point;

[0173] The first determining module 302 is configured to determine camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0174] According to an embodiment of the present disclosure, the camera calibration apparatus 300 further includes:

[0175] A second acquisition module 303 is configured to acquire a texture map of the target object;

[0176] A third acquisition module 304 is configured to extract and match feature points on the image of the target object and the texture map to obtain the plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points;

[0177] The second determining module 305 is configured to determine a plurality of model target points on the target object model constructed based on the texture map corresponding to the plurality of texture map target points;

[0178] The fourth acquisition module 306 is configured to acquire second coordinates of the plurality of object target points according to the plurality of model target points.

[0179] According to an embodiment of the present disclosure, wherein:

[0180] The camera is a monocular camera; and / or

[0181] The first coordinate system is the image coordinate system of the camera, and the second coordinate system is the world coordinate system in the space where the target object is located or the two-dimensional coordinate system on the plane where the target object is located; and / or

[0182] The camera parameters include camera intrinsic parameters and extrinsic parameters.

[0183] According to an embodiment of the present disclosure, the determining of the camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points includes:

[0184] determining a projection matrix of the camera according to first coordinates of a plurality of the image target points and second coordinates of a corresponding plurality of the object target points;

[0185] The camera parameters of the camera are calculated according to the projection matrix.

[0186] According to an embodiment of the present disclosure, the step of calculating the camera parameters of the camera according to the projection matrix includes:

[0187] Perform QR decomposition on the 3*3 matrix consisting of the first three rows and first three columns of the projection matrix to obtain the camera intrinsic parameter matrix K and the rotation matrix R from the second coordinate system to the camera coordinate system under the camera perspective;

[0188] According to the projection matrix, the camera intrinsic parameter matrix K and the rotation matrix R, a translation vector t from the second coordinate system to the camera coordinate system under the camera perspective is determined.

[0189] According to an embodiment of the present disclosure, the camera calibration apparatus 300 further includes:

[0190] A third determining module 307 is configured to determine a third coordinate of the object target point in a third coordinate system according to the second coordinate of the object target point and the extrinsic parameter of the camera;

[0191] a fourth determining module 308 configured to determine, based on the third coordinates of the object target point and the intrinsic parameters of the camera, a fourth coordinate of a point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system;

[0192] A fifth determining module 309 is configured to determine an error function according to the first coordinates of the image target point corresponding to the object target point and the fourth coordinates;

[0193] The optimization module 310 is configured to optimize the camera parameters with the goal of minimizing the error function.

[0194] According to an embodiment of the present disclosure, the third coordinate system is a camera coordinate system of the camera.

[0195] Figure 3B The following is a block diagram of a camera calibration device according to an embodiment of the present disclosure, wherein the device can be implemented as part or all of an electronic device through software, hardware, or a combination of both.

[0196] like Figure 3B As shown, the camera calibration device 320 can be implemented in a server, for example, and in addition to the first acquisition module 301 and the first determination module 302, it also includes a receiving module 311 and a fifth acquisition module 312, wherein:

[0197] The receiving module 311 is configured to receive a camera calibration request, wherein the camera calibration request includes image identification information, and the image identification information is used to identify an image obtained by photographing a target object with a camera;

[0198] The fifth acquisition module 312 is configured to acquire an image of the target object photographed by a camera according to the image identification information.

[0199] According to an embodiment of the present disclosure, the camera calibration device 320 further includes:

[0200] A second acquisition module 303 is configured to acquire a texture map of the target object;

[0201] A third acquisition module 304 is configured to extract and match feature points on the image of the target object and the texture map to obtain the plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points;

[0202] The second determining module 305 is configured to determine a plurality of model target points on the target object model constructed based on the texture map corresponding to the plurality of texture map target points;

[0203] The fourth acquisition module 306 is configured to acquire second coordinates of the plurality of object target points according to the plurality of model target points.

[0204] According to an embodiment of the present disclosure, wherein:

[0205] The camera is a monocular camera; and / or

[0206] The first coordinate system is the image coordinate system of the camera, and the second coordinate system is the world coordinate system in the space where the target object is located or the two-dimensional coordinate system on the plane where the target object is located; and / or

[0207] The camera parameters include camera intrinsic parameters and extrinsic parameters.

[0208] According to an embodiment of the present disclosure, the determining of the camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points includes:

[0209] determining a projection matrix of the camera according to first coordinates of a plurality of the image target points and second coordinates of a corresponding plurality of the object target points;

[0210] The camera parameters of the camera are calculated according to the projection matrix.

[0211] According to an embodiment of the present disclosure, the step of calculating the camera parameters of the camera according to the projection matrix includes:

[0212] Perform QR decomposition on the 3*3 matrix consisting of the first three rows and first three columns of the projection matrix to obtain the camera intrinsic parameter matrix K and the rotation matrix R from the second coordinate system to the camera coordinate system under the camera perspective;

[0213] According to the projection matrix, the camera intrinsic parameter matrix K and the rotation matrix R, a translation vector t from the second coordinate system to the camera coordinate system under the camera perspective is determined.

[0214] According to an embodiment of the present disclosure, the camera calibration device 320 further includes:

[0215] A third determining module 307 is configured to determine a third coordinate of the object target point in a third coordinate system according to the second coordinate of the object target point and the extrinsic parameter of the camera;

[0216] a fourth determining module 308 configured to determine, based on the third coordinates of the object target point and the intrinsic parameters of the camera, a fourth coordinate of a point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system;

[0217] A fifth determining module 309 is configured to determine an error function according to the first coordinates of the image target point corresponding to the object target point and the fourth coordinates;

[0218] The optimization module 310 is configured to optimize the camera parameters with the goal of minimizing the error function.

[0219] According to an embodiment of the present disclosure, the third coordinate system is a camera coordinate system of the camera. The present disclosure also discloses an electronic device, Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0220] like Figure 4 As shown, the electronic device 400 includes a memory 401 and a processor 402, wherein the memory 401 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 402 to implement the method according to the embodiment of the present disclosure.

[0221] According to an embodiment of the present disclosure, a camera calibration method includes:

[0222] For each image target point in an image obtained by photographing a target object with a camera, obtaining a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, where the object target point is a point on the target object corresponding to the image target point;

[0223] The camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0224] According to an embodiment of the present disclosure, a camera calibration method includes:

[0225] receiving a camera calibration request, the camera calibration request including image identification information, the image identification information being used to identify an image obtained by photographing a target object with a camera;

[0226] Acquire an image of the target object photographed by a camera according to the image identification information;

[0227] Acquire each image target point in the image, obtain a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, wherein the object target point is a point on the target object corresponding to the image target point;

[0228] The camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

[0229] According to an embodiment of the present disclosure, the camera calibration method further includes:

[0230] Obtaining a texture map of the target object;

[0231] Extracting and matching feature points on the image of the target object and the texture map to obtain the plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points;

[0232] Determining a plurality of model target points on a target object model constructed based on the texture map corresponding to the plurality of texture map target points;

[0233] The second coordinates of the plurality of object target points are acquired according to the plurality of model target points.

[0234] According to an embodiment of the present disclosure, wherein:

[0235] The camera is a monocular camera; and / or

[0236] The first coordinate system is the image coordinate system of the camera, and the second coordinate system is the world coordinate system in the space where the target object is located or the two-dimensional coordinate system on the plane where the target object is located; and / or

[0237] The camera parameters include camera intrinsic parameters and extrinsic parameters.

[0238] According to an embodiment of the present disclosure, the determining of the camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points includes:

[0239] determining a projection matrix of the camera according to first coordinates of a plurality of the image target points and second coordinates of a corresponding plurality of the object target points;

[0240] The camera parameters of the camera are calculated according to the projection matrix.

[0241] According to an embodiment of the present disclosure, the step of calculating the camera parameters of the camera according to the projection matrix includes:

[0242] Perform QR decomposition on the 3*3 matrix consisting of the first three rows and first three columns of the projection matrix to obtain the camera intrinsic parameter matrix K and the rotation matrix R from the second coordinate system to the camera coordinate system under the camera perspective;

[0243] According to the projection matrix, the camera intrinsic parameter matrix K and the rotation matrix R, a translation vector t from the second coordinate system to the camera coordinate system under the camera perspective is determined.

[0244] According to an embodiment of the present disclosure, the camera calibration method further includes:

[0245] Determine a third coordinate of the object target point in a third coordinate system according to the second coordinate of the object target point and the extrinsic parameter of the camera;

[0246] Determining, according to the third coordinate of the object target point and the intrinsic parameter of the camera, a fourth coordinate of a point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system;

[0247] determining an error function according to the first coordinate and the fourth coordinate of the image target point corresponding to the object target point;

[0248] The camera parameters are optimized with the goal of minimizing the error function.

[0249] According to an embodiment of the present disclosure, the third coordinate system is a camera coordinate system of the camera.

[0250] Figure 5 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0251] like Figure 5 As shown, the computer system 500 includes a processing unit 501, which can execute various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the system 500 are also stored in the RAM 503. The processing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0252] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom can be installed into the storage section 508 as needed. Among them, the processing unit 501 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0253] In particular, according to embodiments of the present disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a computer-readable storage medium, the computer program comprising program code for executing the methods described above. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion 509 and / or installed from a removable medium 511.

[0254] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0255] The units or modules involved in the embodiments described in this disclosure may be implemented by software or programmable hardware. The units or modules described may also be provided in a processor, and the names of these units or modules do not, in certain circumstances, constitute limitations on the units or modules themselves.

[0256] As another aspect, the present disclosure further provides a computer-readable storage medium. This computer-readable storage medium may be included in the electronic device or computer system described in the above embodiments, or may be a standalone computer-readable storage medium not incorporated into the device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the methods described in the present disclosure.

[0257] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

Claims

1. A camera calibration method, comprising: For each image target point in an image obtained by photographing a target object with a camera, obtaining a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, where the object target point is a point on the target object corresponding to the image target point; Wherein, obtaining the second coordinate includes: Obtaining a texture map of the target object; Extracting and matching feature points on the image of the target object and the texture map to obtain a plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points; Determining a plurality of model target points corresponding to the plurality of texture map target points on a target object model constructed based on the texture map, wherein the target object model is used to provide three-dimensional feature points with real geographic information; acquiring second coordinates of a plurality of object target points according to the plurality of model target points; The camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

2. The method according to claim 1, wherein: The camera is a monocular camera; and / or The first coordinate system is the image coordinate system of the camera, and the second coordinate system is the world coordinate system in the space where the target object is located or the two-dimensional coordinate system on the plane where the target object is located; and / or The camera parameters include camera intrinsic parameters and extrinsic parameters.

3. The method according to claim 1, wherein The determining of the camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points comprises: determining a projection matrix of the camera according to first coordinates of a plurality of the image target points and second coordinates of a corresponding plurality of the object target points; The camera parameters of the camera are calculated according to the projection matrix.

4. The method according to claim 3, wherein: The step of calculating the camera parameters of the camera according to the projection matrix includes: Perform QR decomposition on the 3*3 matrix consisting of the first three rows and first three columns of the projection matrix to obtain the camera intrinsic parameter matrix K and the rotation matrix R from the second coordinate system to the camera coordinate system under the camera perspective; According to the projection matrix, the camera intrinsic parameter matrix K and the rotation matrix R, a translation vector t from the second coordinate system to the camera coordinate system under the camera perspective is determined.

5. The method according to claim 1, further comprising: Determine a third coordinate of the object target point in a third coordinate system according to the second coordinate of the object target point and the extrinsic parameter of the camera; Determining, according to the third coordinate of the object target point and the intrinsic parameter of the camera, a fourth coordinate of a point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system; determining an error function according to the first coordinate and the fourth coordinate of the image target point corresponding to the object target point; The camera parameters are optimized with the goal of minimizing the error function, where the camera parameters include at least one or more of the following: a rotation matrix R, a translation vector t, an intrinsic parameter matrix K, and a distortion parameter.

6. The method according to claim 5, wherein: The third coordinate system is the camera coordinate system of the camera.

7. A camera calibration method, comprising: receiving a camera calibration request, the camera calibration request including image identification information, the image identification information being used to identify an image obtained by photographing a target object with a camera; Acquire an image of the target object photographed by a camera according to the image identification information; Acquire each image target point in the image, obtain a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, wherein the object target point is a point on the target object corresponding to the image target point; Wherein, obtaining the second coordinate includes: Obtaining a texture map of the target object; Extracting and matching feature points on the image of the target object and the texture map to obtain a plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points; Determining a plurality of model target points corresponding to the plurality of texture map target points on a target object model constructed based on the texture map, wherein the target object model is used to provide three-dimensional feature points with real geographic information; acquiring second coordinates of a plurality of object target points according to the plurality of model target points; The camera parameters of the camera are determined according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

8. A camera calibration device, comprising: a first acquisition module configured to acquire, for each image target point in an image obtained by photographing a target object with a camera, a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, wherein the object target point is a point on the target object corresponding to the image target point; A second acquisition module is configured to acquire a texture map of the target object; a third acquisition module configured to extract and match feature points on the image of the target object and the texture map to obtain a plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points; a second determining module configured to determine a plurality of model target points corresponding to the plurality of texture map target points on a target object model constructed based on the texture map, the target object model being used to provide three-dimensional feature points having real geographic information; a fourth acquisition module, configured to acquire second coordinates of a plurality of object target points according to the plurality of model target points; The first determining module is configured to determine camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

9. The apparatus according to claim 8, further comprising: a third determining module, configured to determine a third coordinate of the object target point in a third coordinate system according to the second coordinate of the object target point and an extrinsic parameter of the camera; a fourth determining module configured to determine, based on the third coordinate of the object target point and the intrinsic parameter of the camera, a fourth coordinate of a point obtained by projecting the object target point onto the imaging plane of the camera in the first coordinate system; a fifth determining module configured to determine an error function according to the first coordinate of the image target point corresponding to the object target point and the fourth coordinate; The optimization module is configured to optimize the camera parameters with the goal of minimizing the error function.

10. A camera calibration device, comprising: a receiving module configured to receive a camera calibration request, wherein the camera calibration request includes image identification information, and the image identification information is used to identify an image obtained by photographing a target object with a camera; A fifth acquisition module is configured to acquire an image of the target object photographed by the camera according to the image identification information; a first acquisition module configured to acquire each image target point in the image, acquire a first coordinate of the image target point in a first coordinate system and a second coordinate of a corresponding object target point in a second coordinate system, wherein the object target point is a point on the target object corresponding to the image target point; A second acquisition module is configured to acquire a texture map of the target object; a third acquisition module configured to extract and match feature points on the image of the target object and the texture map to obtain a plurality of image target points and a plurality of texture map target points on the texture map corresponding to the plurality of image target points; a second determining module configured to determine a plurality of model target points corresponding to the plurality of texture map target points on a target object model constructed based on the texture map, the target object model being used to provide three-dimensional feature points having real geographic information; a fourth acquisition module, configured to acquire second coordinates of a plurality of object target points according to the plurality of model target points; The first determining module is configured to determine camera parameters of the camera according to the first coordinates of the plurality of image target points and the corresponding second coordinates of the plurality of object target points.

11. An electronic device comprising a memory and a processor; wherein: The memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps according to any one of claims 1 to 7.

12. A readable storage medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the method steps according to any one of claims 1 to 7 are implemented.

Images