A camera external parameter determination method, device and external parameter calibration system

Abstract

The embodiment of the application provides a camera external parameter determination method and device and an external parameter calibration system. The real size of a reference object, the imaging size image of the reference object and the internal parameter of a camera to be calibrated are acquired, wherein the surface of the reference object facing the camera to be calibrated at each spatial position is a spherical surface with the same curvature radius. When measuring the real size and the imaging size of the reference object, only the radius, i.e. the imaging radius and the real radius, is involved. When the real size and the imaging size need to be corresponded in the subsequent process, only the radii need to be corresponded. According to the Gaussian imaging theorem, under the condition that the internal parameter of the camera to be calibrated is known, the distance between the target position and each spatial position can be determined according to the reference object image. According to the three-point positioning rule, the target position can be determined. The deflection angle of the camera to be calibrated can be determined according to the target position, the spatial position of the reference object and each imaging size, thereby improving the calibration efficiency of the camera.

Description

Technical Field

[0001] This application relates to the field of computer software technology, and in particular to a method, apparatus and calibration system for determining camera extrinsic parameters. Background Technology

[0002] In certain application scenarios, it is necessary to obtain spatial parameters such as the camera's latitude, longitude, altitude, and tilt angle to further determine the camera's shooting range. Therefore, accurate calibration of the camera's spatial parameters has a crucial impact on the application's effectiveness.

[0003] Currently, camera calibration still relies on manual data collection. However, for a large number of cameras that need to be calibrated, data collection is too inefficient. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, and extrinsic parameter calibration system for determining camera extrinsic parameters, thereby addressing the problem of low efficiency in camera calibration. The specific technical solution is as follows:

[0005] In a first aspect of this application, a method for determining camera extrinsic parameters is provided, the method comprising:

[0006] The true size of the reference object, the spatial position of the reference object at multiple moments, and the reference object image obtained by the camera to be calibrated at each of the aforementioned moments are obtained; wherein, the surface of the reference object facing the camera to be calibrated when it is at each of the aforementioned spatial positions is a sphere with the same radius of curvature.

[0007] The target position of the camera to be calibrated is determined based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference object in each of the reference object images.

[0008] The deflection angle of the camera to be calibrated is determined based on the spatial location, the target location, and the imaging size.

[0009] In one possible implementation, determining the deflection angle of the camera to be calibrated based on the spatial locations, the target locations, and the imaging sizes includes:

[0010] Based on each of the spatial positions, the target positions, and the imaging sizes, a relative angle is determined, wherein the relative angle is the angle between the position of each spatial position relative to the orientation of the camera to be calibrated and the orientation of the camera to be calibrated;

[0011] Based on the spatial positions and the target positions, calculate the angle between the orientation of each spatial position relative to the camera to be calibrated and the vertical direction, and use it as the azimuth angle;

[0012] The deflection angle of the camera to be calibrated is determined based on the relative angle and the azimuth angle.

[0013] In one possible implementation, determining the deflection angle of the camera to be calibrated based on the spatial locations, the target locations, and the imaging sizes further includes:

[0014] For the same focal length, determine the maximum image size from the image sizes obtained from the same spatial location;

[0015] The spatial position of the reference object when the image of the reference object with the maximum imaging size is captured is determined as the frontal spatial position;

[0016] The deflection angle of the camera to be calibrated is determined based on the frontal spatial position and the target position.

[0017] In one possible implementation, determining the target position of the camera to be calibrated based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference objects in each of the reference object images includes:

[0018] Based on the focal length of the camera to be calibrated, the actual size, and the imaging size in the reference images obtained from at least three spatial locations, the distances between the at least three spatial locations and the camera to be calibrated are determined as the object distances corresponding to each spatial location.

[0019] The position at which the distance from each of the at least three spatial positions is determined as the object distance corresponding to that spatial position is used as the target position of the camera to be calibrated.

[0020] In one possible implementation, determining the target position of the camera to be calibrated based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference objects in each of the reference object images includes:

[0021] Establish a theoretical correspondence between the spatial location, the distance between the reference object and the camera to be calibrated, and the target location;

[0022] Based on the theoretical correspondence, the target position of the camera to be calibrated is determined.

[0023] In a second aspect of this application, a camera extrinsic parameter determination device is provided, the device comprising:

[0024] The acquisition module is used to acquire the true size of the reference object, the spatial position of the reference object at multiple times, and the reference object image obtained by the camera to be calibrated at each of the said times; wherein, the surface of the reference object facing the camera to be calibrated when it is at each of the said spatial positions is a sphere with the same radius of curvature.

[0025] The target location determination module is used to determine the target location of the camera to be calibrated based on the actual size, each of the spatial locations, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference object in each of the reference object images.

[0026] The deflection angle determination module is used to determine the deflection angle of the camera to be calibrated based on the spatial position, the target position, and the imaging size.

[0027] In one possible implementation, the deflection angle determination module is specifically used to determine a relative angle based on each of the spatial positions, the target position, and each of the imaging sizes, wherein the relative angle is the angle between the orientation of each of the spatial positions relative to the camera to be calibrated and the orientation of the camera to be calibrated.

[0028] Based on each of the spatial positions and the target position, calculate the angle between the orientation of each spatial position relative to the camera to be calibrated and the vertical direction, and use it as the azimuth angle;

[0029] The deflection angle of the camera to be calibrated is determined based on the relative angle and the azimuth angle.

[0030] The deflection angle determination module is specifically used to determine the maximum imaging size from the imaging sizes obtained from the same spatial position for the same focal length.

[0031] The spatial position of the reference object when the image of the reference object with the maximum imaging size is captured is determined as the frontal spatial position;

[0032] Based on the frontal spatial position and the target position, determine the deflection angle of the camera to be calibrated;

[0033] The target position determination module is specifically used to determine the distances between at least three spatial positions and the camera to be calibrated, based on the focal length of the camera to be calibrated, the actual size, and the imaging size in reference images obtained from at least three spatial positions, as the object distances corresponding to each spatial position.

[0034] The distance from each of the at least three spatial locations is determined as the object distance corresponding to that spatial location, and this position is used as the target position of the camera to be calibrated. Specifically, the target position determination module is used to establish a theoretical correspondence between the spatial positions, the distance between the reference object and the camera to be calibrated, and the target position.

[0035] Based on the theoretical correspondence, the target position of the camera to be calibrated is determined.

[0036] In a third aspect of this application, an electronic device is provided, characterized in that it comprises:

[0037] Memory, used to store computer programs;

[0038] When a processor executes a program stored in memory, it implements the method described in the first aspect above.

[0039] In a fourth aspect of this application, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the system described in any of the first aspects above.

[0040] In a fifth aspect of this application, a camera extrinsic parameter calibration system is provided, characterized in that the system comprises:

[0041] Vehicle equipment, reference objects, information acquisition equipment, and electronic equipment as described in the third aspect;

[0042] The vehicle device is used to carry the reference object and move within the field of view of the camera to be calibrated;

[0043] The surface of the reference object facing the camera to be calibrated at each spatial position is a sphere with the same radius of curvature.

[0044] The information acquisition device is used to acquire the spatial position of the reference object at multiple different times, and send each spatial position and each time to the electronic device accordingly;

[0045] The electronic device is configured to receive the actual size of the reference object, each of the spatial positions, and reference object images obtained by the camera to be calibrated capturing the reference object at each of the stated times, so as to implement the method described in the first aspect.

[0046] Beneficial effects of the embodiments in this application:

[0047] This application provides a method, apparatus, and calibration system for determining camera extrinsic parameters. This system acquires the true size of a reference object, its image size, and the intrinsic parameters of the camera to be calibrated. The reference object, at each spatial position, faces the camera as a sphere with the same radius of curvature. Measuring the true and image sizes only involves the radii—the image radius and the true radius. When subsequently correlating the true and image sizes, only the radii need to be correlated. Therefore, a correspondence between the true and image sizes can be established even when the target position of the camera to be calibrated is unknown. According to Gaussian imaging theorem, when the intrinsic parameters of the camera to be calibrated are known, the distance from the target position to each spatial position can be determined using the captured image of the reference object. The target position can be determined using the three-point positioning rule, and the deflection angle of the camera to be calibrated can be determined using the target position, the spatial position of the reference object, and the image sizes. In this application, only the actual size and spatial position of the reference object, the intrinsic parameters of the camera to be calibrated, and the image of the reference object captured by the camera to be calibrated are required. Without the need to manually obtain any extrinsic parameters of the camera to be calibrated, the extrinsic parameters of the camera to be calibrated can be calculated, thereby realizing the calibration of the camera to be calibrated and improving the efficiency of camera calibration.

[0048] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0050] Figure 1a A schematic diagram of a camera extrinsic parameter calibration system provided in an embodiment of this application;

[0051] Figure 1b A schematic diagram of the shooting angle of a camera to be calibrated, provided for an embodiment of this application;

[0052] Figure 1c A schematic diagram of a reference object provided for an embodiment of this application;

[0053] Figure 1d A schematic diagram of a camera imaging principle provided in an embodiment of this application;

[0054] Figure 2a This is a schematic flowchart of a method for determining camera extrinsic parameters provided in an embodiment of this application;

[0055] Figure 2b This is a schematic diagram illustrating the principle of camera imaging distance provided in an embodiment of this application;

[0056] Figure 2c This is a schematic diagram illustrating the principle of a camera extrinsic parameter determination method provided in an embodiment of this application.

[0057] Figure 3a A schematic diagram illustrating the relationship between the relative included angle and the deflection angle of a camera, provided in an embodiment of this application;

[0058] Figure 3b This is a schematic flowchart of another method for determining camera extrinsic parameters provided in an embodiment of this application;

[0059] Figure 3c A schematic diagram of the horizontal and vertical tilt angles of a camera provided in an embodiment of this application;

[0060] Figure 4 This is a schematic diagram of a camera extrinsic parameter determination device provided in an embodiment of this application;

[0061] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0063] To better illustrate the camera extrinsic parameter determination method provided in this application, the camera extrinsic parameter calibration system will be described first, such as... Figure 1a As shown, the system includes: a vehicle device 101, a reference object 102, an information acquisition device 103, and an electronic device 104. Among them,

[0064] The vehicle device 101 is used to carry the reference object and move within the field of view of the camera to be calibrated.

[0065] The vehicle can be any device that can move within the field of view of the camera to be calibrated, such as a vehicle. Furthermore, the position of the reference object mounted on the vehicle remains fixed during the determination of the camera's extrinsic parameters.

[0066] Reference object 102, in this embodiment of the application, can be a sphere whose surface facing the camera to be calibrated has the same radius of curvature at every spatial position. It can be understood that if any surface of the reference object is a sphere with the same radius of curvature, that is, the reference object is a sphere, then obviously the surface of the reference object facing the camera to be calibrated at every spatial position is a sphere with the same radius of curvature. Therefore, in this application, the reference object can be a sphere.

[0067] In practice, due to limitations in the shooting angle, a portion of the outer surface of the reference object may never face the camera being calibrated; therefore, this portion does not necessarily have to be spherical. For a clearer explanation, see [link to documentation]. Figure 1b As shown, with the reference object as the coordinate system, the moving reference object captured by the camera to be calibrated is equivalent to the stationary reference object captured by the camera from different positions, forming a coordinate system as follows: Figure 1b The schematic diagram shown is composed of Figure 1b It is evident that a portion of the reference object in the captured image will always be uncaptured. Figure 1b The shaded area enclosed by the box in the middle, even if the surface of the reference object in the shaded area is not a sphere, the surface of the reference object facing the camera to be calibrated at each spatial position will still be a sphere with the same radius of curvature.

[0068] The shape of the shaded area is not limited in this embodiment. For example, as shown... Figure 1b As shown, the shaded area of ​​the reference object can be a sphere, or it can be like... Figure 1c As shown, the shaded area of ​​the reference object is irregular. The specific shaded area can be chosen based on the actual application scenario; this embodiment does not impose any limitations.

[0069] Since the reference object's surface facing the camera to be calibrated is a sphere with the same radius of curvature at all spatial locations, only the radius is involved when determining the reference object's true size and the image size—that is, the true radius and the image radius. When subsequently correlating the true size and the image size, only the radii need to be correlated; there's no need to consider dimensions such as length, width, and height, which change with the shooting angle. Furthermore, when correlating the true size and the image size, the orientation of the reference object relative to the camera to be calibrated doesn't need to be considered. Therefore, a correspondence between the true size and the image size can be established even when the target position of the camera to be calibrated is unknown. According to Gaussian imaging theorem, when the intrinsic parameters of the camera to be calibrated are known, the scaling ratio between the true size and the corresponding image size depends on the object distance, i.e., the distance between the reference object and the camera to be calibrated. This distance can be approximated as the distance between the spatial position of the reference object and the target position. Therefore, by using the captured images of the reference object, the distances from the target position to various spatial locations can be determined. As can be seen from the three-point positioning method, if the distances from the target location to any three different spatial locations are known, the target location can be determined. Therefore, the camera extrinsic parameter calibration system provided in this application can calibrate the target location of the camera to be calibrated even when the extrinsic parameters of the camera to be calibrated are unknown.

[0070] Information acquisition device 103 is used to acquire the spatial position of a reference object at multiple different times and send each spatial position and time to an electronic device.

[0071] In this embodiment, the spatial position of the reference object is its latitude and longitude. When the information acquisition device acquires the spatial position of the reference object, it can establish a communication connection with the reference object to receive and transmit the spatial position of the reference object in real time, or it can install the information acquisition device on the reference object to acquire the spatial position of the reference object in real time. This embodiment does not limit the specific method by which the information acquisition device acquires the spatial position of the reference object.

[0072] Since the target position of the camera to be calibrated needs to be determined by the actual size of the reference object, the spatial position of the reference object, the internal parameters of the camera to be calibrated, and the images of the reference object taken by the camera to be calibrated at various times, it is necessary to match the spatial position collected by the information acquisition device with the images of the reference object taken by the camera to be calibrated. Based on this, the information acquisition device also needs to send the acquisition time of the spatial position of the reference object to the electronic device accordingly.

[0073] For example, suppose that spatial position information 1 (X1, Y1) of reference object A is acquired at time t1. Then, this spatial position information 1 and time t1 are sent to the electronic device.

[0074] Electronic device 104 is used to determine the target position of the camera to be calibrated based on the actual size of the reference object, its spatial position, the intrinsic parameters of the camera to be calibrated, and the reference object images obtained by the camera to be calibrated from the reference object at various times; and to determine the deflection angle of the camera to be calibrated based on its spatial position, target position, and imaging size.

[0075] In this embodiment, the intrinsic parameters of the camera to be calibrated are parameters that can be used to calculate the size relationship between the actual size and the image size of the reference object, such as the focal length and lens magnification of the camera to be calibrated. Figure 1d The imaging principle of the camera to be calibrated, as shown, includes the camera 110 to be calibrated and a reference object 102. The farther the reference object is from the camera to be calibrated, the smaller the image size. There is a size relationship between the actual size of the reference object and the image size. The extrinsic parameters of the camera to be calibrated are parameters that can be used to represent the position, height, and shooting angle of the camera, such as the latitude, longitude, and height of the camera.

[0076] The intrinsic parameters of the camera to be calibrated and the reference images obtained by the camera from the reference objects at various spatial locations can be transmitted in real time by the camera to be calibrated, or they can be acquired and transmitted to the camera to be calibrated through an intermediate device. This application embodiment does not impose any restrictions on how to obtain the intrinsic parameters of the camera to be calibrated and the reference images obtained by the camera from the reference objects at various spatial locations; any method that can acquire data can be applied to this application embodiment. The method for determining the target position and deflection angle of the camera to be calibrated will be described in detail later, and will not be elaborated upon here.

[0077] In one possible implementation, since it is necessary to subsequently map the spatial position of the reference object to the data of the reference object image one-to-one, that is, to ensure that the acquisition of the spatial position of the reference object and the acquisition of the reference object image taken by the camera to be calibrated are at the same time, the information acquisition device and the camera to be calibrated can be synchronized with clock signals in advance. In this way, the spatial position of the reference object and the data of the reference object image can be directly mapped to the time, thereby improving the efficiency of calculating the external parameters of the camera to be calibrated.

[0078] For example, assuming that the information acquisition device for obtaining the spatial position of reference object A and the camera to be calibrated a have been clock synchronized in advance, the spatial position 1 of the reference object is obtained at time t1, and the image of reference object A taken by the camera to be calibrated a at time t1 is the reference object image 1. Then, when calculating the target position of the camera to be calibrated, it can be calculated directly using the spatial position 1 obtained at time t1 and the reference object image 1.

[0079] In one possible implementation, to further improve the computational efficiency of calculating the extrinsic parameters of the camera to be calibrated, the information acquisition device can also send each spatial location and each time point to the electronic device in the form of tuples, wherein each tuple includes: spatial location, and the time at which the spatial location was acquired.

[0080] For example, assuming the spatial position 1 (X1, Y1) of reference object A is obtained at time t1, then time t1 and spatial position 1 can be sent to the electronic device as a tuple. That is, it can be sent to the electronic device in the form of (X1, Y1, t1).

[0081] The following will describe a method for determining camera extrinsic parameters provided in the application embodiments, such as... Figure 2a As shown, the method includes:

[0082] S201. Obtain the true size of the reference object, the spatial position of the reference object at multiple moments, and the reference object image obtained by the camera to be calibrated at each moment. The surface of the reference object facing the camera to be calibrated at each spatial position is a sphere with the same radius of curvature.

[0083] S202. Determine the target position of the camera to be calibrated based on the actual size, spatial position, intrinsic parameters of the camera to be calibrated, and the imaging size of the reference objects in the images of the reference objects.

[0084] S203. Determine the deflection angle of the camera to be calibrated based on the spatial position, target position, and imaging size.

[0085] In this embodiment, by acquiring the true size of the reference object, the image size of the reference object, and the intrinsic parameters of the camera to be calibrated, where the surface of the reference object facing the camera at each spatial position is a sphere with the same radius of curvature, measuring the true size and image size of the reference object only involves the radius, i.e., the image radius and the true radius. When it is necessary to correlate the true size and the image size later, only the radii need to be correlated. Therefore, a correspondence between the true size and the image size can be established even when the target position of the camera to be calibrated is unknown. According to the Gaussian imaging theorem, when the intrinsic parameters of the camera to be calibrated are known, the distance from the target position to each spatial position can be determined by the captured image of the reference object. The target position can be determined by combining the three-point positioning rule, and the deflection angle of the camera to be calibrated can be determined by the target position, the spatial position of the reference object, and each image size. In this application, only the actual size and spatial position of the reference object, the intrinsic parameters of the camera to be calibrated, and the image of the reference object captured by the camera to be calibrated are required. The target position and deflection angle of the camera to be calibrated are determined by collecting the actual size of the reference object, the image data of the reference object captured by the camera to be calibrated, and the parameter information of the camera to be calibrated through an information acquisition device. In this embodiment, the correspondence between the position and size of the reference object and the camera to be calibrated is determined by the reference object and its image. Then, based on this correspondence, the target position and deflection angle of the camera to be calibrated are determined. The extrinsic parameters of the camera to be calibrated can be calculated without manually acquiring any extrinsic parameters, thus achieving the calibration of the camera to be calibrated and improving the efficiency of camera calibration.

[0086] The following is a detailed description of S201-S203:

[0087] In S201, the method of obtaining the spatial position of the reference object at multiple times and the reference object image obtained by the camera to be calibrated at each time is as described in the previous description of the camera extrinsic calibration system, and will not be repeated here.

[0088] The actual size of the reference object can be stored in advance by technicians in the information acquisition device, or it can be measured in real time by technicians during the calculation process and then uploaded to the information acquisition device. In this embodiment of the application, there are no restrictions on how to obtain the actual size of the reference object.

[0089] The shape of the reference object can be used as a reference. Figure 1b As shown in 1c, it can be a sphere or an irregular shape, as long as the surface facing the camera to be calibrated is a sphere with the same radius of curvature at each spatial position.

[0090] In one possible implementation, S202 is specifically as follows:

[0091] S2021. Based on the focal length, actual size, and imaging size in reference images taken from at least three spatial locations, determine the distances between at least three spatial locations and the camera to be calibrated, and use these distances as the object distances corresponding to each spatial location.

[0092] like Figure 2b As shown, the image size of the reference object in the camera to be calibrated is similar to the actual size of the reference object, and satisfies the formula:

[0093]

[0094] in,

[0095] Right now,

[0096] Where R is the true radius of the reference object, r is the imaging radius of the reference object, L is the actual distance from the reference object to the camera to be calibrated, i.e., the object distance, l is the distance from the lens of the camera to be calibrated to the image calculated based on the focal length of the camera to be calibrated, and f is the focal length of the camera to be calibrated.

[0097] In this embodiment, since the actual size and image size of the reference object satisfy the above formula, and for the same camera to be calibrated, after obtaining the intrinsic parameters of the camera to be calibrated, the actual distance L from the reference object to the camera to be calibrated can be calculated using the above formula. When the focal length of the camera to be calibrated is fixed, f in the above formula is a fixed value; when the focal length of the camera to be calibrated is zoom, f in the above formula is the specific focal length value when the image of the reference object is captured at that spatial location.

[0098] For example, assuming the image size of reference object A taken at spatial position 1 is r1, since the actual size of the same reference object is the same, assuming the actual size of reference object A is R, and the focal length of the camera to be calibrated at this time is f1, then we can obtain:

[0099]

[0100] Calculated

[0101] Assuming the image size of the reference object A taken at spatial position 2 is r2, and the focal length of the camera to be calibrated at this time is f2, then we can obtain:

[0102]

[0103] Calculated

[0104] Assuming the image size of the reference object A taken at spatial position 3 is r3, and the focal length of the camera to be calibrated at this time is f3, then we can obtain:

[0105]

[0106] Calculated Depending on the selected spatial location, the calculated L1, L2, and L3 may be three completely different distances, or they may be three completely identical or partially identical distances. For example, if the selected spatial locations 1-3 are exactly located on a circle centered on the camera to be calibrated, then the calculated L1, L2, and L3 will be completely identical.

[0107] S2022. The position that simultaneously satisfies at least three distances that are the same as the object distance is taken as the target position of the camera to be calibrated.

[0108] In this embodiment, according to the three-point positioning method, if the distances from the target location to any three different spatial locations are known, the target location can be determined. Therefore, the image size of the same reference object captured at least three random spatial locations can be selected. Then, based on the image size, at least three object distances can be determined. Finally, based on these at least three object distances, the location whose distance from each of the at least three spatial locations is equal to the object distance corresponding to that spatial location is the target location of the camera to be calibrated.

[0109] The following example, using three imaging sizes obtained from three spatial positions of reference object A, illustrates how to determine the target position of the camera to be calibrated.

[0110] First, assuming the image size of reference object A taken at spatial position 1 is r1, the calculated object distance is L1. Assuming the image size of reference object A taken at spatial position 2 is r2, the calculated object distance is L2. Assuming the image size of reference object A taken at spatial position 3 is r3, the calculated object distance is L3. The latitude and longitude of reference object A at spatial position 1 are (X1, Y1), at spatial position 2 are (X2, Y2), and at spatial position 3 are (X3, Y3). The relationship between the horizontal distances d1, d2, and d3 between the camera to be calibrated and reference object A and the latitude and longitude (x, y) of the camera to be calibrated satisfies the following formula:

[0111]

[0112] Among them, R 地 The radius is the Earth's radius.

[0113] See Figure 2cIt can be seen that the reference object lies on the XOY plane, the distance between the reference object and the camera to be calibrated (i.e., the object distance) is L, the height of the camera to be calibrated is H, and the horizontal distance between the camera to be calibrated and the reference object is d. Furthermore, L, H, and d satisfy the Pythagorean theorem for a right triangle. That is, d... 2 +H 2 =L 2

[0114] Based on the distances L1, L2, and L3 from the reference object A to the camera to be calibrated, which can be obtained from the reference object A at three different spatial positions, and using the Pythagorean theorem, we can obtain:

[0115]

[0116] By combining equations (1) and (2) above, the latitude and longitude (x, y) and altitude H of the camera to be calibrated can be calculated.

[0117] Since L1, L2, and L3 can be calculated through step S2021, and H is the same in the three sets of data, the latitude and longitude (x, y) and altitude H of the camera to be calibrated can be calculated. The specific calculation process can be performed by substituting the actual acquired data into the above formulas (1) and (2):

[0118] Combining (1) and (2), we can obtain

[0119]

[0120] The following describes the calculation process using L1 as an example. L1 can be calculated through the above steps S2021, and the equation...

[0121] {R 地 *arcos[cos(y)*cos(Y1)*cos(x-X1)+sin(y)*sin(Y1)]} 2 +H 2 =L1 2 , where R 地 Let (x, y) be the Earth's radius, which can be obtained using existing technology. (X1, Y1) represents the latitude and longitude of the reference point, which are known quantities. Therefore, cos(Y1) and sin(Y1) can be calculated to obtain specific values, and are also known quantities. Then, cos(y), cos(x-X1), sin(y), and H... 2 Let R be an unknown quantity. Then the equation {R} is... 地 *arcos[cos(y)*cos(Y1)*cos(x-X1)+sin(y)*sin(Y1)]} 2 +H 2 =L1 2The equations L2, L3, and L1 are the same, and will not be elaborated further here. Therefore, the above system of equations consists of three unknowns, and the three equations are non-linearly related, thus the system of equations has a unique solution.

[0122] The methods for solving the above system of equations can vary depending on the application scenario. For example, the specific values ​​of x, y, and H can be obtained through numerical calculations. Alternatively, linear expressions for x, y, and H can be derived through linear relationships, or mathematical modeling can be used to establish a solution. This application does not impose any limitations on the specific methods for solving the above system of equations.

[0123] In one possible implementation, the theoretical correspondence between the various parameters can be calculated in advance using formulas. Then, based on the actual measured values, the latitude and longitude (x, y) and altitude H of the camera to be calibrated can be directly calculated using these theoretical correspondences. Specifically, this includes:

[0124] S2021a. Establish the theoretical correspondence between spatial position, the distance between the reference object and the camera to be calibrated, and the target position.

[0125] S2022a. Determine the target position of the camera to be calibrated based on the theoretical correspondence.

[0126] As can be seen from the above S2021-S2022,

[0127] d = R 地 *arcos[cos(y)*cos(Y)*cos(xX)+sin(y)*sin(Y)]

[0128] Among them, R 地 Let (x, y) be the Earth's radius, (x, y) be the latitude and longitude of the reference object, and (x, y) be the latitude and longitude of the camera to be calibrated.

[0129] According to the Gaussian imaging theorem:

[0130] Where R is the true radius of the reference object, r is the imaging radius of the reference object, L is the actual distance from the reference object to the camera to be calibrated, and f is the focal length of the camera to be calibrated.

[0131] Using the Pythagorean theorem, we have: d 2 +H 2 =L 2

[0132] Where L is the distance between the reference object and the camera to be calibrated, H is the height of the camera to be calibrated, and d is the horizontal distance between the camera to be calibrated and the reference object.

[0133] The above formula can be used to obtain...

[0134]

[0135] By substituting the image size of the reference object obtained from at least three spatial locations, the actual size of the reference object, and the focal length of the camera to be calibrated into the above formula, the latitude and longitude (x, y) and height H of the camera to be calibrated can be calculated.

[0136] In one possible implementation, S203 is specifically as follows:

[0137] S2031. Determine the relative angle based on each spatial position, target position, and imaging size, where the relative angle is the angle between the orientation of each spatial position relative to the camera to be calibrated and the orientation of the camera to be calibrated.

[0138] Understandably, based on the transformation relationship between image coordinates and spatial coordinates between the spatial position and the target position, the reference object can be transformed into the camera coordinate system, thus obtaining the image size based on the true size. According to the Gaussian imaging theorem, this transformation relationship depends on the focal length of the camera to be calibrated, the target position, the spatial position of the reference object, and the relative angle between the spatial position and the orientation of the camera to be calibrated.

[0139] When the focal length, target position, and spatial position of the camera to be calibrated are known, the image size can be considered as a variable that changes with the relative angle, and conversely, the relative angle can also be considered as a variable that changes with the image size. Therefore, the relative angle can be deduced from the actual image size. Any method that can calculate the relative angle in the embodiments of this application can be applied to this application, and this application does not limit the specific method for calculating the relative angle.

[0140] S2032. Based on each spatial position and target position, calculate the angle between the orientation of each spatial position relative to the calibrated camera and the known direction, and use it as the azimuth angle.

[0141] In this step, such as Figure 3a As shown, for ease of description, the target position of the camera to be calibrated is marked as point A, and the spatial position of the reference object is marked as point B. A spatial rectangular coordinate system is established with the vertical point of the camera to be calibrated as the origin 0. Here, AB is the actual shooting direction of the camera to be calibrated, that is, the shooting direction when shooting the reference object A, and AC is the positive shooting angle of the camera to be calibrated, that is, the shooting direction on the camera axis.

[0142] From the actual shooting direction of the camera to be calibrated, the origin O, point A, and point B can form a triangle, where line segment OB lies on the XOY plane and line segment AO lies on the YOZ plane. Therefore, ∠AOB is a right angle, and triangle AOB is a right triangle. The distance of line segment AO is the height H of the camera to be calibrated, and the distance of line segment AB is the object distance from the camera to the reference object. ∠β is the azimuth angle of the camera to be calibrated calculated based on the acquired data. The actual distances of line segments AO and AB can be calculated through the aforementioned steps.

[0143] S2033. Determine the deflection angle of the camera to be calibrated based on the relative angle and azimuth angle.

[0144] In this step, the spatial positions of the same reference object at at least two locations and the target position of the camera to be calibrated corresponding to those spatial positions can be randomly selected to determine the deflection angle of the camera to be calibrated. Since ∠θ and ∠β can be calculated, the specific position of the relative angle within the azimuth angle can be determined using the two spatial positions. If the relative angle is above the azimuth angle, then the deflection angle of the camera to be calibrated, ∠OAC = ∠β - ∠θ; if the relative angle is below the azimuth angle, then the deflection angle of the camera to be calibrated, ∠OAC = ∠β + ∠θ.

[0145] During the process of obtaining an image of a reference object from a camera under calibration, there will be a certain degree of perspective deviation, resulting in a certain deviation in the shooting angle when photographing the reference object, i.e., a relative angle will appear. When a relative angle exists, if the focal length of the camera under calibration is constant, the larger the relative angle, the smaller the image size obtained at the same spatial position. If the spatial position of the reference object is on the shooting angle line (axis of the camera under calibration), the image size obtained will be larger than the size when a relative angle exists. That is, when the relative angle is 0°, the image size of the reference object taken by the camera under calibration at the same spatial position and focal length is the largest.

[0146] In one possible implementation, to ensure that the acquired image size is obtained along the axis of the camera to be calibrated, the data with the largest image size can be selected for calculation. The largest image size obtained at the same focal length and spatial location is used to calculate the deflection angle of the camera to be calibrated. This eliminates the need to calculate relative angles, thus improving the efficiency of deflection angle calculation. Figure 3b As shown, step S203 above can specifically be:

[0147] S2034. For the same focal length, determine the maximum imaging size from the imaging sizes obtained from the same spatial location.

[0148] S2035. The spatial position of the reference object when the image of the reference object with the largest imaging size is obtained is taken as the frontal spatial position.

[0149] S2036. Determine the deflection angle of the camera to be calibrated based on the frontal spatial position and the target position.

[0150] In this embodiment of the application, the determination of the target position of the camera to be calibrated in S2034-S2036 is similar to that in S201-S203, and will not be described in detail here.

[0151] Since the maximum image size obtained for the same focal length and spatial position is captured on the axis of the camera to be calibrated, the relative angle between the spatial position and the orientation of the camera to be calibrated is 0°. Therefore, the deflection angle of the camera to be calibrated is ∠β calculated using S2032 above.

[0152] Therefore, when the image size is the maximum image size obtained from the same focal length and spatial location, the tilt angle β of the camera to be calibrated is:

[0153] Where L is the distance from the camera to be calibrated to the reference object, i.e., the object distance, and H is the height of the camera to be calibrated.

[0154] In this embodiment of the application, when the imaging size is the maximum imaging size, the azimuth angle of the frontal spatial position relative to the target position in the vertical direction is the vertical deflection angle of the camera to be calibrated.

[0155] The azimuth angle of the camera's position relative to the target position in the horizontal direction is the horizontal deflection angle of the camera to be calibrated.

[0156] Based on the distances between the camera to be calibrated, the reference object, and the camera, combined with trigonometric relationships and the world coordinate system (where the X-axis points east, the Y-axis points north, and the horizontal deflection angle is the horizontal rotation angle α, and the vertical deflection angle is the downward tilt angle β), as shown below. Figure 2c As shown, the horizontal rotation angle of the camera to be calibrated is... Figure 2c In the figure, ∠α has a vertical deflection angle of . Figure 2c The method for calculating ∠β has been described in steps S201-S203 above, and will not be repeated here.

[0157] The following describes how to calculate the horizontal rotation angle ∠α: For example... Figure 3cAs shown, the right side represents the horizontal deflection angle of the camera to be calibrated, as observed from the XOY plane. Based on the calculated latitude and longitude (x, y) of the camera to be calibrated and the latitude and longitude (X, Y) of the reference object, with the camera to be calibrated as the origin, the relative position (x1, y1) of the reference object relative to the camera to be calibrated can be obtained, where x1 = Xx and y1 = Yy. The horizontal rotation angle is then...

[0158] Corresponding to the camera extrinsic parameter determination method provided in this application, this application also provides a camera extrinsic parameter determination device, such as... Figure 4 As shown, the apparatus includes:

[0159] The acquisition module 401 is used to acquire the true size of the reference object, the spatial position of the reference object at multiple times, and the reference object image obtained by the camera to be calibrated at each of the aforementioned times; wherein, the surface of the reference object facing the camera to be calibrated when it is at each spatial position is a sphere with the same radius of curvature.

[0160] The target position determination module 402 is used to determine the target position of the camera to be calibrated based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference object in each of the reference object images;

[0161] The deflection angle determination module 403 is used to determine the deflection angle of the camera to be calibrated based on the spatial position, the target position, and the imaging size.

[0162] In one possible embodiment, the deflection angle determination module 403 is specifically used to determine a relative angle based on each of the spatial positions, the target position, and each of the imaging sizes, wherein the relative angle is the angle between the orientation of each of the spatial positions relative to the camera to be calibrated and the orientation of the camera to be calibrated.

[0163] Based on the spatial positions and the target positions, calculate the angle between the azimuth of each spatial position relative to the camera to be calibrated and the vertical direction, and use it as the azimuth angle;

[0164] The deflection angle of the camera to be calibrated is determined based on the relative angle and the azimuth angle.

[0165] In one possible embodiment, the deflection angle determination module 403 is used to determine the maximum imaging size from the imaging sizes obtained from the same spatial location for the same focal length.

[0166] The spatial position of the reference object when the image of the reference object with the maximum imaging size is captured is determined as the frontal spatial position;

[0167] Based on the frontal spatial position and the target position, determine the deflection angle of the camera to be calibrated;

[0168] In one possible implementation, the target position determination module 402 is specifically used to determine the distances between at least three spatial positions and the camera to be calibrated, based on the focal length of the camera to be calibrated, the actual size, and the imaging size in reference images captured at at least three spatial positions, as the object distances corresponding to each spatial position; and to determine the position at which the distance from each of the at least three spatial positions is the object distance corresponding to that spatial position, as the target position of the camera to be calibrated.

[0169] The target position determination module 402 is specifically used to establish the theoretical correspondence between the spatial position, the distance between the reference object and the camera to be calibrated, and the target position;

[0170] Based on the theoretical correspondence, the target position of the camera to be calibrated is determined.

[0171] This application also provides an electronic device, such as... Figure 5 As shown, it includes:

[0172] Memory 501 is used to store computer programs;

[0173] When processor 502 executes the program stored in memory 501, it performs the following steps:

[0174] The true size of the reference object, the spatial position of the reference object at multiple moments, and the reference object image obtained by the camera to be calibrated at each of the aforementioned moments are obtained; wherein, the surface of the reference object facing the camera to be calibrated when it is at each of the aforementioned spatial positions is a sphere with the same radius of curvature.

[0175] The target position of the camera to be calibrated is determined based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference object in each of the reference object images.

[0176] The deflection angle of the camera to be calibrated is determined based on the spatial location, the target location, and the imaging size.

[0177] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 502, communication interface, and memory 501 communicating with each other via the communication bus.

[0178] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0179] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0180] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0181] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0182] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described methods for determining camera extrinsic parameters.

[0183] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the camera extrinsic parameter determination methods described in the above embodiments.

[0184] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0185] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0186] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0187] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for determining camera extrinsic parameters, characterized in that, The method includes: The true size of the reference object, the spatial position of the reference object at multiple moments, and the reference object image obtained by the camera to be calibrated at each of the aforementioned moments are obtained; wherein, the surface of the reference object facing the camera to be calibrated when it is at each of the aforementioned spatial positions is a sphere with the same radius of curvature. The target position of the camera to be calibrated is determined based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference object in each of the reference object images. The deflection angle of the camera to be calibrated is determined based on the spatial location, the target location, and the imaging size.

2. The method of claim 1, wherein, Determining the deflection angle of the camera to be calibrated based on the spatial positions, target positions, and imaging sizes includes: Based on each of the spatial positions, the target positions, and the imaging sizes, a relative angle is determined, wherein the relative angle is the angle between the position of each spatial position relative to the orientation of the camera to be calibrated and the orientation of the camera to be calibrated; Based on the spatial positions and the target positions, calculate the angle between the orientation of each spatial position relative to the camera to be calibrated and the vertical direction, and use it as the azimuth angle; The deflection angle of the camera to be calibrated is determined based on the relative angle and the azimuth angle.

3. The method of claim 1, wherein, Determining the deflection angle of the camera to be calibrated based on the spatial positions, target positions, and imaging sizes includes: For the same focal length, determine the maximum image size from the image sizes obtained from the same spatial location; The spatial position of the reference object when the image of the reference object with the maximum imaging size is captured is determined as the frontal spatial position; The deflection angle of the camera to be calibrated is determined based on the frontal spatial position and the target position.

4. The method of claim 1, wherein, The step of determining the target position of the camera to be calibrated based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference objects in each of the reference object images includes: Based on the focal length of the camera to be calibrated, the actual size, and the imaging size in the reference object images captured at least three of the spatial positions, the distances between the at least three spatial positions and the camera to be calibrated are determined as the object distances corresponding to each spatial position. The position at which the distance from each of the at least three spatial positions is determined as the object distance corresponding to that spatial position is used as the target position of the camera to be calibrated.

5. The method of claim 4, wherein, The step of determining the target position of the camera to be calibrated based on the actual size, each of the spatial positions, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference objects in each of the reference object images includes: Establish a theoretical correspondence between the spatial location, the distance between the reference object and the camera to be calibrated, and the target location; Based on the theoretical correspondence, the target position of the camera to be calibrated is determined.

6. An apparatus for determining camera extrinsic parameters, the apparatus comprising: The device includes: The acquisition module is used to acquire the true size of the reference object, the spatial position of the reference object at multiple times, and the reference object image obtained by the camera to be calibrated at each of the said times; wherein, the surface of the reference object facing the camera to be calibrated when it is at each of the said spatial positions is a sphere with the same radius of curvature. The target location determination module is used to determine the target location of the camera to be calibrated based on the actual size, each of the spatial locations, the intrinsic parameters of the camera to be calibrated, and the imaging size of the reference object in each of the reference object images. The deflection angle determination module is used to determine the deflection angle of the camera to be calibrated based on the spatial position, the target position, and the imaging size.

7. The apparatus according to claim 6, characterized in that, The deflection angle determination module is specifically used to determine the relative angle based on each of the spatial positions, the target position, and each of the imaging sizes, wherein the relative angle is the angle between the position of each of the spatial positions relative to the orientation of the camera to be calibrated and the orientation of the camera to be calibrated. Based on the spatial positions and the target positions, calculate the angle between the orientation of each spatial position relative to the camera to be calibrated and the vertical direction, and use it as the azimuth angle; The deflection angle of the camera to be calibrated is determined based on the relative angle and the azimuth angle. The deflection angle determination module is specifically used to determine the maximum imaging size from the imaging sizes obtained from the same spatial position for the same focal length. The spatial position of the reference object when the image of the reference object with the maximum imaging size is captured is determined as the frontal spatial position; Based on the frontal spatial position and the target position, determine the deflection angle of the camera to be calibrated; The target position determination module is specifically used to determine the distances between at least three spatial positions and the camera to be calibrated, based on the focal length of the camera to be calibrated, the actual size, and the imaging size in reference images obtained from at least three spatial positions, as the object distances corresponding to each spatial position. The position at which the distance from each of the at least three spatial positions is determined as the object distance corresponding to the spatial position is used as the target position of the camera to be calibrated. The target location determination module is specifically used to establish a theoretical correspondence between the spatial location, the distance between the reference object and the camera to be calibrated, and the target location; Based on the theoretical correspondence, the target position of the camera to be calibrated is determined.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-5.

10. A camera extrinsic parameter calibration system, characterized in that, The system includes: Vehicle equipment, reference object, information acquisition equipment, and electronic equipment as described in claim 8; The vehicle device is used to carry the reference object and move within the field of view of the camera to be calibrated; The surface of the reference object facing the camera to be calibrated at each spatial position is a sphere with the same radius of curvature. The information acquisition device is used to acquire the spatial position of the reference object at multiple different times, and send each spatial position and each time to the electronic device accordingly; The electronic device is configured to receive the actual size of the reference object, each of the spatial positions, and reference object images obtained by the camera to be calibrated capturing the reference object at each of the stated times, so as to implement the method as described in any one of claims 1 to 5.

Images