Method and device for calibrating camera by observing far target

Abstract

The invention discloses a method and a device for calibrating a camera by observing a far target. The method comprises the following steps: S1, determining a principal point coordinate of the camera according to a characteristic that an image of a line segment coinciding with an optical axis on the camera is a principal point; s2, in combination with celestial body observation, acquiring real field angles of at least two remote targets relative to the optical center of the camera; s3, solving the focal length of the camera based on the principal point coordinates and the real field angle; and S4, gradually expanding from a distortion known region in the center of the pixel plane, and solving the full-pixel plane distortion characteristic of the camera. The required equipment is simple, and the cost is low; wherein the far target can be simulated by an optical method, so that the target identification degree is improved and the calibration field is reduced. The calibration process is carried out step by step, large-scale parameter optimization and falling into local minimum are avoided, and the reliability of the calibration result is improved.

Description

Technical Field

[0001] This invention relates to camera calibration domains, and in particular to a method and apparatus for camera calibration by observing distant targets. Background Technology

[0002] Camera calibration is fundamental to computer vision research, especially photogrammetry. Current camera calibration methods can be categorized into autonomous and non-autonomous calibration based on whether a calibration object is required. Autonomous calibration utilizes characteristics of natural scenery in images, such as parallel roads and perpendicular corner lines, but the accuracy of the results is limited.

[0003] Non-autonomous calibration requires a calibration object and is further divided into active vision calibration and traditional calibration based on whether the camera's motion characteristics need to be known. Active vision calibration requires knowledge of the specific parameters of the camera's rigid body motion, thus placing high demands on the equipment. Traditional calibration is further divided into one-dimensional, two-dimensional, and three-dimensional calibration based on the dimension of the calibration object. One-dimensional calibration suffers from drawbacks such as sensitivity to noise in calibration accuracy, poor robustness of calibration results, and high implementation complexity. Three-dimensional calibration suffers from drawbacks such as high manufacturing cost of calibration objects, complex algorithms, limited calibration efficiency, and poor environmental adaptability. Two-dimensional calibration is currently the mainstream method, represented by Zhang Zhengyou's checkerboard calibration method, which has low manufacturing cost of calibration objects and readily available software to support the calibration algorithm. However, practical applications and numerous references show that this method has a low calibration success rate, and the calibration results often deviate from the true values. The fundamental reason for this is that this method requires large-scale parameter optimization, making it prone to getting trapped in local minima. Although many scholars have adopted more advanced optimization methods to solve this problem, the long optimization time and the need to pre-define the distortion model still limit the accuracy and efficiency of this method. Summary of the Invention

[0004] The main objective of this invention is to overcome the shortcomings of existing camera calibration methods, such as the tendency to get trapped in local minima during large-scale parameter optimization, low calibration success rate, limited accuracy, complex operation, and high cost. This invention proposes a method and apparatus for camera calibration by observing distant targets. It eliminates the need for complex calibration objects, simplifies the calibration process, improves calibration accuracy and robustness, flexibly adapts to the distortion characteristics of different lenses, and meets the needs of high-precision, low-cost, and high-reliability camera calibration in various scenarios.

[0005] The present invention adopts the following technical solution:

[0006] A method for camera calibration by observing distant targets includes the following steps:

[0007] S1: Determine the principal point coordinates of the camera based on the characteristic that the image of the line segment coinciding with the optical axis is the principal point on the camera.

[0008] S2: Combine astronomical observations to obtain the true subtended angles of at least two distant targets relative to the camera's optical center;

[0009] S3: Based on the principal point coordinates and the true subtended angle, calculate the focal length of the camera;

[0010] S4: Starting from the known distortion region at the center of the pixel plane, gradually expand to solve the full pixel plane distortion characteristics of the camera.

[0011] Step S1 specifically includes the following steps:

[0012] S11: The camera is mounted on a multi-degree-of-freedom turntable, and a gravity sensor is set at the multi-degree-of-freedom turntable. The multi-degree-of-freedom turntable includes a two-dimensional rotating platform and a three-dimensional rotating platform set up vertically. The camera lens faces upward and is placed on the two-dimensional rotating platform.

[0013] S12: Install the plumb line assembly on the two-dimensional rotating platform, adjust the three-dimensional rotating platform so that one axis is vertically upward, and then adjust the two-dimensional rotating platform. Based on the dual criteria of the unchanged gravity sensor reading and the fixed position of the plumb line image, make the camera optical axis vertically upward.

[0014] S13: Move the plumb line assembly to a fixed translation platform, and translate the translation platform so that the plumb line forms a point image in the camera;

[0015] S14: Extract the sub-pixel coordinates of the point image through edge detection, least squares fitting, and straight-line distance calculation. These coordinates are the camera principal point coordinates.

[0016] Step S14 specifically includes:

[0017] S141: Extract the two side contours of the vertical line using an edge detection algorithm;

[0018] S142: The central axis of the vertical line is obtained by fitting the two side contours using the least squares method;

[0019] S143: Extract the line segment equations of the three suspension lines at the bottom of the vertical line on the pixel plane, wherein the three suspension lines are used to suspend the ring;

[0020] S144: Find the point on the pixel plane that is closest to the line segment equations of the three suspension lines, and use this point as the sub-pixel coordinates of the bottom endpoint of the vertical line, which are the camera principal point coordinates.

[0021] Step S2 specifically includes the following steps:

[0022] S21: Record celestial bodies on video and simultaneously record the timestamp of each frame of the image;

[0023] S22: Extract the pixel coordinates of the centroid of the celestial body in each frame of the image, and then establish a linear relationship between the displacement of the celestial body on the pixel plane and the time interval.

[0024] S23: Calculate the change in the viewing angle of celestial bodies within the corresponding time interval based on the Earth's rotation angular velocity, construct the calibration relationship between pixel displacement and viewing angle change, and obtain the pixel quantity coefficient corresponding to each degree of viewing angle.

[0025] S24: Adjust the camera orientation so that the line connecting the two distant targets to be measured in the image is aligned with the direction of the celestial motion trajectory. Based on the pixel coordinate difference between the two targets and the calibration relationship, determine the true subtended angle between them and the camera's optical center.

[0026] Step S3 specifically includes the following steps:

[0027] S31: Select or construct two or more target points that are close to each other and fixed in size, and ensure that the target points can be clearly imaged within the camera's field of view;

[0028] S32: Mount the camera on the gimbal so that the camera can roll around the optical axis. During the roll, record the target point to obtain multiple frames of images and extract the pixel coordinates of the target point in each frame.

[0029] S33: Based on the known principal point coordinates of the camera and the assumed focal length, calculate the direction vector of the target point in the camera's normalized coordinate system in each frame, and calculate the angle between any two target points relative to the optical center using the vector dot product formula;

[0030] S34: Set the calculated included angle to be equal to the true subtended angle determined in step S2, and construct an overdetermined system of equations about the focal length;

[0031] S35: Solve the overdetermined system of equations using the nonlinear least squares method to obtain the optimal focal length value that minimizes the overall error.

[0032] The constructed overdetermined system of equations concerning the focal length is as follows:

[0033]

[0034] in: For the first The angle between any two target points relative to the optical center is calculated using the vector dot product formula at frame time. This is a hypothetical focal length; For the first The true subtended angle between two targets determined by astronomical observations at frame time.

[0035] In step S33, the principal point coordinates are set as unknowns and participate in the calculation of the direction vector together with the assumed focal length value, thus omitting step S1 of determining the principal point coordinates; and near the center of the pixel plane, assuming that the camera has no distortion or adopts a simple distortion model, the distortion parameters are jointly determined by the overdetermined equation system while solving for the optimal focal length in step S35.

[0036] Step S4 specifically includes the following:

[0037] S41: Select three or more distant target points, adjust the camera gimbal so that the target point image trajectory forms a concentric circle centered on the main point, and the trajectory radius covers the entire pixel plane from small to large, record multiple frames of images and extract the pixel coordinates of each target point;

[0038] S42: For the point P to be determined that falls in the region of unknown distortion, calculate its distorted normalized coordinates based on its pixel coordinates, the known principal point coordinates of the camera, and the focal length.

[0039] S43: Select two or more target points that fall within the known distortion area in the same frame image. Using the known true angle between the point P to be determined and the target point, establish a set of constraint equations about the distortion-free normalized coordinates of the point P to be determined through the vector dot product formula, and solve them to obtain the distortion-free normalized coordinates of the point P to be determined.

[0040] S44: Include the point P to be determined and its surrounding area into the known distortion area, repeat steps S42-S43, and gradually expand the known distortion area until it covers the entire pixel plane; finally, obtain the distortion characteristics of the entire pixel plane.

[0041] Step S4 also includes S45: Based on the dataset of the correspondence between the distorted normalized coordinates and the undistorted normalized coordinates of the full-pixel plane, a two-dimensional interpolation function is constructed using the scattered interpolation method to solve for the undistorted coordinates corresponding to any distorted coordinates, or the model parameters are solved by fitting a parameterized distortion model to characterize the distortion characteristics in analytical form.

[0042] An apparatus for camera calibration by observing distant targets, used to implement the above-described method for camera calibration by observing distant targets, comprising:

[0043] A multi-degree-of-freedom turntable assembly is used to support the camera and achieve attitude adjustment, which includes a three-dimensional rotary platform and a two-dimensional rotary platform;

[0044] The benchmark calibration module includes a plumb line assembly and a translation platform. The plumb line assembly is used to provide a principal point calibration benchmark, and the translation platform is used to translate the plumb line to form a point image.

[0045] A gravity sensing component is mounted on the multi-degree-of-freedom turntable assembly to assist in adjusting the axis of the three-dimensional rotating platform and the optical axis of the camera to a vertical state.

[0046] The image acquisition component, connected in conjunction with a camera, is used to record celestial bodies and distant targets, synchronously recording the timestamp of each frame of the image and outputting clear multi-frame images;

[0047] The data processing module is connected to the multi-degree-of-freedom turntable component, the reference calibration module, the gravity sensing component, and the image acquisition component. It is used to extract the pixel coordinates of the plumb line, celestial bodies, and distant targets in the image, establish calibration relationships, construct and solve the equations related to the principal point coordinates, focal length, and distortion characteristics, and finally output the camera principal point coordinates, focal length, and full-pixel plane distortion characteristics.

[0048] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. This invention adopts a step-by-step solution strategy, which avoids the problem of getting trapped in local minima in large-scale multi-parameter synchronous optimization in the prior art. At the same time, relying on the characteristic that the angle subtended by the distant target point relative to the optical center is almost unaffected by the camera's extrinsic parameters, it provides stable constraints for parameter solution, which greatly improves the calibration success rate and the reliability of the results.

[0050] 2. This invention does not rely on complex calibration objects, and the required equipment has a simple structure and low cost. Distant targets can be directly adopted from natural celestial bodies or simulated through optical methods (such as active emission, reflection, etc.), which can improve the target recognition and effectively reduce the calibration site requirements. Coupled with convenient operation procedures, it further reduces the threshold for calibration.

[0051] 3. This invention determines the true angle of a distant target relative to the optical center by observing the motion of celestial bodies based on astronomical theory. The measurement accuracy is high and it is not affected by camera extrinsic parameters, providing a precise benchmark for subsequent calculation of focal length and distortion parameters. At the same time, it simplifies the focal length calculation process by utilizing the characteristic of small distortion near the principal point, further ensuring the calibration accuracy of the core parameters.

[0052] 4. The present invention solves the distortion characteristics by gradually expanding from the known area at the center of the pixel plane to the entire plane. It does not require pre-setting the distortion model form, can flexibly adapt to the distortion characteristics of different lenses, does not require adjusting the model parameters for specific lenses, and is compatible with the calibration requirements of various cameras and optical equipment.

[0053] 5. The method of this invention is simple to implement, requires no complicated operating skills, and takes into account both high precision and convenience; it can be widely used in the factory calibration and subsequent calibration of various cameras such as hyperspectral cameras, panoramic cameras, and multi-view cameras, as well as optical equipment such as mobile phone camera systems, automotive autonomous driving vision systems, telescopes, and microscopes. It is suitable for large-scale production and practical application scenarios, and has broad industrialization prospects. Attached Figure Description

[0054] Figure 1 To enable the device of the optical axis plumb bob;

[0055] Figure 2 A device used to determine the position of the camera's principal point;

[0056] Figure 3(a) is a schematic diagram showing that the relative optical center angle of distant adjacent points remains unchanged when the optical center of the camera changes within a small range;

[0057] Figure 3(b) is a schematic diagram showing that the angle of a celestial body relative to a ground camera during its motion depends only on time.

[0058] Figure 3(c) Schematic diagram of the method for measuring the relative optical center angle of distant adjacent points based on astronomical theory;

[0059] Figure 4 This is a diagram illustrating the gradual expansion of the known distortion characteristics within a given region.

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0061] The present invention will be further described below through specific embodiments.

[0062] A method for camera calibration by observing distant targets includes the following steps:

[0063] S1: Determine the principal point coordinates based on the characteristic that the image of a line segment coinciding with the optical axis on the camera is the principal point. This includes the following steps:

[0064] S11: The camera is mounted on a multi-degree-of-freedom turntable, which includes a vertically positioned two-dimensional rotation platform and a three-dimensional rotation platform. The camera lens faces upwards and is fixed to the two-dimensional rotation platform. A mobile phone is placed on the two-dimensional rotation platform, and its built-in gravity sensor monitors the turntable's attitude in real time to assist in subsequent optical axis alignment. Figure 1 As shown.

[0065] S12: Install the plumb line assembly on the two-dimensional rotating platform, adjust the three-dimensional rotating platform so that one axis is vertically upward, and then adjust the two-dimensional rotating platform. Based on the dual criteria of the unchanged gravity sensor reading and the fixed position of the plumb line image, make the camera optical axis vertically upward.

[0066] In this step, the plumb line assembly includes a support and a plumb line. The support is fixed to the two-dimensional rotating platform, and the plumb line is soaked in phosphor solution to improve image clarity. Both ends of the plumb line are fixed to iron rings with thin lines to ensure the straightness of the plumb line and facilitate subsequent data processing. Then, the three-dimensional rotating platform is adjusted so that one of its axes (hereinafter referred to as the z-axis) is vertically upward. The adjustment is based on the fact that the reading of the gravity sensor set at the multi-degree-of-freedom turntable remains unchanged when the three-dimensional rotating platform rotates around the z-axis. Then, the three-dimensional rotating platform is fixed, and the attitude of the two-dimensional rotating platform is adjusted until the optical axis of the camera lens is vertically upward. The judgment criterion is: when the three-dimensional rotating platform rotates around the z-axis, the pixel coordinate position of the plumb line in the image acquired by the camera remains unchanged, which indicates that the optical axis coincides with the plumb line and is in a vertical state.

[0067] S13: Remove the plumb line and its support from the two-dimensional rotating platform, fix the plumb line assembly to the translation platform, which is fixed relative to the ground and is a two-dimensional translation platform. Translate the translation platform so that the plumb line forms a point image in the camera, that is, the image of the plumb line on the camera is a point, and the pixel coordinates corresponding to this point are the principal point coordinates.

[0068] S14: Use computer vision methods to determine the sub-pixel coordinates of the bottom endpoint of the vertical line. Use these as the principal point coordinates. Extract the sub-pixel coordinates of the point image through edge detection, least squares fitting, and straight-line distance calculation. These coordinates are the camera principal point coordinates. Specifically, this includes:

[0069] S141: Extract the two side contours of the vertical line using an edge detection algorithm;

[0070] S142: The central axis of the vertical line is obtained by fitting the contours on both sides using the least squares method.

[0071] S143: Extend this central axis downwards, and suspend a ring at the bottom of the plumb line using three lines, such as... Figure 2 As shown, extract the line segment equations of the three hanging lines at the bottom of the plumb line on the pixel plane;

[0072] S144: Find the point on the pixel plane that is closest to the equations of the line segments of the three suspension lines, and use this point as the sub-pixel coordinates of the bottom endpoint of the vertical line, which are the coordinates of the camera principal point.

[0073] In this step, the plumb line segment is aligned with the plumb line's optical axis by translating the plumb line. Based on the property that the image remains unchanged when the camera rotates around the plumb line, the camera's optical axis is adjusted to a plumb position. The rotation axis is adjusted to a plumb position by utilizing the property that the gravity sensor reading remains unchanged when the camera rotates around the plumb line. The projection of the plumb line onto the pixel plane is processed using computer vision methods to obtain the sub-pixel principal point coordinates.

[0074] S2: Combining astronomical observations, obtain the true subtended angles of at least two distant targets relative to the camera's optical center; this specifically includes the following steps:

[0075] S21: Record celestial bodies on video and simultaneously record the timestamp of each frame of the image;

[0076] S22: Extract the pixel coordinates of the celestial body's center of mass in each frame of the image, and then establish a linear relationship between the displacement of the celestial body on the pixel plane and the time interval. For convenience, fix the camera to record video of the sun, moon, or stars for a sufficient period of time, leaving images of these celestial bodies on the camera and recording the recording time of each photo.

[0077] S23: Based on the Earth's rotation angular velocity, calculate the change in angular perspective of celestial bodies within corresponding time intervals, establish a calibration relationship between pixel displacement and angular perspective change, and obtain the pixel count coefficient corresponding to each degree of angular perspective. Specifically, this includes the following:

[0078] Extracting the centroid pixel coordinates of celestial bodies from each frame of the celestial motion sequence image. Specifically, for planar celestial bodies such as the Sun and Moon, their centers are calculated using threshold segmentation and gray-scale weighted centroid method; for point-like celestial bodies such as stars, their centers are located using two-dimensional Gaussian fitting.

[0079] Subsequently, the relationship between pixel displacement and time interval in the celestial image was established through data processing. This relationship is represented as follows:

[0080] ( )

[0081] in: Indicates the time interval Within, the distance the celestial body's center of mass moves on the pixel plane (unit: pixels); Indicates the time interval between adjacent image frames (unit: seconds); This represents the average speed of celestial bodies moving on the pixel plane (unit: pixels / second), which is calculated by linear regression on multiple groups. Data fitting was used to obtain the results; Represents the pixel coordinates at time t; Represents the pixel coordinates at time t+Δt.

[0082] Calculate the time interval between adjacent images based on their timestamps. (Unit: seconds). According to astronomical theory, the motion of celestial bodies relative to an observer on Earth over a short period of time is primarily due to the Earth's rotation, and its angular velocity is approximately constant. Therefore, the change in the angular angle of the celestial body relative to the camera's optical center... (Unit: degree) can be calculated using the following formula:

[0083] ( )

[0084] in: Indicates the time interval The angular distance a celestial body moves on the celestial sphere, i.e., the change in angular perspective; Indicates the time interval between adjacent images (unit: seconds); This represents the angular velocity of a celestial body relative to an observer on Earth (unit: degrees per second). For the Sun, Moon, and most stars, this angular velocity... It can be taken from the average angular velocity of the Earth's rotation, that is:

[0085] ( )

[0086] Based on the results obtained in the first two steps, establish the celestial pixel coordinate change. With the change in perspective The calibration relationship between them. This relationship can be characterized as:

[0087] ( )

[0088] in: The calibration coefficient (unit: pixels / degree) represents the number of pixels per degree of view of the camera in the direction of the celestial body's motion.

[0089] S24: Adjust the camera orientation so that the line connecting the two distant targets in the image aligns with the direction of the celestial trajectory. Based on the pixel coordinate difference and calibration relationship between the two targets, determine the true angle between them relative to the camera's optical center. Specifically, point the camera at two fixed distant target points whose angle needs to be measured and take a picture. Adjust the camera's azimuth angle so that the line connecting these two target points in the image aligns with the direction of the celestial trajectory used for calibration in step S23. Based on the pixel difference between adjacent points, interpolate the angle of view change according to the result of step S23. The result is the angle between adjacent points and the optical center. Figure 3(a) shows that the angle subtended by a distant adjacent point relative to the optical center remains unchanged when the optical center of the camera changes within a small range; Figure 3(b) shows that the angle subtended by a celestial body relative to the ground camera during celestial motion is only related to time; Figure 3(c) is a schematic diagram of the method for measuring the angle subtended by a distant adjacent point relative to the optical center based on astronomical theory.

[0090] In this embodiment, high-precision astronomical calculations of the angle subtended by celestial bodies relative to an observer on Earth are used to measure the angle subtended by a distant adjacent target relative to the camera's optical center. The relationship between the change in the celestial body's position on the pixel plane and the passage of time is obtained by recording celestial videos and recording the time. The relationship between the change in the angle subtended by the celestial body relative to the camera and the passage of time is obtained by comparing it with astronomical theories. Finally, the angle subtended by the distant adjacent point and the celestial body's trajectory on the pixel plane is determined by comparing the projections of these two points onto the pixel plane.

[0091] S3: Determine the camera's focal length based on the principal point coordinates and the true subtended angle. This step determines the camera's focal length based on the characteristic that the angle between adjacent points far from the optical center and the optical center remains constant when the camera's optical center changes within a small range. Specifically, it includes the following steps:

[0092] S31: Select or construct two or more closely spaced and fixed target points to ensure that the target points can be clearly imaged within the camera's field of view. The target points can be artificially placed fixed point light sources, or feature points on buildings, towers, or mountaintops, etc.

[0093] S32: Mount the camera on a gimbal so that it can roll around the optical axis. During the roll, record video of the target point to obtain multiple frames. Extract the pixel coordinates of the target point in each frame. Specifically, when mounting the camera on the gimbal, allow it to roll, pitch, and yaw around the optical axis, with the camera's optical center approximately located on the roll axis. Adjust the gimbal and the target point so that the image of the target point falls near the center of the pixel plane. Record video of the target point during the roll to obtain multiple frames, ensuring that each frame contains images of two or more target points. Extract the pixel coordinates of the target point in each frame. .

[0094] S33: Based on the known principal point coordinates of the camera and the assumed focal length, calculate the direction vector of the target point in the camera's normalized coordinate system in each frame, and calculate the angle between any two target points relative to the optical center using the vector dot product formula. Details are as follows:

[0095] For any two target points A and B in any frame of an image, based on the known coordinates of the camera principal point... and the assumed focal length value Calculate the angles between them and the camera's optical center O. The calculation formula is as follows:

[0096] First, calculate the direction vectors of target points A and B in the camera's normalized coordinate system. :

[0097] ( )

[0098] ( )

[0099] Then, the included angle is calculated using the vector dot product formula. :

[0100] ( )

[0101] in:

[0102] This represents the pixel coordinates of target points A and B in the current frame image; This represents the known coordinates of the camera principal point; Indicates a hypothetical camera focal length; This represents the direction vectors of target points A and B in the camera coordinate system; Indicates the use of the currently assumed focal length. The angle between two points A and B is calculated.

[0103] S34: Set the calculated included angle to be equal to the true subtended angle determined in step S2, and construct an overdetermined system of equations regarding the focal length. In this step, the system of equations regarding the camera focal length is established. The equation:

[0104] ( )

[0105] in: Is it using an assumed focal length? The included angle calculated according to formula (7) is the focal length. The function; It is the true angle between two target points determined through astronomical observation, and is a known constant value.

[0106] Since multiple frames of images are acquired during the camera's rotation (let the total number of frames be M), and the above equations can be calculated once for target points A and B in each frame, an overdetermined system of equations containing M equations can be constructed:

[0107] ( )

[0108] in: For the first The angle between any two target points relative to the optical center is calculated using the vector dot product formula at frame time. This is a hypothetical focal length; For the first The true subtended angle between two targets determined by astronomical observations at frame time.

[0109] S35: Solve the overdetermined system of equations using the nonlinear least squares method to obtain the optimal focal length value that minimizes the overall error. In this step, the overdetermined system of equations is solved using the nonlinear least squares method to find the optimal focal length value. This minimizes the overall error between the calculated values ​​and the true values ​​of all equations. That is, it involves solving the following optimization problem:

[0110] ( )

[0111] The optimal solution obtained This is the final determination of the optimal camera focal length.

[0112] Furthermore, if the data processing accuracy is high enough, in step S33, the principal point coordinates can be set as unknowns and used together with the assumed focal length value to participate in the calculation of the direction vector, thus omitting step S1 of determining the principal point coordinates; and near the center of the pixel plane, assuming the camera has no distortion or uses a simple distortion model, while solving for the optimal focal length in step S35, the distortion parameters are jointly determined by the overdetermined equation system.

[0113] In this embodiment, the camera focal length is determined based on the characteristic that the angle between adjacent points far from the optical center remains unchanged relative to the optical center when the camera's optical center changes within a small range. The negligible distortion effect near the camera's principal point further simplifies the constraint relationship between the target and the image. The angle between the adjacent points and the optical center is calculated using the pixel coordinates of the distant adjacent points, the principal point coordinates, and the camera focal length. A system of equations regarding the camera focal length is constructed by aligning this angle with the result previously determined using astronomical methods; the camera focal length is determined by solving this system of equations. By improving the accuracy of data processing, this method can also treat the principal point coordinates as unknowns for solution, thus omitting step S1. By arranging more distant adjacent points and employing a simplified distortion model to characterize the distortion characteristics of the central region, calibration accuracy is improved.

[0114] S4: Starting from the known distortion region at the center of the pixel plane, gradually expand to solve for the full-pixel plane distortion characteristics of the camera. Specifically, this includes the following:

[0115] S41: Select three or more distant target points, adjust the camera gimbal so that the target point image trajectory forms a concentric circle centered on the main point, and the trajectory radius covers the entire pixel plane from small to large, record multiple frames of images and extract the pixel coordinates of each target point;

[0116] S42: For the point P to be determined, which falls within the region of unknown distortion, calculate its distorted normalized coordinates based on its pixel coordinates and the known principal point coordinates and focal length of the camera. The calculation formula is as follows:

[0117] Extract the pixel coordinates of the point P to be determined. Calculate the distorted normalized coordinates of the point based on the camera's intrinsic parameters. The calculation formula is as follows:

[0118] ( )

[0119] ( )

[0120] in: This represents the coordinates of point P in the image pixel coordinate system (unit: pixels); This represents the calculated distortion-normalized coordinates, which are dimensionless physical coordinates that include the effects of lens distortion. Indicates the principal point coordinates of the camera (unit: pixels); This indicates the camera's focal length (in pixels) on the x and y axes.

[0121] S43: Select two or more target points falling within the known distortion region in the same frame image. Using the known true angle between the target point and the point to be determined (P), establish a system of constraint equations regarding the distortion-free normalized coordinates of the target point P using the vector dot product formula. Solve these equations to obtain the distortion-free normalized coordinates of the target point P. The specific calculation method is as follows:

[0122] Let the distortion-free normalized coordinates of the point P to be determined be: For each known point A, its distortion-free normalized coordinates are... Given. The true angle between point P and point A needs to be determined. These are known quantities. The core calculation formula is based on the camera model with shared optical centers, and the vector... and The included angle between them should be equal to the subtended angle. Thus, the constraint equations are established:

[0123] ( )

[0124] Substituting the vector, we get:

[0125] ( )

[0126] in: : are the distortion-free normalized coordinates of the point P to be determined; The distortion-free normalized coordinates of the known point A; The known true angle between point P and point A is the angle to be determined. It represents the magnitude of the vector.

[0127] By combining two or more such constraint equations (i.e., constructing equations about the point P to be solved using points A, B, ... respectively), a solution is formed for the unknown. The system of equations is overdetermined because the number of equations exceeds the number of unknowns. Therefore, optimization algorithms such as the least squares method can be used to solve the system, allowing for the accurate determination of the distortion-free normalized coordinates of the point P. .

[0128] S44: Include the point P to be determined and its surrounding area into the known distortion area, repeat steps S42-S43, and gradually expand the known distortion area until it covers the entire pixel plane; finally, obtain the distortion characteristics of the entire pixel plane.

[0129] For a point falling within an area of ​​unknown distortion characteristics, if two points within a frame can be found that fall within an area of ​​known distortion characteristics, then the normalized coordinates of this point can be determined using the known angle between it and the target point within the known distortion characteristics area. Since both the normalized coordinates and the distorted normalized coordinates of the point discussed in step S43 are known, the area surrounding this point can be incorporated into the area of ​​known distortion characteristics. This process continues, and the area of ​​known distortion characteristics becomes increasingly larger, eventually yielding the distortion characteristics of the entire pixel plane, such as... Figure 4 As shown.

[0130] S45: A dataset based on the correspondence between the normalized coordinates after distortion and the normalized coordinates without distortion on the full-pixel plane. A two-dimensional interpolation function is constructed using the scattered interpolation method to solve for the coordinates without distortion corresponding to any distorted coordinates, or the model parameters are solved by fitting a parameterized distortion model to characterize the distortion characteristics in analytical form.

[0131] Specifically, the above steps yielded a discrete distortion mapping dataset. The superscript i represents the i-th known point. Based on this dataset, the camera's distortion characteristics can be characterized in the following two ways:

[0132] Scatter interpolation:

[0133] For any distorted normalized coordinate Its corresponding distortion-free normalized coordinates It can be obtained through a two-dimensional interpolation function:

[0134] ( )

[0135] ( )

[0136] in: As input, the known distorted and normalized coordinates; The output is the distortion-free normalized coordinates to be determined.

[0137] Based on the aforementioned discrete dataset The constructed two-dimensional interpolation function.

[0138] Distortion model parameter fitting method:

[0139] Given a parameterized distortion model, for example:

[0140] ( )

[0141] ( )

[0142] in: ; For radial distortion parameters; These are tangential distortion parameters;

[0143] Then model parameters This can be obtained by solving the following nonlinear least squares problem:

[0144] ( )

[0145] Where: N is the total number of known data points; The known, distortion-free, normalized coordinates of the i-th point; The function defined in the above distortion model outputs a pair of parameters k. The predicted value; Let be the normalized coordinates of the i-th point after known distortion.

[0146] This method determines camera distortion characteristics based on the property that the angle between adjacent points far from the optical center remains unchanged relative to the optical center when the optical center changes within a small range. This method starts from the central region where the distortion characteristics are known and gradually determines the distortion characteristics of adjacent regions, finally extending to the entire pixel plane. For each distant target located near the region with known distortion characteristics, its distorted normalized coordinates are calculated based on its pixel coordinates and camera intrinsic parameters; its normalized coordinates are determined based on the normalized coordinates of the target and two or more adjacent points falling within the region with known distortion characteristics, as well as the angle between them relative to the optical center; finally, the correspondence between the normalized coordinates and the distorted normalized coordinates is used to obtain the distortion characteristics near this point, thus expanding the region with known distortion characteristics. The specific distortion model uses scattered interpolation for representation, or a distortion model can be introduced for parameter identification to achieve an analytical representation of the distortion. (See Figure 3.)

[0147] In this embodiment, a device for camera calibration by observing distant targets is also proposed to implement the above-described method for camera calibration by observing distant targets, including:

[0148] A multi-degree-of-freedom turntable assembly is used to support a camera and achieve attitude adjustment. It includes a three-dimensional rotary platform and a two-dimensional rotary platform.

[0149] The benchmark calibration module includes a plumb line assembly and a translation platform. The plumb line assembly is used to provide the benchmark for the principal point calibration, and the translation platform is used to translate the plumb line to form a point image.

[0150] The gravity sensing component is mounted on the multi-degree-of-freedom turntable assembly to assist in adjusting the axis of the three-dimensional rotating platform and the optical axis of the camera to a vertical state.

[0151] The image acquisition component, connected in conjunction with a camera, is used to record celestial bodies and distant targets, synchronously recording the timestamp of each frame and outputting clear multi-frame images.

[0152] The data processing module is connected to the multi-degree-of-freedom turntable component, the reference calibration module, the gravity sensing component, and the image acquisition component. It is used to extract the pixel coordinates of the plumb line, celestial bodies, and distant targets in the image, establish calibration relationships, construct and solve the equations related to the principal point coordinates, focal length, and distortion characteristics, and finally output the camera principal point coordinates, focal length, and full-pixel plane distortion characteristics.

[0153] The present invention provides a method and apparatus for camera calibration by observing distant targets. With distant target observation as its core, it employs four core processes: determining the principal point coordinates, acquiring the true sub-angle of the distant target, solving for the camera focal length, and extending the full-pixel plane distortion characteristics. The method utilizes a plumb line and a multi-degree-of-freedom turntable to achieve precise principal point positioning, leverages the high-precision characteristics of astronomical observation to determine the target sub-angle, and solves for the focal length and distortion characteristics step-by-step based on the principle that the sub-angle of a distant target is unaffected by small-scale changes in the optical center. Finally, it completes the full parameter calibration of the camera. The supporting apparatus, through the synergistic action of the multi-degree-of-freedom turntable assembly, the reference calibration module, the gravity sensing assembly, the image acquisition assembly, and the data processing module, provides stable and reliable hardware support and data processing assurance for the implementation of the calibration method.

[0154] This invention can be extended to the design, production, and sale of various long-range targets related to camera calibration. These long-range targets include various simulated long-range targets generated by active emission, reflection technology, refraction technology, spectral technology, parallel light source focusing, and combinations thereof, which can be directly adapted to the calibration requirements of this invention.

[0155] This invention can be extended to the design, production, and sale of visual target imaging point position control devices related to camera calibration. The device includes visual target imaging point position control functions achieved by controlling the movement of the target, controlling the movement of the camera, and a combination of both, providing auxiliary support for the calibration process.

[0156] This invention can be extended to the development of camera calibration software. The software includes various forms such as application toolkits for large software programs, remote websites, and applications (Apps) for devices such as mobile phones and tablets, integrating the core logic of the calibration method of this invention.

[0157] This invention can be applied to the mass production or calibration of various related products. Specifically, it includes the calibration of various cameras (including but not limited to hyperspectral cameras, panoramic cameras, and multi-view cameras), camera-equipped vision systems (mobile phones, automobiles, etc.), and optical devices (telescopes, microscopes, and optical lenses) before and after leaving the factory, adapting to the needs of large-scale applications.

[0158] All of the above-mentioned extended applications rely on the core calibration method of this invention, forming a technology transformation path from core technology to related products, software and large-scale applications, thus ensuring the full extension of the technical value of this invention.

[0159] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0160] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0161] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A method for camera calibration by observing distant targets, characterized in that, Includes the following steps: S1: Determine the principal point coordinates of the camera based on the characteristic that the image of the line segment coinciding with the optical axis is the principal point on the camera. S2: Combine astronomical observations to obtain the true subtended angles of at least two distant targets relative to the camera's optical center; S3: Based on the principal point coordinates and the true subtended angle, calculate the focal length of the camera; S4: Starting from the known distortion region at the center of the pixel plane, gradually expand to solve the full pixel plane distortion characteristics of the camera.

2. The method for camera calibration by observing distant targets according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11: The camera is mounted on a multi-degree-of-freedom turntable, and a gravity sensor is set at the multi-degree-of-freedom turntable. The multi-degree-of-freedom turntable includes a two-dimensional rotating platform and a three-dimensional rotating platform set up vertically. The camera lens faces upward and is placed on the two-dimensional rotating platform. S12: Install the plumb line assembly on the two-dimensional rotating platform, adjust the three-dimensional rotating platform so that one axis is vertically upward, and then adjust the two-dimensional rotating platform. Based on the dual criteria of the unchanged gravity sensor reading and the fixed position of the plumb line image, make the camera optical axis vertically upward. S13: Move the plumb line assembly to a fixed translation platform, and translate the translation platform so that the plumb line forms a point image in the camera; S14: Extract the sub-pixel coordinates of the point image through edge detection, least squares fitting, and straight-line distance calculation. These coordinates are the camera principal point coordinates.

3. The method for camera calibration by observing distant targets as described in claim 2, characterized in that, Step S14 specifically includes: S141: Extract the two side contours of the vertical line using an edge detection algorithm; S142: The central axis of the vertical line is obtained by fitting the two side contours using the least squares method; S143: Extract the line segment equations of the three suspension lines at the bottom of the vertical line on the pixel plane, wherein the three suspension lines are used to suspend the ring; S144: Find the point on the pixel plane that is closest to the line segment equations of the three suspension lines, and use this point as the sub-pixel coordinates of the bottom endpoint of the vertical line, which are the camera principal point coordinates.

4. The method for camera calibration by observing distant targets as described in claim 1, characterized in that, Step S2 specifically includes the following steps: S21: Record celestial bodies on video and simultaneously record the timestamp of each frame of the image; S22: Extract the pixel coordinates of the centroid of the celestial body in each frame of the image, and then establish a linear relationship between the displacement of the celestial body on the pixel plane and the time interval. S23: Calculate the change in the viewing angle of celestial bodies within the corresponding time interval based on the Earth's rotation angular velocity, construct the calibration relationship between pixel displacement and viewing angle change, and obtain the pixel quantity coefficient corresponding to each degree of viewing angle. S24: Adjust the camera orientation so that the line connecting the two distant targets to be measured in the image is aligned with the direction of the celestial motion trajectory. Based on the pixel coordinate difference between the two targets and the calibration relationship, determine the true subtended angle between them and the camera's optical center.

5. The method for camera calibration by observing distant targets as described in claim 1, characterized in that, Step S3 specifically includes the following steps: S31: Select or construct two or more target points that are close to each other and fixed in size, and ensure that the target points can be clearly imaged within the camera's field of view; S32: Mount the camera on the gimbal so that the camera can roll around the optical axis. During the roll, record the target point to obtain multiple frames of images and extract the pixel coordinates of the target point in each frame. S33: Based on the known principal point coordinates of the camera and the assumed focal length, calculate the direction vector of the target point in the camera's normalized coordinate system in each frame, and calculate the angle between any two target points relative to the optical center using the vector dot product formula; S34: Set the calculated included angle to be equal to the true subtended angle determined in step S2, and construct an overdetermined system of equations about the focal length; S35: Solve the overdetermined system of equations using the nonlinear least squares method to obtain the optimal focal length value that minimizes the overall error.

6. The method for camera calibration by observing distant targets as described in claim 5, characterized in that, The constructed overdetermined system of equations concerning the focal length is as follows: ； in: For the first The angle between any two target points relative to the optical center is calculated using the vector dot product formula at frame time. This is a hypothetical focal length; For the first The true subtended angle between two targets determined by astronomical observations at frame time.

7. The method for camera calibration by observing distant targets as described in claim 5, characterized in that, In step S33, the principal point coordinates are set as unknowns and participate in the calculation of the direction vector together with the assumed focal length value, thus omitting step S1 of determining the principal point coordinates; and near the center of the pixel plane, assuming that the camera has no distortion or adopts a simple distortion model, the distortion parameters are jointly determined by the overdetermined equation system while solving for the optimal focal length in step S35.

8. The method for camera calibration by observing distant targets as described in claim 1, characterized in that, Step S4 specifically includes the following: S41: Select three or more distant target points, adjust the camera gimbal so that the target point image trajectory forms a concentric circle centered on the main point, and the trajectory radius covers the entire pixel plane from small to large, record multiple frames of images and extract the pixel coordinates of each target point; S42: For the point P to be determined that falls in the region of unknown distortion, calculate its distorted normalized coordinates based on its pixel coordinates, the known principal point coordinates of the camera, and the focal length. S43: Select two or more target points that fall within the known distortion area in the same frame image. Using the known true angle between the point P to be determined and the target point, establish a set of constraint equations about the distortion-free normalized coordinates of the point P to be determined through the vector dot product formula, and solve them to obtain the distortion-free normalized coordinates of the point P to be determined. S44: Include the point P to be determined and its surrounding area into the known distortion area, repeat steps S42-S43, and gradually expand the known distortion area until it covers the entire pixel plane; finally, obtain the distortion characteristics of the entire pixel plane.

9. The method for camera calibration by observing distant targets as described in claim 8, characterized in that, Step S4 also includes S45: Based on the dataset of the correspondence between the distorted normalized coordinates and the undistorted normalized coordinates of the full-pixel plane, a two-dimensional interpolation function is constructed using the scattered interpolation method to solve for the undistorted coordinates corresponding to any distorted coordinates, or the model parameters are solved by fitting a parameterized distortion model to characterize the distortion characteristics in analytical form.

10. An apparatus for camera calibration by observing distant targets, used to implement the method for camera calibration by observing distant targets as described in any one of claims 1-9, characterized in that, include: A multi-degree-of-freedom turntable assembly is used to support the camera and achieve attitude adjustment, which includes a three-dimensional rotary platform and a two-dimensional rotary platform; The benchmark calibration module includes a plumb line assembly and a translation platform. The plumb line assembly is used to provide a principal point calibration benchmark, and the translation platform is used to translate the plumb line to form a point image. A gravity sensing component is mounted on the multi-degree-of-freedom turntable assembly to assist in adjusting the axis of the three-dimensional rotating platform and the optical axis of the camera to a vertical state. The image acquisition component, connected in conjunction with a camera, is used to record celestial bodies and distant targets, synchronously recording the timestamp of each frame and outputting clear multi-frame images; The data processing module is connected to the multi-degree-of-freedom turntable component, the reference calibration module, the gravity sensing component, and the image acquisition component. It is used to extract the pixel coordinates of the plumb line, celestial bodies, and distant targets in the image, establish calibration relationships, construct and solve the equations related to the principal point coordinates, focal length, and distortion characteristics, and finally output the camera principal point coordinates, focal length, and full-pixel plane distortion characteristics.

Images