Pan / tilt / zoom (PTZ) camera calibration method and device for high-precision geographic positioning

Abstract

The invention provides a PTZ (Pan / Tilt / Zoom) camera calibration method and device for high-precision geographic positioning. The method comprises the following steps: step S101, acquiring internal parameters of a PTZ camera under a plurality of zoom parameters; s102, training a preset machine learning model by taking the internal parameters of the PTZ camera under the plurality of zoom parameters as training data, and establishing a change relationship between the zoom parameters of the PTZ camera and the internal parameters according to the machine learning model after the training is completed; step S103, establishing a plurality of control points and a plurality of check points on the ground, calculating external parameters of the PTZ camera based on the plurality of control points, the plurality of check points and the change relation between the zoom parameters and the internal parameters, and verifying the calibration precision and / or adjusting the machine learning model; and step S104, based on the pan-tilt of the PTZ camera and the machine learning model, obtaining a change relationship between the zoom parameter and the external parameter of the PTZ camera. According to the method, the internal parameters and the external parameters of the PTZ camera can be accurately obtained under multiple zoom parameters, the calibration process is simplified, and the calibration efficiency and precision are improved.

Description

Technical Field

[0001] The present invention relates to the field of geographic information system and computer vision technology, and in particular to a PTZ camera calibration method and device for high-precision geographic positioning. Background Art

[0002] With the continuous development of geographic information systems (GIS) and computer vision technology, high-precision geolocation is increasingly being used in many fields, such as intelligent transportation, urban planning, and environmental monitoring. In these applications, PTZ (Pan-Tilt-Zoom, pan-tilt-zoom omnidirectional (left / right / up / down) movement and lens zoom and zoom control) cameras are an important data acquisition device, and the accuracy of their geolocation directly affects the results of subsequent data processing and analysis. However, due to the zoom characteristics of PTZ cameras, their internal parameters (such as focal length, image principal point position, etc.) and external parameters (such as camera posture, position, etc.) will change with the zoom operation, which brings great challenges to high-precision geolocation.

[0003] Traditional camera calibration methods, such as the checkerboard method, are usually suitable for cameras with fixed focal lengths. These methods calculate the camera's intrinsic parameters by taking images of specific patterns and extracting feature points using image processing techniques. However, for PTZ cameras, since their focal lengths change with zoom operations, traditional calibration methods cannot accurately obtain their intrinsic parameters. In addition, other intrinsic parameters of PTZ cameras (such as the position of the principal point) and distortion parameters (such as mirror distortion parameters and tangential distortion parameters) also change with zoom operations, making high-precision geolocation more difficult.

[0004] Although some studies have proposed calibration methods for PTZ cameras, these methods often require complex calibration processes and have limited accuracy. Therefore, how to achieve high-precision geolocation of PTZ cameras while simplifying the calibration process has become an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a PTZ camera calibration method and device for high-precision geographic positioning, which is used to solve the problems of changes in internal and external parameters caused by the zoom operation of the PTZ camera in the prior art, as well as the shortcomings of the traditional calibration method in terms of accuracy and efficiency, so as to achieve high-precision geographic positioning of the PTZ camera and simplify the calibration process.

[0006] In order to achieve the above-mentioned object, in a first aspect, the present invention provides a PTZ camera calibration method for high-precision geolocation, comprising:

[0007] Step S101, obtaining internal parameters of a PTZ camera under multiple zoom parameters;

[0008] Step S102: using the internal parameters of the PTZ camera under multiple zoom parameters as training data to train a preset machine learning model; after the training is completed, establishing a relationship between the zoom parameters of the PTZ camera and the internal parameters according to the machine learning model;

[0009] Step S103: Establish multiple control points and multiple check points on the ground, calculate the external parameters of the PTZ camera based on the multiple control points, multiple check points and the changing relationship between the zoom parameters and the internal parameters, and verify the calibration accuracy and / or adjust the machine learning model;

[0010] Step S104: Based on the pan / tilt of the PTZ camera and the machine learning model, the relationship between the zoom parameters of the PTZ camera and the external parameters is obtained.

[0011] According to a PTZ camera calibration method for high-precision geo-positioning provided by the present invention, step S101 specifically includes:

[0012] Provide a chessboard, and take multiple photos at multiple zoom parameters through a PTZ camera;

[0013] Extract the pixel coordinates of the corner points of the chessboard in each photo. The relationship between the pixel coordinates of the corner points and the image coordinates is:

[0014]

[0015] Where (u0, v0) is the pixel coordinate of the principal point, dx and dy are the length and width of the unit pixel, (x, y) is the image coordinate, and (u, v) is the pixel coordinate;

[0016] Perform distortion correction on the PTZ camera to obtain the corrected image coordinates. The expression of the corrected image coordinates is:

[0017]

[0018] Where (x′, y′) is the corrected image coordinate, (x dr ,y dr ) is radial distortion, (x dt ,y dt ) is the tangential distortion, k1, k2, k3 and p1, p2 are the distortion parameters of radial distortion and tangential distortion respectively;

[0019] According to the camera coordinates of the PTZ camera and the corrected image coordinates, the corrected camera coordinates are obtained. The expression of the corrected camera coordinates is:

[0020]

[0021] In the formula, (X c ,Y c ,Zv ) is the camera coordinate, (X v ′,Y c ′,Z c ′) is the corrected camera coordinate, and f is the focal length;

[0022] The least square method is used for fitting to calculate the focal length, image principal point position and distortion parameters under multiple zoom parameters.

[0023] According to a PTZ camera calibration method for high-precision geo-positioning provided by the present invention, step S101 specifically includes:

[0024] Extract n pairs of matching feature points from two images taken by the PTZ camera:

[0025] {p i |p i =(x i ,y i ), i=1, 2, ... n}

[0026] {p′ i |p′ i =(x′ i , y′ i ), i=1, 2, ... n}

[0027] Among them, p i , p′ i Respectively represent the matching feature points in the two images;

[0028] The equation system is constructed by matching feature points in the two images, and then solved by the least squares method to obtain the homography matrix. The expression of the equation system is:

[0029] p′ i =Hp i

[0030] in,

[0031]

[0032] Where, h1, h2, h3, h4, h5, h6, h7, h8, h9 are the parameters to be solved; H is the homography matrix;

[0033] The homography matrix is ​​decomposed to obtain the focal length, image principal point position and distortion parameters under multiple zoom parameters.

[0034] According to a PTZ camera calibration method for high-precision geographic positioning provided by the present invention, a relationship between a zoom parameter and an internal parameter of a PTZ camera is established according to a machine learning model, including:

[0035] The relationship between the zoom parameter and the focal length is calculated by the relationship between the projection size and the physical size of the chessboard under multiple zoom parameters;

[0036] Linear regression, polynomial regression or neural network is used for fitting to obtain the change curve of focal length with zoom parameter;

[0037] Linear regression, polynomial regression or neural network is used for fitting to obtain a curve of the distortion parameter changing with the zoom parameter.

[0038] According to a PTZ camera calibration method for high-precision geo-positioning provided by the present invention, the training of the machine learning model uses the mean square error as the loss function.

[0039] According to a PTZ camera calibration method for high-precision geo-positioning provided by the present invention, step S103 specifically includes:

[0040] Based on Beidou high-precision measurement technology, multiple control points and multiple checkpoints are established on the ground;

[0041] Fix the PTZ camera at multiple positions, and collect the pixel coordinates of the control points in the image under multiple zoom parameters, azimuth angles, and pitch angles;

[0042] According to the azimuth, pitch angle and corrected camera coordinates, the world coordinates are calculated.

[0043]

[0044] Where R is the rotation matrix; T = [T x ,T y ,T z ] T , is the translation vector; (X c ′,Y c ′,Z c ′) is the corrected camera coordinate; (X′, Y′, Z′) is the world coordinate; p is the azimuth angle; t is the pitch angle;

[0045] Combining the changing relationship between zoom parameters and internal parameters, the external parameters of the PTZ camera are calculated;

[0046] The control points are matched with the pixel coordinates in the image using the least squares method to optimize the rotation matrix and zoom parameters. The checkpoints are then used for verification. When the error exceeds the preset threshold, the machine learning model is adjusted to optimize the external parameters and the changing relationship between the zoom parameters and the internal parameters.

[0047] According to a PTZ camera calibration method for high-precision geo-positioning provided by the present invention, step S104 specifically includes:

[0048] The azimuth and pitch angles are obtained through the pan / tilt of the PTZ camera, and the corresponding zoom parameters are obtained using the machine learning model. According to the zoom parameters, azimuth and pitch angles, the external parameters are optimized through the nonlinear optimization method, and then the changing relationship between the zoom parameters and the external parameters is fitted.

[0049] In a second aspect, the present invention further provides a PTZ camera calibration device for high-precision geographic positioning, comprising:

[0050] An acquisition unit, used to acquire internal parameters of the PTZ camera under multiple zoom parameters;

[0051] A first calibration unit is used to train a preset machine learning model using the internal parameters of the PTZ camera under multiple zoom parameters as training data, and after the training is completed, establish a change relationship between the zoom parameters of the PTZ camera and the internal parameters according to the machine learning model;

[0052] A calculation verification unit, used to establish multiple control points and multiple checkpoints on the ground, calculate the external parameters of the PTZ camera based on the multiple control points, multiple checkpoints and the changing relationship between the zoom parameters and the internal parameters, and verify the calibration accuracy and / or adjust the machine learning model;

[0053] The second calibration unit is used to obtain the relationship between the zoom parameters of the PTZ camera and the external parameters based on the pan / tilt of the PTZ camera and the machine learning model.

[0054] The technical solution of the present invention has at least the following technical effects:

[0055] The present invention provides a PTZ camera calibration method and device for high-precision geographic positioning, the method comprising: step S101, obtaining the internal parameters of the PTZ camera under multiple zoom parameters; step S102, using the internal parameters of the PTZ camera under multiple zoom parameters as training data to train a preset machine learning model, after the training is completed, establishing the change relationship between the zoom parameters and the internal parameters of the PTZ camera according to the machine learning model; step S103, establishing multiple control points and multiple checkpoints on the ground, based on the multiple control points, multiple checkpoints and the change relationship between the zoom parameters and the internal parameters, calculating the external parameters of the PTZ camera, and verifying the calibration accuracy and / or adjusting the machine learning model; step S104, based on the pan / tilt of the PTZ camera and the machine learning model, obtaining the change relationship between the zoom parameters and the external parameters of the PTZ camera. The present invention can accurately obtain the internal and external parameters of the PTZ camera under multiple zoom parameters, and at the same time simplifies the calibration process, and improves the efficiency and accuracy of the calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] In the attached picture:

[0058] Figure 1 It is a flow chart of the PTZ camera calibration method for high-precision geo-positioning of the present invention;

[0059] Figure 2 This is a schematic diagram of the internal reference calibration using a chessboard in the present invention;

[0060] Figure 3 A schematic diagram of calculating a homography matrix according to a printed chessboard calibration diagram and a photograph taken by the present invention;

[0061] Figure 4 A flow chart of the internal calibration of the PTZ camera for the machine learning model of the present invention;

[0062] Figure 5 Schematic diagram of external parameter calibration of the PTZ camera of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Some embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0065] See also Figure 1 The embodiment of the present invention provides a PTZ camera calibration method for high-precision geographic positioning, including:

[0066] Step S101, obtaining internal parameters of a PTZ camera under multiple zoom parameters;

[0067] Specifically, the PTZ camera can be calibrated using a camera checkerboard calibration method or a camera self-calibration method to obtain the internal parameters of the PTZ camera under multiple zoom parameters;

[0068] Among them, the specific process of calibrating the PTZ camera using the camera chessboard calibration method is as follows:

[0069] A chessboard with a physical size of D is set in the laboratory. Each corner point of the chessboard has a known world coordinate (X W , Y W , Z W ). By controlling the zoom parameters of the PTZ camera, from the minimum zoom Z min To maximum zoom Z max Take multiple photos of the chessboard. In each photo, extract the pixel coordinates (u, v) of each corner point of the chessboard. Figure 2 It is a schematic diagram of intrinsic calibration using a chessboard.

[0070] First, the relationship between the pixel coordinates of the corner points and the image coordinates is:

[0071]

[0072] Among them, (u0, v0) is the pixel coordinate of the principal point, dx and dy are the length and width of the unit pixel respectively, (x, y) is the image coordinate, and (u, v) is the pixel coordinate.

[0073] Then, the PTZ camera is subjected to distortion correction to obtain the corrected image coordinates. For a given image coordinate (x, y), the corrected image coordinate (x', y') includes two parts: radial distortion and tangential distortion.

[0074]

[0075] The radial distortion (x dr ,y dr ) and tangential distortion (x dt ,y dt ) can be calculated by the following formula:

[0076]

[0077] Where k1, k2, k3 and p1, p2 are the distortion parameters of radial distortion and tangential distortion respectively.

[0078] Then according to the camera coordinates (X c , Y c , Z c ) and the corrected image coordinates (x′, y′), and the corrected camera coordinates (X′ c , Y′ c , Z′ c )for:

[0079]

[0080] Finally, the least square method is used to fit all calibration parameters, and the focal length f, the image principal point position (i.e., the pixel coordinates of the principal point (u o , v0)) and distortion parameters. Among them, the objective function estimated by the least squares method is the reprojection error between the pixel coordinates of the reference point and the estimated pixel coordinates. The reprojection error can be calculated by the following formula:

[0081] e=∑((uu′) 2 +(vv′) 2 )

[0082] Where e is the reprojection error, (u, v) is the pixel coordinate of the reference point, and (u′, v′) is the estimated pixel coordinate.

[0083] The checkerboard method has high accuracy, but has high requirements on the environment. Therefore, in actual scenarios, in order to reduce environmental restrictions, the present invention further introduces self-calibration technology and uses a camera self-calibration method to calibrate the PTZ camera, that is, to perform internal parameter calibration by extracting and matching natural feature points in the image. Through the feature points extracted from the image and their matching relationships, the camera internal parameter calibration is performed at multiple focal lengths. Feature extraction and matching uses feature point algorithms based on SIFT, ORB, etc. to automatically extract matching points from multiple images taken by the PTZ camera. Using these matching points, the camera's internal parameters are determined by calculating the homography matrix H and combining it with geometric constraints.

[0084] For images taken under multiple zoom parameters, n pairs of matching feature points are extracted from two of the images:

[0085] {p i | p i =(x i ,y i ), i=1,2,…n}

[0086] {p′ i |p′ i =(x′ i , y′ i ), i=1, 2, ... n}

[0087] Among them, p i , p′ i Respectively represent the matching feature points in the two images;

[0088] Use the matching feature points (p i , p′ i), calculate the homography matrix H, and determine the internal parameters of the PTZ camera according to the geometric constraints of multiple perspectives. Specifically, the equation group is constructed by matching feature points in the two images, and then solved by the least squares method to obtain the homography matrix. The expression of the equation group is:

[0089] p′ i =Hp i

[0090] Where H is a 3×3 homography matrix.

[0091]

[0092] In the formula, h1, h2, h3, h4, h5, h6, h7, h8, and h9 are parameters to be solved.

[0093] The homography matrix is ​​decomposed to obtain internal parameters such as focal length, image principal point position and distortion parameters under multiple zoom parameters. Figure 3 The diagram shows how to calculate the homography matrix H based on the printed chessboard calibration diagram and the taken photos. Figure 3 , five pairs of mutually matching feature points p1 to p5 and p′1 to p′5 are shown.

[0094] Step S102: using the camera internal parameters of the PTZ camera under multiple zoom parameters as training data to train a preset machine learning model; after the training is completed, establishing a relationship between the zoom parameters and the internal parameters of the PTZ camera according to the machine learning model;

[0095] In some embodiments, step S102 trains a machine learning model based on the internal parameters of the PTZ camera under multiple zoom parameters obtained in step S101 as training data. Specifically, it can be: analyzing the relationship between the focal length f of the reference point and the projection size d and fitting the relationship between the zoom parameters and the focal length f and the distortion parameters k1, k2, k3, p1, p2 according to the machine learning model.

[0096] The relationship between the zoom parameter Z and the focal length f is calculated by the relationship between the projection size d and the physical size D of the chessboard under multiple zoom parameters. Furthermore, linear regression, polynomial regression or neural network fitting is used to obtain a curve of focal length changing with zoom parameters; for example, a neural network is used for fitting, and the functional relationship of the change curve is:

[0097] f(Z)=NN(Z)

[0098] Among them, NN is a neural network function, f(Z) is the focal length output by the neural network function, and Z is the input independent variable zoom parameter.

[0099] The variation relationship of the distortion parameters can also be fitted in a similar manner, that is, linear regression, polynomial regression or neural network fitting is used to obtain the variation curve of the distortion parameters with the zoom parameters. For example:

[0100] k i (Z) = ω i0 +ω i1 Z

[0101] Among them, k i (Z) is the i-th distortion parameter under the zoom parameter Z, ω i0 ,ω i1 is the fitting parameter.

[0102] When training the machine learning model, the mean square error (MSE) is used as the loss function to optimize the model parameters so that the predicted internal parameters match the actual measured values ​​as much as possible. After the training is completed, the machine learning model can predict the internal parameters in real time based on the input zoom parameter Z, thereby dynamically adjusting the camera's internal parameters. Figure 4 The specific steps of the machine learning model in the internal calibration of the PTZ camera are shown:

[0103] 1. Data collection: First, collect the camera's internal parameters under multiple zoom parameters Z.

[0104] 2. Regression fitting of focal length and zoom parameter: Fit the relationship between focal length f and zoom parameter Z.

[0105] 3. Fitting of distortion parameters and zoom parameters. Fit the distortion parameters k1, k2, k3, p1, p2 and the zoom parameter Z.

[0106] 4. Train the machine learning model. Use the collected data to train the neural network model.

[0107] 5. Optimize model prediction parameters. Optimize the model and output the predicted value of the internal parameter.

[0108] 6. Output calibration results and optimization parameters. Get the final calibration results and optimized internal parameters.

[0109] Step S103, establishing a plurality of control points and a plurality of check points on the ground, calculating the external parameters of the PTZ camera based on the plurality of control points, the plurality of check points and the changing relationship between the zoom parameters and the internal parameters, and verifying the calibration accuracy and / or adjusting the machine learning model;

[0110] Specifically, step S103 can be combined with Beidou high-precision measurement technology to establish ground control points and check points for verifying and optimizing the external parameters of the camera, especially for use in actual scenarios.

[0111] The present invention is based on high-precision measurement technology to establish multiple control points, each control point CPi Contains its precise latitude, longitude and elevation (L i , B i , H i ). These control points are used to calibrate the camera’s external parameters, and several checkpoints are set to verify the calibration accuracy.

[0112] Fix the PTZ camera at different positions and collect control points CP under multiple zoom parameters Z and multiple azimuth angles p and pitch angles t. i The pixel coordinates in the image (u i , v i ). At the same time, according to the azimuth angle p, pitch angle t, and the corrected camera coordinates (X′ c , Y′ c , Z′ c ), the world coordinates (X′, Y′, Z′) can be calculated as shown in the following formula:

[0113]

[0114]

[0115] Where R is the rotation matrix, T = [T x , T y , T Z ] T is the translation vector. The rotation matrix can be obtained based on the angle values ​​p and t corresponding to the azimuth and pitch angle parameters of the camera gimbal.

[0116] Furthermore, the external parameters of the camera are calculated based on the above data combined with the changing relationship between the zoom parameter Z and the internal parameters obtained by fitting the machine learning model in step S102.

[0117] Furthermore, the least square method is used to transform the actual measured control points (L i , B i , H i ) and the pixel coordinates in the image (u i , v i ) to match and optimize the rotation matrix R and zoom parameter Z. Verify through checkpoints. When the error exceeds the preset threshold, adjust the machine learning model to optimize the external parameters and the relationship between the zoom parameters and the internal parameters.

[0118] Step S104: Based on the pan / tilt of the PTZ camera and the machine learning model, the relationship between the zoom parameters of the PTZ camera and the external parameters is obtained.

[0119] Specifically, in this step, the present invention further optimizes the zoom parameters based on the rotation information of the camera gimbal (including the azimuth angle p, the pitch angle t and the zoom parameter Z) and the results of the previous machine learning model to achieve a higher calibration accuracy. Figure 5 , Figure 5 The figure shows the schematic diagram of the external parameter calibration of the PTZ camera. Based on the image processing technology, Figure 5 The camera coordinate system (O c -X c Y c Z c ), pixel coordinate system (uv), image coordinate system (xy), world coordinate system (O w -X w Y w Z w ) are calibrated.

[0120] The acquisition of PTZ external parameters can obtain the rotation information of the PTZ camera in real time through the equipped sensors, including the azimuth angle p and pitch angle t. Using these extrinsic parameter data of the PTZ, the zoom parameters of the camera can be accurately predicted through the machine learning model, and the external parameters can be optimized.

[0121] Combining the zoom parameter Z obtained by machine learning model fitting and the azimuth angle p and pitch angle t information of the PTZ, the extrinsic parameters are further optimized through nonlinear optimization methods (such as the Levenberg-Marquardt algorithm), making the extrinsic parameters and zoom parameters of the PTZ camera more accurate in multiple zoom states. The optimized extrinsic parameters can be used to accurately reproject control points and checkpoints and verify their accuracy.

[0122] Furthermore, according to multiple zoom parameters Z, a curve of the camera's external parameters (such as azimuth angle and pitch angle) changing with the zoom parameters is further fitted. This curve can describe the dynamic change of the camera's external parameters during the zoom process. The zoom curve obtained by fitting can help the system better grasp the imaging characteristics of the PTZ camera and improve the calibration accuracy in dynamic scenes in practical applications.

[0123] Based on this, the present invention provides an effective PTZ camera internal and external parameter calibration method, which can accurately obtain the camera's imaging parameters under multiple zoom parameters. By combining Beidou high-precision measurement technology and laboratory chessboard calibration, the present invention can significantly improve the imaging accuracy of PTZ cameras in field applications and the accuracy of ground control point positioning.

[0124] Based on the same inventive concept, another embodiment of the present invention provides a PTZ camera calibration device for high-precision geographic positioning, which corresponds to the method of the above embodiment, and includes:

[0125] An acquisition unit, used to acquire internal parameters of the PTZ camera under multiple zoom parameters;

[0126] A first calibration unit is used to train a preset machine learning model using the internal parameters of the PTZ camera under multiple zoom parameters as training data, and after the training is completed, establish a change relationship between the zoom parameters of the PTZ camera and the internal parameters according to the machine learning model;

[0127] A calculation verification unit, used to establish a plurality of control points and a plurality of checkpoints on the ground, calculate the external parameters of the PTZ camera based on the plurality of control points, the plurality of checkpoints and the change relationship between the zoom parameter and the internal parameter, and verify the calibration accuracy and / or adjust the machine learning model;

[0128] The second calibration unit is used to obtain the relationship between the zoom parameter and the external parameter of the PTZ camera based on the pan / tilt of the PTZ camera and the machine learning model.

[0129] In summary, the main technical features and innovations of the present invention are as follows:

[0130] 1. Calibration of PTZ camera internal parameters. The present invention adopts a laboratory calibration method based on a chessboard to achieve accurate calibration of the internal parameters of the PTZ camera (including focal length f, image principal point position) and distortion parameters (including radial distortion parameters k1, k2, k3 and tangential distortion parameters p1, p2). min to Z max ) and use the least squares adjustment method to process the data, and establish the functional relationship between the above parameters and the zoom parameter Z. The relationship between the focal length f and the zoom parameter Z is described by a polynomial function, and the position of the image principal point is assumed to be unchanged by the zoom parameter Z. All distortion parameters are linearly related to the zoom parameter Z, that is, each distortion parameter is controlled by two parameters: gain and bias.

[0131] 2. Calibration of external parameters of PTZ camera. The present invention adopts the method of collecting ground control points in the field to perform external parameter calibration. Beidou high-precision measurement technology is used to obtain the precise longitude, latitude, and altitude (BLH value) of the control points, and these points are divided into control points and check points. After fixing the position of the camera, the ground control points are observed multiple times under different zoom parameters Z to collect the uv coordinates of the corresponding picture and the corresponding p, t, and Z values. Combined with the BLH coordinates of the control points, the least squares adjustment method is used to solve the external parameters of the camera, that is, the p value when the lens points to due north and the t value when the horizon is at the vertical center of the lens. The positioning accuracy of the ground control points is verified by the checkpoints. If the error exceeds the preset threshold, the focal length of the camera is considered as a parameter to be solved in the external parameter calibration, and the least squares calibration is performed together with the p and t values.

[0132] Through the above technical solution, the present invention can realize high-precision geographic positioning of PTZ cameras, simplify the calibration process, and improve the efficiency and accuracy of calibration. The present invention is not only suitable for the calibration of a single PTZ camera, but can also be extended to a network of multiple PTZ cameras to realize large-scale geographic positioning applications, and has broad application prospects and commercial value.

[0133] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the embodiments disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A PTZ camera calibration method for high-precision geolocation, characterized in that: include: Step S101, obtaining internal parameters of a PTZ camera under multiple zoom parameters; Step S102: using the internal parameters of the PTZ camera under multiple zoom parameters as training data to train a preset machine learning model, and after the training is completed, establishing a change relationship between the zoom parameters of the PTZ camera and the internal parameters according to the machine learning model; Step S103, establishing a plurality of control points and a plurality of check points on the ground, calculating the external parameters of the PTZ camera based on the plurality of control points, the plurality of check points and the changing relationship between the zoom parameters and the internal parameters, and verifying the calibration accuracy and / or adjusting the machine learning model; Step S104: Based on the pan / tilt of the PTZ camera and the machine learning model, a relationship between a zoom parameter of the PTZ camera and an external parameter is obtained.

2. The PTZ camera calibration method for high-precision geographic positioning according to claim 1, characterized in that: The step S101 specifically includes: A checkerboard is provided, and a plurality of pictures are taken by the PTZ camera under a plurality of zoom parameters; Extract the pixel coordinates of the corner points of the chessboard in each photo. The relationship between the pixel coordinates of the corner points and the image coordinates is: Where (u0, v0) is the pixel coordinate of the principal point, dx and dy are the length and width of the unit pixel, (x, y) is the image coordinate, and (u, v) is the pixel coordinate; The PTZ camera is subjected to distortion correction to obtain corrected image coordinates. The expression of the corrected image coordinates is: Where (x′, y′) is the corrected image coordinate, (x dr ,y ar ) is radial distortion, (x dt ,y at ) is the tangential distortion, k1, k2, k3 and p1, p2 are the distortion parameters of radial distortion and tangential distortion respectively; According to the camera coordinates of the PTZ camera and the corrected image coordinates, the corrected camera coordinates are obtained, and the expression of the corrected camera coordinates is: In the formula, (X c , Y c , Z c ) is the camera coordinate, (X′ c , Y′ c , Z′ c ) is the corrected camera coordinate, f is the focal length; The least square method is used for fitting to calculate the focal length, image principal point position and distortion parameters under multiple zoom parameters.

3. The PTZ camera calibration method for high-precision geographic positioning according to claim 1, characterized in that: The step S101 specifically includes: Extract n pairs of mutually matching feature points from the two images taken by the PTZ camera: {p i | p i =(x i ,y i ),i=1,2,…n} {p′ i |p′ i =(x′ i ,y′ i ),i=1,2,…n} Among them, p i , p′ i Respectively represent the matching feature points in the two images; The equation group is constructed by matching feature points in the two images, and then solved by the least square method to obtain the homography matrix. The expression of the equation group is: p′ i =Hp i in, Where, h1, h2, h3, h4, h5, h6, h7, h8, h9 are the parameters to be solved; H is the homography matrix; The homography matrix is ​​decomposed to obtain focal lengths, image principal point positions and distortion parameters under multiple zoom parameters.

4. The PTZ camera calibration method for high-precision geographic positioning according to claim 2, characterized in that: The step of establishing a relationship between a zoom parameter of a PTZ camera and an internal parameter according to the machine learning model includes: The relationship between the zoom parameter and the focal length is calculated by the relationship between the projection size and the physical size of the chessboard under multiple zoom parameters; Linear regression, polynomial regression or neural network is used for fitting to obtain the change curve of focal length with zoom parameter; Linear regression, polynomial regression or neural network is used for fitting to obtain a curve of the distortion parameter changing with the zoom parameter.

5. The PTZ camera calibration method for high-precision geographic positioning according to claim 1, characterized in that: The training of the machine learning model uses mean square error as the loss function.

6. The PTZ camera calibration method for high-precision geographic positioning according to claim 2, characterized in that: The step S103 specifically includes: Based on Beidou high-precision measurement technology, multiple control points and multiple checkpoints are established on the ground; The PTZ camera is fixed at multiple positions, and the pixel coordinates of the control points in the image are collected under multiple zoom parameters and azimuth and pitch angles; According to the azimuth, pitch angle and corrected camera coordinates, the world coordinates are calculated. Where R is the rotation matrix; T = [T x , T y , T Z ] T , is the translation vector; (X′ c , Y′ c , Z′ c ) is the calibrated camera coordinate; (X′, Y′, Z′) is the world coordinate; p is the azimuth angle; t is the pitch angle; Calculate the external parameters of the PTZ camera by combining the change relationship between the zoom parameter and the internal parameter; The control points are matched with the pixel coordinates in the image using the least squares method to optimize the rotation matrix and zoom parameters; and the checkpoints are used for verification. When the error exceeds a preset threshold, the machine learning model is adjusted to optimize the external parameters and the changing relationship between the zoom parameters and the internal parameters.

7. The PTZ camera calibration method for high-precision geographic positioning according to claim 1, characterized in that: The step S104 specifically includes: The azimuth and pitch angles are obtained through the pan / tilt of the PTZ camera, and the corresponding zoom parameters are obtained using the machine learning model. According to the zoom parameters, azimuth and pitch angles, the external parameters are optimized by a nonlinear optimization method, and then the changing relationship between the zoom parameters and the external parameters is fitted.

8. A PTZ camera calibration device for high-precision geographic positioning, characterized in that: include: An acquisition unit, used to acquire internal parameters of the PTZ camera under multiple zoom parameters; A first calibration unit is used to train a preset machine learning model using the internal parameters of the PTZ camera under multiple zoom parameters as training data, and after the training is completed, establish a change relationship between the zoom parameters of the PTZ camera and the internal parameters according to the machine learning model; A calculation verification unit, used to establish a plurality of control points and a plurality of checkpoints on the ground, calculate the external parameters of the PTZ camera based on the plurality of control points, the plurality of checkpoints and the change relationship between the zoom parameter and the internal parameter, and verify the calibration accuracy and / or adjust the machine learning model; The second calibration unit is used to obtain the relationship between the zoom parameter and the external parameter of the PTZ camera based on the pan / tilt of the PTZ camera and the machine learning model.

Images