Monitoring camera visual axis course measurement method and device

Abstract

The invention belongs to the technical field of monitoring, and particularly relates to a monitoring camera visual axis course measurement method and device, and the method comprises the steps: collecting a plurality of original angle data when a camera is aligned with a target point, collecting a plurality of original coordinate data of the target point in a monitoring scene, and carrying out the time synchronization processing; validity verification is carried out on the original coordinate data, coordinate system conversion and data precision verification are carried out, and converted coordinate data are obtained; calculating theoretical angle data based on the converted coordinate data, calculating an angle deviation between the theoretical angle data and the original angle data, and constructing an error function; and performing joint optimization on the pose parameter of the camera and the cradle head deviation parameter through a nonlinear optimization algorithm based on the error function to obtain an initial calibration result, calculating the calibration precision and confidence interval of the initial calibration result, verifying the validity and rationality of the initial calibration result, and outputting a target calibration result based on the verification result. According to the invention, the precision and reliability of monitoring camera visual axis course measurement are improved.

Description

Technical Field

[0001] This application belongs to the field of surveillance technology, and in particular relates to a method and device for measuring the line-of-sight heading of a surveillance camera. Background Technology

[0002] In the field of surveillance technology, the accuracy of a camera's line-of-sight heading directly determines the accuracy of the monitoring coverage and the reliability of target tracking. Whether it's urban security monitoring, traffic condition monitoring, or park perimeter protection, it's essential to accurately measure the camera's line-of-sight heading to ensure that the camera can precisely aim at the monitored target, avoid blind spots, and achieve collaborative operation between multiple cameras.

[0003] However, traditional calibration plate-based measurement methods rely on calibration plates with known geometric dimensions as a reference. Limited by the physical size and arrangement distance of the calibration plates, the accuracy drops sharply when the measurement distance exceeds 50 meters, and the error can reach the meter level above 500 meters. Although the SFM (Structure from Motion) method based on natural features does not require a calibration plate, it suffers from scale uncertainty, cannot obtain the absolute line-of-sight heading, and the cumulative error amplifies with the increase of the measurement distance. The accuracy of multi-sensor fusion methods (combining IMU, LiDAR, etc.) is limited by the pre-calibrated errors between sensors, and the synchronization requirements are even higher when used at long distances, further amplifying the errors.

[0004] Currently, there is an urgent need for a method and device that can achieve high-precision line-of-sight heading measurement in long-distance, large-scale scenarios, in order to solve the problems of limited accuracy, scale uncertainty, error accumulation and difficulty in synchronizing multiple sensors in traditional measurement techniques. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method and apparatus for measuring the line-of-sight heading of a surveillance camera. It employs a collaborative data acquisition mode based on a gimbal angle sensor and an RTK-GPS receiver. Through multi-dimensional data verification and coordinate system transformation technology, it can achieve millimeter-level accuracy in line-of-sight heading measurements in long-distance scenarios, thereby improving the accuracy and reliability of line-of-sight heading measurements for surveillance cameras.

[0006] Firstly, this application provides a method for measuring the line-of-sight heading of a surveillance camera, the method comprising: The camera acquires several raw angle data when it is pointing at the target point using a PTZ angle sensor, and acquires several raw coordinate data of the target point in the monitoring scene using an RTK-GPS receiver. The camera then performs time synchronization processing on the raw angle data and the corresponding raw coordinate data. The validity of several original coordinate data is verified to obtain valid coordinate data. The valid coordinate data is then transformed into a coordinate system, and the accuracy of the transformed data is verified. Based on the verification results, the transformed coordinate data is obtained. Theoretical angle data is calculated based on the transformed coordinate data. The angle deviation between the theoretical angle data and the original angle data is calculated. An error function is constructed based on the angle deviation. Based on the error function, the camera pose parameters and gimbal deviation parameters are jointly optimized using a nonlinear optimization algorithm to obtain the initial calibration result. The calibration accuracy and confidence interval of the initial calibration result are calculated, and the validity and rationality of the initial calibration result are verified. Based on the verification result, the target calibration result is output.

[0007] Furthermore, The validity of several of the original coordinate data is validated, specifically including: Verify whether the original latitude and longitude values ​​in the original coordinate data are within the preset valid range of latitude and longitude; Verify whether the original altitude values ​​in the original coordinate data are within the preset valid altitude range; Verify whether the original angle data is within its physical range; as well as, Outliers in the original latitude and longitude values ​​and the original altitude values ​​are identified and removed using statistical methods.

[0008] Furthermore, Coordinate system transformation is performed on the valid coordinate data, specifically including: The effective coordinate data of the target point is converted from WGS84 geographic coordinates to coordinate data in the ECEF geocentric and geofixed coordinate system to obtain the first coordinate data. Based on the approximate location of the camera, the first coordinate data is converted from the ECEF geocentric-ground-fixed coordinate system to the ENU local coordinate system with the camera's optical center as the origin, thus obtaining the transformed coordinate data.

[0009] Furthermore, Based on the verification results, the transformed coordinate data is obtained, specifically including: For any set of transformed coordinate data, the inverse coordinate transformation is performed to obtain the inverse transformed coordinates. The difference between the inverse transformed coordinates and the effective coordinate data is calculated to obtain the first precision deviation. For the valid coordinate data of the same target point, the coordinate system transformation process is repeated a preset number of times, and the coordinate deviation between the multiple transformation results is calculated to obtain the second precision deviation; The transformation coordinate data is determined by comparing the first and second accuracy deviations with a preset accuracy deviation threshold.

[0010] Furthermore, Theoretical angle data is calculated based on transformed coordinate data, specifically including: Calculate the theoretical azimuth angle based on the transformed coordinate data and the azimuth angle calculation formula; The theoretical pitch angle is calculated based on the transformed coordinate data and the pitch angle calculation formula.

[0011] Furthermore, An error function is constructed based on the angle deviation, specifically including: The theoretical azimuth and theoretical pitch angles are mapped to theoretical gimbal angles using a calibration model that includes azimuth deviation, pitch deviation, reference azimuth, and reference pitch angle. Calculate the residual between the theoretical gimbal angle and the raw angle data collected by the gimbal angle sensor; An error function is constructed based on the residuals, and the Huber loss function is used to process the residuals to suppress the influence of outliers.

[0012] Furthermore, Based on the error function, a nonlinear optimization algorithm is used to jointly optimize the camera's pose parameters and gimbal deviation parameters, specifically including: Construct an optimization problem with the objective of minimizing the sum of squared angular residuals at all observation points; Configure the Trust Region optimization strategy using the Ceres Solver library and select the Doglg method as the iterative solver; Set the maximum number of iterations, function value tolerance, gradient tolerance, and parameter change tolerance as convergence criteria; The iterative optimization process begins with initial parameter values ​​set based on the statistical characteristics of the observed data, and continues until the convergence condition is met or the maximum number of iterations is reached. The optimized parameter vector is then output as the initial calibration result.

[0013] Furthermore, Calculate the calibration accuracy and confidence interval of the initial calibration results, specifically including: After optimization convergence, the final residual vector of the optimization problem is extracted, and its root mean square error is calculated as an evaluation index of the overall calibration accuracy. Calculate the covariance matrix estimate of the optimized parameter vector, and derive the standard error of each calibration parameter based on this estimate; Based on the standard error and the selected confidence level, calculate the confidence interval for each calibration parameter.

[0014] Furthermore, Verify the validity and reasonableness of the initial calibration results, specifically including: Check whether all optimized calibration parameters are within their preset reasonable physical range; Analyze the statistical distribution characteristics of the final residuals to determine whether there are systematic biases or non-convergent observations; Compare the calibration accuracy evaluation index with the preset application accuracy requirements to verify whether the results meet the usage needs. Based on the above judgment and verification results, it is determined whether the initial calibration result will be used as the target calibration result or whether a recalibration process will be triggered.

[0015] Secondly, based on the same inventive concept, this application provides a surveillance camera line-of-sight heading measurement device, the device comprising: The data acquisition module is used to acquire several raw angle data when the camera is aimed at the target point based on the pan-tilt angle sensor, acquire several raw coordinate data of the target point in the monitoring scene based on the RTK-GPS receiver, and perform time synchronization processing on the several raw angle data and the corresponding several raw coordinate data. The conversion module is used to verify the validity of several original coordinate data, obtain valid coordinate data, perform coordinate system transformation on the valid coordinate data, verify the accuracy of the transformed data, and obtain the transformed coordinate data based on the verification results. The calculation module is used to calculate theoretical angle data based on the transformed coordinate data, calculate the angle deviation between the theoretical angle data and the original angle data, and construct an error function based on the angle deviation. The result output module is used to jointly optimize the camera pose parameters and gimbal deviation parameters based on the error function using a nonlinear optimization algorithm to obtain the initial calibration result, calculate the calibration accuracy and confidence interval of the initial calibration result, verify the validity and rationality of the initial calibration result, and output the target calibration result based on the verification result.

[0016] Compared with the prior art, this application has the following advantages: 1. This application overcomes the limitation of the traditional calibration board-based measurement method, which suffers from a sharp drop in accuracy at long distances, by combining a gimbal angle sensor with an RTK-GPS receiver. It enables high-precision line-of-sight heading measurement in long-distance, wide-area scenarios, effectively solving the problem of limited accuracy in traditional measurement techniques.

[0017] 2. This application performs validity verification and coordinate system transformation on the original coordinate data and verifies the data accuracy, avoiding the scale uncertainty problem existing in the SFM method based on natural features, and can obtain accurate absolute line-of-sight heading, reducing the situation where the cumulative error is amplified as the measurement distance increases.

[0018] 3. This application uses a nonlinear optimization algorithm based on the error function to jointly optimize the camera pose parameters and gimbal deviation parameters. Compared with multi-sensor fusion methods, it does not rely on the pre-calibrated accuracy between sensors, reduces the problem of further error superposition caused by high synchronization requirements in long-distance applications, and improves the accuracy and reliability of measurement.

[0019] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for measuring the line-of-sight heading of a surveillance camera according to an embodiment of this application is shown. Figure 2 A structural block diagram of a surveillance camera line-of-sight heading measurement device according to an embodiment of this application is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Figure 1 A flowchart illustrating a method for measuring the line-of-sight heading of a surveillance camera according to an embodiment of this application is shown, as follows: Figure 1 As shown in the figure, an embodiment of this application provides a method for measuring the line-of-sight heading of a surveillance camera, including: S1: Based on the PTZ angle sensor, collect several raw angle data when the camera is aimed at the target point. Based on the RTK-GPS receiver, collect several raw coordinate data of the target point in the monitoring scene. Then, perform time synchronization processing on the several raw angle data and the corresponding several raw coordinate data. In this embodiment, the time synchronization accuracy is better than 10 milliseconds.

[0024] S2, perform validity verification on several original coordinate data to obtain valid coordinate data, perform coordinate system transformation on the valid coordinate data, verify the accuracy of the transformed data, and obtain transformed coordinate data based on the verification results; In this embodiment of the application, step S2 specifically includes: S21, verify whether the original latitude and longitude values ​​in the original coordinate data are within the preset valid range of latitude and longitude; S22, verify whether the original altitude value in the original coordinate data is within the preset effective altitude range; S23, verify whether the original angle data is within its physical range; S24, and, Outliers in the original latitude and longitude values ​​and the original altitude values ​​are identified and removed using statistical methods.

[0025] In this embodiment of the application, the preset effective range of latitude and longitude is based on the WGS84 geographic coordinate system, the physical range of Earth's longitude is [-180°~+180°], and the physical range of latitude is [-90°~+90°]. The physical range of the gimbal angle sensor is as follows: the physical range of the azimuth angle is the range of horizontal rotation of the gimbal, usually [0°~360°], and the physical range of the pitch angle is the range of vertical rotation of the gimbal, usually [-90°~+90°]. The preset effective altitude range needs to be preset in conjunction with the actual physical environment of the monitoring scenario, such as [ 100 meters, +10000 meters].

[0026] In this embodiment of the application, the 3σ criterion is commonly used to identify and remove outliers in the original latitude and longitude values ​​and the original altitude values ​​based on statistical methods (assuming that the data follows a normal distribution, and about 99.7% of the data falls within the range of "mean μ ± 3 times standard deviation σ", and those exceeding this range are judged as outliers).

[0027] For the datasets of latitude, longitude, and altitude, calculate the mean μ and standard deviation σ respectively. If the value of any data point is b, then |b| satisfies the following condition. If μ∣>3σ, it is considered an outlier and removed.

[0028] In this embodiment of the application, step S2 further includes: S25, convert the effective coordinate data of the target point from WGS84 geographic coordinates to coordinate data in the ECEF geocentric and geofixed coordinate system to obtain the first coordinate data; S26, based on the approximate position of the camera, the first coordinate data is converted from the ECEF geocentric-ground-fixed coordinate system to the ENU local coordinate system with the camera's optical center as the origin, and the transformed coordinate data is obtained.

[0029] In this embodiment of the application, WGS84 (World Geodetic System 1984) geographic coordinates are represented by "longitude λ, latitude φ, altitude h", which need to be converted to ECEF (Earth-Centered, Earth-Fixed) rectangular coordinates (x, y, z), where the origin O (0, 0, 0) is the Earth's center of mass, the z-axis is parallel to the Earth's axis and points to the North Pole, the x-axis points to the intersection of the Prime Meridian and the equator, and the y-axis is perpendicular to the xOz plane (i.e., the intersection of 90 degrees east longitude and the equator), forming a right-handed coordinate system.

[0030] In this embodiment, the Earth ellipsoid parameters are used: semi-major axis a = 6378137.0 meters (major axis radius of the Earth ellipsoid). First eccentricity square e 2 =0.00669438 (a parameter describing the "flattening" of an ellipsoid).

[0031] In this embodiment of the application, the WGS84 to ECEF conversion specifically includes:

[0032] x=(N(φ)+h)cosφcosλ y=(N(φ)+h)cosφsinλ z=(N(φ)(1-e²)+h)sinφ Where: a = 6,378,137.0 meters (major semi-axis), e 2 =0.00669438 (square of the first eccentricity).

[0033] In this embodiment of the application, the ECEF to ENU (East, North, Up coordinate system) transformation is as follows: Let the camera's coordinates under ECEF be (x r y r , z r） (Based on its approximate location (λ) r , φ r h r） (Calculated using the WGS84→ECEF method described above), the target's coordinates under ECEF are (x... i y i , z i） Then the ECEF coordinate difference between the target point and the camera is: x=x i -x r; y=y i -y r; z=z i -z r; Using the approximate location of the camera, latitude and longitude λ r , φ r Construct a rotation matrix to project the ECEF coordinate difference onto the ENU axis. The rotation matrix and transformation formula are as follows:

[0034] Expand and calculate each component: Eastward coordinate E: E= sinλ r ×Δx+cosλ r ×Δy; North coordinate N: N = -sinφ r cosλ r ×Δx-sinφ r sinλ r ×Δy+cosφ r× Δz; Celestial coordinates U: U=cosφ r cosλ r ×Δx+cosφ r sinλ r ×Δy+sinφ r ×Δz.

[0035] In this embodiment of the application, step S2 further includes: S27. For any set of transformed coordinate data, the inverse coordinate transformation is performed to obtain the inverse transformed coordinates. The difference between the inverse transformed coordinates and the effective coordinate data is calculated to obtain the first precision deviation. S28, For the valid coordinate data of the same target point, repeat the coordinate system transformation process a preset number of times, calculate the coordinate deviation between the multiple transformation results, and obtain the second precision deviation; S29, Verification is performed based on the comparison results of the first precision deviation and the second precision deviation with the preset precision deviation threshold to determine the transformed coordinate data.

[0036] In this embodiment of the application, the coordinate inverse transformation is ENU→ECEF→WGS84 inverse transformation, wherein the ENU→ECEF inverse transformation includes: The inverse rotation matrix and its formula are as follows:

[0037] in, x inv =x ecef，inv -x r，ecef ; y=y ecef, inv -y r, ecef; z=z ecef, inv -z r, ecef; In the formula, x r，ecef y r，ecef z r，ece These are the ECEF coordinates corresponding to the approximate position of the camera (given by λ). r , φ r h r (Calculated using the forward WGS84→ECEF formula), x can be solved from the above formula. ecef，inv =x r，ecef + x inv ; y ecef, inv =y r, ecef + y; z ecef, inv =z r, ecef + z; ECEF→WGS84 inverse conversion includes: Calculate the initial latitude φ0: ; Calculate the radius of curvature N0 of the meridian circle based on the initial latitude φ0: ; Correction Dimensions : Repeat the iteration until (Meets centimeter-level accuracy), ultimately yielding φ inv .

[0038] Calculate longitude and altitude: ; , where N final This represents the radius of curvature of the meridian corresponding to the final latitude.

[0039] In this embodiment of the application, the inverse transformation coordinates (λ) are compared. inv , φ inv h inv ) and the original valid coordinate data (λ) org , φ org h org The deviation is calculated according to the following dimensions: Longitude deviation: Δλ = |λ inv λ org |, converted to meters (1° longitude ≈ 111319.9 meters, using the conversion factor corresponding to the latitude of the monitored area). Latitude deviation: Δφ=∣φ inv φ org |, converted to meters (1° latitude ≈ 111319.9 meters, approximately constant globally); Altitude deviation: Δh = |h inv h org | (in meters); Overall first precision deviation: Take the maximum value of the deviations of the above three dimensions, and denote it as Dev1.

[0040] In this embodiment of the application, the effective coordinate data (λ) of the same target point are selected. org , φ org h org ), to ensure that the data has not been tampered with or filtered; Based on the "high-precision calibration" requirement in the appendix, the number of repeated conversions is set to 3 to 5 times (too few times will not reflect stability, and too many times will increase the computational cost). Each repeated transformation uses the same Earth ellipsoid parameters (a, e) 2 ), approximate location of the camera (λr, φ) r h r (This only repeats the process without changing the input parameters.)

[0041] For the same target point's valid coordinate data, repeatedly execute the "WGS84→ECEF→ENU" forward transformation process to obtain multiple sets of transformed coordinate data, denoted as (E1, N1, U...). 1） (E2, N2, U) 2） , ..., (E k N k U k） (k is the preset number of repetitions, such as k=5).

[0042] For the three axes of East (E), North (N), and Sky (U) in the ENU coordinate system, the deviations of multiple transformations are calculated, with the core indicators being: Maximum deviation of each axis: ΔE max =max(E1, E2, ..., E k ) min(E1, E2, ..., E k ), ΔN max =max(N1, ..., N) k) min(N1, ..., N) k ), ΔU max =max(U1, ..., U k ) min(U1, ..., U k ); Overall second precision deviation: Take ΔE max ΔN max ΔU max The maximum value is denoted as Dev2.

[0043] In this embodiment of the application, if Dev1≤Thr1 and Dev2≤Thr2 are satisfied simultaneously, it indicates that the set of transformed coordinate data is "accurate (small deviation in inverse operation)" and "stable (small deviation in repeated transformation)". The set of transformed coordinate data (E) is then processed. conv N conv U conv The data is determined as the "final valid transformed coordinate data" and used for subsequent theoretical angle calculations. The first precision deviation threshold Thr1 = 1 mm, and the second precision deviation threshold Thr2 = 0.5 mm. Otherwise, the data is reprocessed and transformed.

[0044] S3, calculate theoretical angle data based on transformed coordinate data, calculate the angle deviation between theoretical angle data and the original angle data, and construct an error function based on the angle deviation; In this embodiment of the application, step S3 specifically includes: S31, Calculate the theoretical azimuth angle based on the transformed coordinate data and the azimuth angle calculation formula; S32, calculate the theoretical pitch angle based on the transformed coordinate data and the pitch angle calculation formula.

[0045] In this embodiment of the application, the theoretical azimuth angle A z，theory The calculation formula is: A z, theory =arctan2(E, N)×180° / π; The formula for calculating the theoretical pitch angle El, theory is as follows: E l, theory =arctan2(U, )×180° / π.

[0046] In this embodiment of the application, step S3 further includes: S33, using a calibration model that includes azimuth deviation, pitch deviation, reference azimuth, and reference pitch, maps the theoretical azimuth and theoretical pitch angles to theoretical gimbal angles; S34, Calculate the residual between the theoretical gimbal angle and the original angle data collected by the gimbal angle sensor; S35, construct an error function based on the residuals, and use the Huber loss function to process the residuals to suppress the influence of outliers.

[0047] In this embodiment, the original angle is mapped to the theoretical gimbal angle (i.e., the calibrated angle value that can be compared with the observed theoretical angle) using the following formula: Theoretical gimbal azimuth angle: A z，theory =(A z，raw ref az )+d az ; Theoretical gimbal tilt angle: E l，theory =(E l，raw ref el )+d el In the formula, d az To compensate for azimuth deviation, systematic deviations in gimbal azimuth measurement (such as mechanical offset of the gimbal's horizontal rotation axis); d el To compensate for pitch angle deviation, systematic deviations in gimbal pitch angle measurement (such as mechanical offset of the gimbal's vertical rotation axis); ref az For reference azimuth, define the absolute azimuth corresponding to the "0° azimuth" of the gimbal (e.g., the actual north deflection angle pointed to by the 0° scale on the gimbal); ref el For reference pitch angle, define the absolute pitch corresponding to "0° pitch" of the gimbal (such as the actual horizontal deflection angle corresponding to the 0° scale of the gimbal). Ensure that the mapped theoretical gimbal angle conforms to physical logic: Theoretical gimbal azimuth angle A z，theory The range is mapped to [0°, 360°]; Theoretical gimbal pitch angle E l，theory Range mapping to [ [90°, 90°], to avoid angle exceeding limits due to abnormal parameters.

[0048] In this embodiment of the application, the formula for calculating the azimuth residual is as follows: A z，residual =((A z，theory A z，obs +180°) mod 360°) After correction of 180°, the azimuth residual range is constrained to [ [180°, 180°], accurately reflecting the angular difference between the two (e.g., the residual between 350° and 10° is corrected to -20°).

[0049] In this embodiment of the application, the formula for calculating the pitch angle residual is as follows: E l，residual =E l，theory E l，obs The pitch angle residual range is [ [180°, 180°] If the absolute value of the residual is too large (e.g., exceeding 10°), its impact will be suppressed by the Huber loss function.

[0050] In this embodiment, the Huber loss function is substituted into the basic error function to obtain the final robustness error function, as shown in the following formula: Where n is the number of effective target points, A z，residual，i E l，residual，i Let be the azimuth residual and the pitch residual of the i-th target point, respectively; Huber loss function definition (two cases): when the absolute value of the residual |r| ≤ δ (small residual, non-outlier): L(r) = 1 / 2 × r 2 (Maintain secondary loss to ensure accuracy); When the absolute value of the residual |r| > δ (large residual, outlier): L(r) = δ |r| 1 / 2×δ 2 (Switching to linear growth, reducing outlier weights); where r is a single residual (A z，residual，i or E l，residual，i ), δ=1.0 (fixed parameter, verified to effectively filter out up to 20% of abnormal data).

[0051] S4. Based on the error function, the camera pose parameters and gimbal deviation parameters are jointly optimized using a nonlinear optimization algorithm to obtain the initial calibration result. The calibration accuracy and confidence interval of the initial calibration result are calculated, and the validity and rationality of the initial calibration result are verified. Based on the verification result, the target calibration result is output.

[0052] In this embodiment of the application, step S4 specifically includes: S41, construct an optimization problem with the objective of minimizing the sum of squared angular residuals at all observation points; S42, use the Ceres Solver library to configure the Trust Region optimization strategy and select the Doglg method as the iterative solver; S43 sets the maximum number of iterations, function value tolerance, gradient tolerance, and parameter change tolerance as convergence criteria; S44 starts the iterative optimization process from the initial parameter values ​​set based on the statistical characteristics of the observed data until the convergence condition is met or the maximum number of iterations is reached, and outputs the optimized parameter vector as the initial calibration result.

[0053] In this embodiment, the maximum number of iterations serves as a safety net to prevent the algorithm from looping indefinitely. For example, setting it to 10,000 iterations will force a stop even if the algorithm does not fully converge.

[0054] Function value tolerance: If the decrease in total error between two iterations is negligible (less than a certain threshold, such as 1e) -12 This indicates that the goal is essentially beyond improvement and can be stopped.

[0055] Gradient tolerance: The gradient points in the direction of the fastest increase in error. A gradient value close to 0 means that the current point is at a "valley" (extreme point), and there is no longer a direction to reduce the error, so it is time to stop.

[0056] Parameter variation tolerance: If the change in the parameter itself is very small between two iterations (less than a certain threshold), it means that the parameter has basically stabilized and can be stopped.

[0057] In this embodiment of the application, step S4 further includes: S45. After the optimization converges, the final residual vector of the optimization problem is extracted, and its root mean square error is calculated as an evaluation index of the overall calibration accuracy. S46, calculate the covariance matrix estimate of the optimized parameter vector, and derive the standard error of each calibration parameter based on this; S47, Calculate the confidence intervals for each calibration parameter based on the standard error and the selected confidence level.

[0058] In the embodiments of this application, the confidence level is typically chosen to be 95% or 99%. This means that there is a 95% (or 99%) certainty that the true parameter value falls within this range.

[0059] In this embodiment of the application, the confidence interval is calculated as follows: For a given confidence level (e.g., 95%), the corresponding critical value can be found in the t-distribution or normal distribution table (when the degrees of freedom are large, 1.96 is often used to approximate the 95% confidence level).

[0060] The formula for calculating the confidence interval of a certain parameter is: parameter estimate ± (critical value × standard error of the parameter).

[0061] In this embodiment of the application, step S4 further includes: Check whether all optimized calibration parameters are within their preset reasonable physical range; Analyze the statistical distribution characteristics of the final residuals to determine whether there are systematic biases or non-convergent observations; Compare the calibration accuracy evaluation index with the preset application accuracy requirements to verify whether the results meet the usage needs. Based on the above judgment and verification results, it is determined whether the initial calibration result will be used as the target calibration result or whether a recalibration process will be triggered.

[0062] In this embodiment of the application, the reasonable physical range for the camera's longitude is preset as follows: If the value exceeds 180°, it violates the WGS84 standard specification and the parameter is deemed invalid. The reasonable physical range for the camera's latitude preset is [ If the coordinates exceed 90°, the geographic coordinates are invalid, and the parameter is deemed invalid. The reasonable physical range for camera altitude preset is ground height ~ the height of the tallest building in the scene + 5 meters. If it is lower than the ground or much higher than the top of the building (e.g., 1000 meters, in a flat scene), the parameters are considered abnormal. The reasonable physical range for the preset azimuth deviation is [ If the value exceeds 5°, it indicates that the gimbal's mechanical deviation is too large (there may be a malfunction), and the parameter is deemed invalid. The reasonable physical range for the preset pitch angle deviation is [ [5°, 5°], similarly, if it exceeds, the parameter is considered invalid; The reasonable physical range of the reference azimuth angle is [0°, 360°]. If it exceeds this range, it does not conform to the physical description of the azimuth angle, and the parameter is deemed invalid. The reasonable physical range for the preset pitch angle is [ [90°, 90°], exceeding this range does not conform to the physical motion range of the pitch angle, and the parameter is deemed invalid.

[0063] In this embodiment of the application, the azimuth residual and elevation residual are plotted relative to variables such as target ID, azimuth, and elevation.

[0064] If the residuals show a clear trend (e.g., all residuals are positive) or a regular change (e.g., they increase with increasing angle), it indicates that the model has not fully fitted the data and there may be unmodeled error factors (such as lens distortion, gimbal nonlinearity, etc.).

[0065] Identify observations whose residuals significantly exceed the RMSE (e.g., more than 3 times the standard deviation). These points may be non-converged, and their data may contain issues (e.g., sudden changes in GPS signal, targeting the wrong target, etc.).

[0066] In this embodiment, the preset application accuracy requirement is set according to actual business needs.

[0067] Based on the above method, this application also provides a surveillance camera line-of-sight heading measurement device corresponding to the above method. Figure 2 A structural block diagram of a surveillance camera line-of-sight heading measurement device according to an embodiment of this application is shown. See also: Figure 2 As shown, the device includes: The data acquisition module 10 is used to acquire several raw angle data when the camera is aimed at the target point based on the pan-tilt angle sensor, acquire several raw coordinate data of the target point in the monitoring scene based on the RTK-GPS receiver, and perform time synchronization processing on the several raw angle data and the corresponding several raw coordinate data. The conversion module 20 is used to perform validity verification on several original coordinate data, obtain valid coordinate data, perform coordinate system transformation on the valid coordinate data, verify the accuracy of the transformed data, and obtain transformed coordinate data based on the verification results. Calculation module 30 is used to calculate theoretical angle data based on transformed coordinate data, calculate the angle deviation between the theoretical angle data and the original angle data, and construct an error function based on the angle deviation; The result output module 40 is used to jointly optimize the camera pose parameters and gimbal deviation parameters based on the error function using a nonlinear optimization algorithm to obtain the initial calibration result, calculate the calibration accuracy and confidence interval of the initial calibration result, verify the validity and rationality of the initial calibration result, and output the target calibration result based on the verification result.

[0068] Based on the same inventive concept disclosed above, this application also provides an electronic device. The electronic device of this application includes at least one processor and at least one memory electrically connected to the processor. The memory is electrically connected to the processor, wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0069] It should be noted that the electrical connections between the above-mentioned units do not necessarily represent the connections between lines. Indirect connections are applicable to the embodiments of this application as long as they achieve the purpose of this application.

[0070] Based on the same inventive concept, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the above method.

[0071] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for monitoring camera boresight heading measurement, characterized in that, the method comprises, collecting a plurality of original angle data of the camera aiming at a target point based on a gimbal angle sensor, collecting a plurality of original coordinate data of the target point in a monitoring scene based on an RTK-GPS receiver, and performing time synchronization processing on the plurality of original angle data and the corresponding plurality of original coordinate data; performing validity check on the plurality of original coordinate data to obtain valid coordinate data, performing coordinate system conversion on the valid coordinate data, and verifying the data precision of the conversion, obtaining converted coordinate data based on the verification result; calculating theoretical angle data based on the converted coordinate data, calculating the angle deviation of the theoretical angle data and the original angle data, and constructing an error function based on the angle deviation; performing joint optimization on the pose parameters of the camera and the gimbal deviation parameters based on the error function through a nonlinear optimization algorithm to obtain an initial calibration result, calculating the calibration precision and confidence interval of the initial calibration result, verifying the validity and reasonableness of the initial calibration result, and outputting a target calibration result based on the verification result.

2. The method of claim 1, characterized in that, the validity check on the plurality of original coordinate data specifically comprises: checking whether the original latitude and longitude values in the original coordinate data are within a preset latitude and longitude effective range; checking whether the original altitude values in the original coordinate data are within a preset altitude effective range; checking whether the original angle data is within its physical range; and, identifying and eliminating outliers in the original latitude and longitude values and the original altitude values based on a statistical method.

3. The method of claim 2, characterized in that, the coordinate system conversion on the valid coordinate data specifically comprises: converting the valid coordinate data of the target point from WGS84 geographic coordinates to ECEF geocentric coordinates in the ECEF geocentric coordinates system to obtain first coordinate data; based on the approximate position of the camera, converting the first coordinate data from the ECEF geocentric coordinates system to the ENU local coordinates system with the optical center of the camera as the origin to obtain converted coordinate data.

4. The method of claim 3, characterized in that, the converted coordinate data is obtained based on the verification result, specifically comprising: for any group of converted coordinate data, inverse coordinate conversion is performed to obtain inverse converted coordinates, the difference between the inverse converted coordinates and the valid coordinate data is calculated to obtain a first precision deviation; for the valid coordinate data of the same target point, the coordinate system conversion process is repeated for a preset number of times, the coordinate deviation between the multiple conversion results is calculated to obtain a second precision deviation; the converted coordinate data is determined based on the comparison result of the first and second precision deviations with a preset precision deviation threshold.

5. The method of claim 1, characterized in that, the theoretical angle data is calculated based on the converted coordinate data, specifically comprising: calculating the theoretical azimuth angle according to the converted coordinate data and the azimuth angle calculation formula; calculating the theoretical pitch angle according to the converted coordinate data and the pitch angle calculation formula.

6. The method of claim 5, wherein, an error function is constructed based on the angle deviation, specifically comprising: mapping the theoretical azimuth angle and the theoretical pitch angle to a theoretical PTZ angle through a calibration model containing the azimuth angle deviation, the pitch angle deviation, the reference azimuth angle and the reference pitch angle; calculating the residual error between the theoretical PTZ angle and the raw angle data collected by the PTZ angle sensor; constructing an error function based on the residual error and processing the residual error with a Huber loss function to suppress the influence of outliers.

7. The method of claim 6, wherein, the pose parameters of the camera and the PTZ deviation parameters are jointly optimized through a nonlinear optimization algorithm based on the error function, specifically comprising: constructing an optimization problem aiming to minimize the sum of squares of angle residual errors of all observation points; configuring a Trust Region optimization strategy using the Ceres Solver library and selecting a Dogleg method as an iterative solver; setting the maximum number of iterations, the function value tolerance, the gradient tolerance and the parameter variation tolerance as the convergence criteria; starting the iterative optimization process from the initial parameter values set based on the statistical characteristics of the observation data, until the convergence condition is met or the maximum number of iterations is reached, and outputting the optimized parameter vector as the initial calibration result.

8. The method of claim 7, wherein, the calibration accuracy and the confidence interval of the initial calibration result are calculated, specifically comprising: after the optimization converges, extracting the final residual error vector of the optimization problem, and calculating the root mean square error thereof as an evaluation index of the overall calibration accuracy; calculating the covariance matrix estimate of the optimized parameter vector, and deriving the standard error of each calibration parameter based on it; calculating the confidence interval of each calibration parameter based on the standard error and the selected confidence level.

9. The method of claim 8, wherein, the validity and reasonableness of the initial calibration result are verified, specifically comprising: checking whether all the optimized calibration parameters are within their pre-set reasonable physical range; analyzing the statistical distribution characteristics of the final residual error to determine whether there is a systematic deviation or an un-converged observation point; comparing the calibration accuracy evaluation index with the pre-set application accuracy requirement to verify whether the result meets the use demand; based on the above judgment result and verification result, determining whether to take the initial calibration result as the target calibration result or trigger the re-calibration process.

10. A monitoring camera boresight heading measurement device, wherein, the device comprises: a data acquisition module configured to acquire a plurality of raw angle data based on a PTZ angle sensor when a camera is aimed at a target point, acquire a plurality of raw coordinate data based on an RTK-GPS receiver for a target point in a monitoring scene, and perform time synchronization processing on the plurality of raw angle data and the corresponding plurality of raw coordinate data; a conversion module configured to perform validity verification on the plurality of raw coordinate data to obtain valid coordinate data, perform coordinate system conversion on the valid coordinate data, verify the data accuracy of the conversion, and obtain converted coordinate data based on the verification result. The computing module is configured to calculate theoretical angle data based on the converted coordinate data, calculate an angle deviation between the theoretical angle data and the original angle data, and construct an error function based on the angle deviation; The result output module is configured to jointly optimize the pose parameters of the camera and the gimbal deviation parameters based on the error function by using a nonlinear optimization algorithm, obtain an initial calibration result, calculate a calibration precision and a confidence interval of the initial calibration result, verify validity and rationality of the initial calibration result, and output a target calibration result based on a verification result.

Images