A camera image principal point fast calibration method and system based on night light tracking

Abstract

The application provides a kind of camera inner reference point calibration method and system based on night light tracking, and the image main point is the intersection of optical axis and imaging plane, it is one of many inner references, although the position of main point can be estimated by existing algorithm and calibration technology, but due to his relatively more complicated process, and accuracy and robustness are difficult to guarantee.The application utilizes the characteristics that all feature points in monitoring camera picture will gather to the direction of image main point with the reduction of focal length, and the virtual focus phenomenon of light spot in night scene, through threshold segmentation, contour fitting and target tracking technical method, the light spot in night monitoring image scene is detected, positioned and tracked multiple iterations, and the position of light spot center in monitoring picture is used to realize the determination of monitoring camera image main point inner reference.

Description

Technical Field

[0001] This invention belongs to the field of camera calibration, specifically relating to a method and system for rapid calibration of camera principal points based on nighttime light tracking. Background Technology

[0002] Currently, surveillance cameras are installed in most parts of the country and along roads. When it is necessary to perform measurement tasks on these pre-installed cameras that are not easily disassembled, calibration is required. However, existing target calibration methods are not suitable for calibrating such cameras. Currently, the calibration method for outdoor cameras that cannot be disassembled mainly relies on manual calibration. This requires placing a calibration reference object on the camera and accurately measuring the geographic spatial coordinates and image coordinates of the reference object to determine the imaging geometric model between them. Optimization methods are then used to determine the camera's intrinsic and extrinsic parameters. However, there are numerous intrinsic and extrinsic parameters for cameras, and various errors can occur due to mechanical, optical, and measurement factors during the calibration process. Therefore, existing methods suffer from problems such as being time-consuming and labor-intensive, and being susceptible to environmental influences affecting the accuracy of point selection. Summary of the Invention

[0003] To reduce the complexity of calibration, improve calibration accuracy and operability, reduce the introduction of error factors during the calibration process, and effectively utilize high-accuracy calibration reference objects within the regulatory scope, this invention proposes a calibration method for the main point of a surveillance camera image based on nighttime light detection and tracking.

[0004] This invention overcomes the defects and shortcomings of existing technologies. Based on surveillance video images in nighttime scenes, it utilizes the characteristics of the principal point in the surveillance camera and the features of nighttime light spots to calculate and extract the position of light spots and the fitted contour center in real time. Based on the tracking results of the light spots at different focal lengths, it rapidly determines the intrinsic parameters of the camera's principal point. The invention proposes an implementation process and method. The surveillance video images are acquired in real time by outdoor cameras already installed. The central server needs to acquire images within the field of view of each video camera.

[0005] The technical solution of this invention is a method for rapid calibration of camera principal points based on nighttime light tracking, comprising the following steps:

[0006] Step 1: By configuring the camera patrol strategy, real-time monitoring video images in nighttime scenes are collected to obtain image information, and the center of the image is used as the initial value of the main image point;

[0007] Step 2: Control the monitoring camera to acquire image when the focal length is the smallest, select a light spot as the object to be monitored, use threshold segmentation to filter the edge and contour information of the light spot in the image, and further extract the location information of the light spot in the night scene.

[0008] Step 3: Based on the light spot contour generation result obtained in Step 2, perform ellipse fitting operation to accurately describe the position and center location of the light spot;

[0009] Step 4: Increase the focal length based on the light spot center obtained in Step 3, and at the same time adjust the camera's rotation and pitch angles to place the light spot center at the initial position of the principal image point.

[0010] Step 5: Determine if the camera's focal length is at its maximum. If not, repeat steps 2-4; otherwise, use the light spot at the starting position of the principal point as the target to be monitored.

[0011] Step 6: After selecting the target to be monitored according to Step 5, keep the camera rotation angle p and pitch angle t constant, and gradually zoom in from near to far. During this process, execute Step 2 and Step 3 frame by frame to extract the position of the light spot, and use the Kalman filter algorithm to filter out other light spot interference in the image range when the focal length decreases. When the minimum focal length is adjusted, use the image coordinates of the center of the light spot determined by the tracking results to update the initial position of the principal point.

[0012] Step 7: Repeat steps 2-6 multiple times, using the coordinates of the center of the light spot after iteration as a set of principal point coordinates of the undetermined memory image (u i v i );

[0013] Step 8: Repeat steps 2-7 to obtain the principal point coordinates of multiple undetermined intrinsic parameter images, calculate the initial cluster centroid, and filter out discrete points with errors greater than a certain threshold based on the distance threshold. Then, use the centroid of the filtered coordinate group as the final principal point coordinates of the intrinsic parameter images.

[0014] Furthermore, in step 2, the spot image is segmented using the Otsu's method (maximum inter-class variance). The specific implementation method is as follows:

[0015] First, a threshold k is set to divide the image into two regions based on grayscale values. The larger the inter-class variance of the grayscale values ​​between the two regions, the greater the difference between the foreground and background, and the better the threshold. The formula for calculating the inter-class variance is defined as:

[0016]

[0017] Where m1 and m2 are the average gray values ​​of the two regions, m G Let P1 be the global grayscale mean, and P2 be the proportions of the two regions to the whole. The k value with the largest inter-class variance is obtained by iterative traversal algorithm.

[0018] Furthermore, in step 3, an ellipse is fitted using the least squares method, and the parametric equation of the ellipse is:

[0019] Ax 2+Bxy+Cy 2 +Dx+Ey+1=0

[0020] Where A, B, C, D, and E are coefficients, let P i (x i y i Let N be the N contour points on the elliptical contour. The more contour points, the better the fitting effect. Let i = 1, 2, ..., N. According to the least squares principle, the objective function for fitting is expressed as:

[0021]

[0022] To minimize the objective function F, using the extremum theorem, we can obtain:

[0023]

[0024] Solving the above system of linear equations yields the values ​​of A, B, C, D, and E. Based on the geometric principles of ellipses, the coordinates of the ellipse's center (x0, y0), the semi-major axis a, the semi-minor axis b, and the inclination angle θ of the major axis can be further calculated.

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] Furthermore, the specific implementation method of step 4 is as follows:

[0031] A Cartesian coordinate system is established with the initial position of the principal point as the origin. The horizontal direction is one axis, parallel to the direction of the camera rotation angle, and the positive direction of the horizontal axis is defined to be consistent with the direction of the increase of the camera rotation angle. The vertical direction is another axis, parallel to the direction of the camera pitch angle, and the positive direction of the vertical axis is defined to be consistent with the direction of the increase of the camera pitch angle. The image can be divided into upper and lower parts by the horizontal axis and into left and right parts by the vertical axis.

[0032] When the center of the light spot is on the left side of the image, by decreasing the camera's rotation angle p, the image moves to the left while the center of the light spot moves to the right; when the center of the light spot is on the right side of the image, by increasing the camera's rotation angle p, the image moves to the right while the center of the light spot moves to the left; when the center of the light spot is on the top side of the image, by increasing the camera's pitch angle t, the image moves to the top while the center of the light spot moves down; when the center of the light spot is on the bottom side of the image, by decreasing the camera's pitch angle t, the image moves to the bottom while the center of the light spot moves up.

[0033] Furthermore, the specific implementation method of the Kalman filter algorithm in step 6 is as follows:

[0034] First, we define a state vector for a stochastic discrete-time process, which can be represented by a discrete stochastic difference equation:

[0035] x k =Ax k-1 +Bu k-1 +ω k-1

[0036] Where, x k Let x represent the system state variable at time k. k-1 Let A represent the system state variables at time k-1, A be the state escape matrix (which links the states at time k to k-1), B be the control matrix (which measures how the control input u acts on the current system state), and ω be the system state variables at time k-1. k-1 Indicates noise;

[0037] Then, define the observation variables:

[0038] Z k =Hx k +V k

[0039] Among them, Z k Represents the observed value, x k Let H be the state variable defined in the previous equation, where x is x. k For Z k Gain, V k This indicates observation noise.

[0040] During the prediction phase, the state at the current moment is estimated based on the posterior estimate from the previous moment, thus obtaining the prior estimate for the current moment:

[0041]

[0042] in, Let A be the prior state estimate at time k, which is the result predicted at time k based on the optimal estimate at time k-1; let B be the state transition matrix; and let B be the control matrix, which is how the control variable u acts on the current state. P represents the filtered result of the posterior state estimate at time k-1. k-1 The posterior estimated covariance matrix at time k-1 is used to measure state uncertainty and is the result of filtering. Let be the prior estimate covariance matrix at time k; Q is the noise covariance matrix.

[0043] During the measurement update phase, the estimated value from the prediction phase is corrected using the current observation value; that is, the current posterior estimate is obtained from the prior estimate. The update equation for the Kalman filter state is shown below:

[0044]

[0045] in, Let k be the prior state estimate at time k; P is the prior estimate of the covariance matrix at time k; k Let V be the posterior estimated covariance matrix at time k; H is the transformation matrix from state variables to observations; R is the measurement noise covariance; V k Kalman coefficients; Z represents the posterior state estimate at time k; k The observed values ​​are the input values ​​for filtering. The residuals between the actual observations and the predicted values ​​are used to correct the prior using the Kalman coefficients, and finally the posterior is obtained.

[0046] Furthermore, in step 6, during the actual lens scaling process, the center spot position of the previous frame image is first used as the observation value at time k-1. The estimated value at time k is calculated, and combined with the actual observation value at time k, the Kalman coefficient is calculated to update the posterior estimate and the posterior covariance matrix. Starting from the second frame, the prior state estimate at the current time is first calculated, and the Euclidean distance between the detected center positions of all spots within a certain range of the current spot center and the prior state estimate is calculated within the image range. The closest spot is taken as the observation value at the current time, and then the state of the filter is updated. When the focal length is adjusted to the minimum, the state is no longer updated.

[0047] Furthermore, the specific implementation method of step 8 is as follows:

[0048] Step 8.1: Obtain the principal point coordinates of multiple undetermined intrinsic parameters {p} according to steps 2-7. i =(u i v i The coordinates of the n principal points (i = 1, 2, ..., n) are considered as a group. Based on the coordinates of the n principal points in the coordinate group, the centroid of this group of coordinates in the two-dimensional image coordinates is calculated. The centroid coordinates are derived from the mean of the coordinates in each dimension and are represented as μ.u μ v The calculation formula is:

[0049]

[0050] Where u and v represent the horizontal and vertical coordinates of the principal point, respectively, and j represents the principal point number;

[0051] Step 8.2: Calculate the Euclidean distance between the centroid coordinates and each principal point of the image to be determined. The calculation formula is:

[0052]

[0053] Where p represents the coordinates of the principal point of the image to be determined, μ represents the centroid coordinates, u and v represent the horizontal and vertical axes of the image coordinate system, respectively, and d represents the Euclidean distance between the centroid coordinates and the principal point p of the image to be determined.

[0054] Step 8.3: Calculate the average distance from all undetermined principal points of the image within the cluster to the current centroid. The calculation formula is:

[0055]

[0056] In the formula, m represents the number of principal points to be determined, μ is the centroid coordinate, d represents the distance between the j-th principal point and the centroid, and mdist represents the average distance between each principal point to be determined and the centroid.

[0057] Step 8.4: Compare mdist with a predetermined threshold T. If mdist is greater than T, then start from the principal point {p} of the undetermined image. i =(u i v i In the array {i = 1, 2, ..., n}, the principal image point furthest from the centroid is removed using the formula: del(max{d(p)} j ,μ), j=1,2,...m}), repeat steps 8.1 to 8.4 until mdist is not greater than the predetermined threshold T;

[0058] Step 8.5, after iterations 8.1 to 8.4, the centroid (μ) is obtained. u μ v The principal point coordinates of the image are output as intrinsic parameters of the camera.

[0059] Furthermore, the image information in step 1 is V. ptz ,

[0060] p∈[0, 360]

[0061] t∈[-180, 180]

[0062] z∈[1, 50]

[0063] Where p represents the range of movement of the camera in the horizontal direction; t represents the range of movement in the vertical direction; and z represents the zoom range.

[0064] This invention also provides a rapid camera principal point calibration system based on nighttime light tracking, comprising the following modules:

[0065] The image acquisition module is used to acquire real-time surveillance video images in nighttime scenes by configuring the camera patrol strategy, obtain image information, and use the image center as the initial position of the image principal point.

[0066] The light location information acquisition module is used to control the surveillance camera to acquire image images when the focal length is at its minimum, select a light spot as the object to be monitored, use threshold segmentation to filter the edge and contour information of the light spot in the image, and further extract the location information of the light spot in the night scene.

[0067] The center positioning module is used to perform ellipse fitting operation based on the obtained light spot contour generation result, so as to accurately describe the position and center positioning of the light spot;

[0068] The spot center adjustment module is used to increase the focal length based on the acquired spot center, while adjusting the camera's rotation and pitch angles to place the spot center at the initial position of the principal image point.

[0069] The judgment module is used to determine whether the camera's focal length is at its maximum at this time. If not, the light position information acquisition module, center positioning module, and light spot center adjustment module are executed repeatedly; otherwise, the light spot at the starting position of the main point is taken as the target to be monitored.

[0070] The principal point coordinate acquisition module, based on the selected target to be monitored, keeps the camera rotation angle p and pitch angle t constant, and gradually zooms in from near to far. During this process, the position of the light spot is extracted frame by frame, and the Kalman filter algorithm is used to filter out other light spot interference in the image range when the focal length decreases. When the minimum focal length is adjusted, the image coordinates of the light spot center determined by the tracking results are used to update the initial position of the principal point, and this process is repeated multiple times. The coordinates of the light spot center after the iteration are used as a set of undetermined intrinsic parameter principal point coordinates.

[0071] The calibration module is used to obtain the principal point coordinates of multiple undetermined intrinsic parameter images, calculate the initial cluster centroid, and filter out discrete points with errors greater than a certain threshold based on a distance threshold. The centroid of the filtered coordinate group is then used as the final principal point coordinates of the intrinsic parameter images.

[0072] The present invention also provides a camera principal point rapid calibration device based on nighttime light tracking, comprising:

[0073] One or more processors;

[0074] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the aforementioned method for fast camera principal point calibration based on nighttime light tracking.

[0075] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows:

[0076] (I) This invention utilizes the characteristic that the principal point of a camera is usually near the center of the image, and that the feature points at the center of the image gradually move closer to the principal point as the focal length decreases. It proposes a calibration method for the principal point of a camera intrinsic parameter based on nighttime light tracking. After verification in real-world scenarios, the calibration accuracy of the principal point of this invention meets the actual requirements.

[0077] (II) This invention utilizes the characteristics of light spot defocusing phenomenon and high contrast between light spot and background in outdoor night scene, and adopts detection methods such as threshold segmentation and contour fitting to quickly complete the detection and center positioning of light spot in monitoring video images in night scene. Combined with the position tracking of the target in multi-magnification zoom image, it realizes the rapid calibration of the intrinsic parameters of the main point of the video camera. It has the characteristics of convenience and strong operability. Attached Figure Description

[0078] Figure 1 This is the overall flowchart of the present invention;

[0079] Figure 2 This is a schematic diagram of light spot threshold segmentation and contour extraction;

[0080] Figure 3 To select the light spot to be detected for this embodiment of the invention, the focal length is adjusted to the maximum, and the center of the light spot is placed in the center of the image.

[0081] Figure 4 To track Figure 3 The light spot in the image is located within the red box, and the camera focal length is adjusted to the minimum. The position of the light spot is the principal point of the camera image.

[0082] Figure 5 This is a schematic diagram of the iterative tracking process. a is the position of the real principal point of the image, b is the position of the image center, the first tracking calculation yields the principal point c, the second calculation yields the principal point d, and so on. Detailed Implementation

[0083] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0084] This invention proposes a method for principal point calibration of surveillance camera images used in nighttime light detection and tracking. The overall flowchart is as follows: Figure 1 As shown, it includes the following steps:

[0085] Step 1: Configuring a camera patrol strategy to collect real-time surveillance video images in nighttime scenes, obtaining image information, and transmitting it to the central server; the image information is V. ptz The center of the image is taken as the initial value of the principal point.

[0086] p∈[0, 360]

[0087] t∈[-180, 180]

[0088] z∈[1, 50]

[0089] Where p represents the horizontal movement of the camera, i.e., the camera's rotation; t represents the vertical movement, i.e., the camera lens's pitch; and z represents zoom, i.e., adjusting the camera's focal length. Then V ptz This indicates the patrol images taken by the video camera under the PTZ (Pan-Touch) mode. Currently, most cameras support panoramic patrol or preset point patrol, so a patrol strategy can be configured to capture all monitoring images within the field of view of the tower video camera.

[0090] Step 2: The central server controls the surveillance cameras to acquire images when the focal length is at its minimum. A light spot is selected as the target to be monitored. Threshold segmentation is used to filter the edge and contour information of the light spot in the image, and the location information of the light highlights in the night scene is further extracted. The specific steps are as follows:

[0091] First, the spot image is segmented using the Otsu thresholding method. Otsu thresholding, also known as the maximum inter-class variance method, is a commonly used adaptive gray-level thresholding method in digital image processing. This method is unaffected by image brightness and contrast. Its basic idea is to set a threshold k to divide the image into two regions based on gray values. The larger the inter-class variance of the gray values ​​between the two regions, the greater the difference between the foreground and background, and the better the threshold. The formula for calculating the inter-class variance is defined as:

[0092]

[0093] Where m1 and m2 are the average gray values ​​of the two regions, m G Let P1 be the global grayscale mean, and P2 be the proportions of the two regions to the whole. The k value with the largest inter-class variance can be obtained by iterative traversal algorithm.

[0094] The generation of light spot contours based on edge extraction mainly utilizes edge enhancement operators to detect areas with significant changes in grayscale values ​​and defines these areas as edges. Commonly used operators include the Sobel operator, Roberts operator, Prewitt operator, LOG operator, and Canny operator. Since light spot edges generally have good symmetry characteristics, the choice of operator has little impact on the extraction accuracy of the light spot center based on circle fitting. Here, the Sobel operator is selected for edge extraction.

[0095] Step 3: The central server performs a circle fitting operation based on the light spot contour generated in Step 2 to accurately describe the position and center location of the light spot. The specific method is as follows:

[0096] First, a circle fitting algorithm based on the least squares method is used. This algorithm is a commonly used algorithm for extracting the center of the light spot. Due to different imaging angles, the shape of the light spot changes when the light is incident at an oblique angle. It will not be perfectly circular, but mostly elliptical. Ellipse fitting can describe the position and shape of the light spot more accurately than circle fitting. The parametric equation of the ellipse is:

[0097] Ax 2 +Bxy+Cy 2 +Dx+Ey+1=0

[0098] Where A, B, C, D, and E are coefficients, let P i (x i y i (i = 1, 2, ..., N) represents N contour points on the elliptical contour. The more contour points, the better the fitting effect. Based on the least squares principle, the objective function for fitting is expressed as:

[0099]

[0100] To minimize the objective function F, using the extremum theorem, we can obtain...

[0101]

[0102] Solving the above system of linear equations yields the values ​​of A, B, C, D, and E. Based on the principles of elliptic geometry, the coordinates of the ellipse's center (x, y) can be further calculated. o (y0), major semi-axis a, minor semi-axis b, and major axis inclination angle θ:

[0103]

[0104]

[0105]

[0106]

[0107]

[0108] Step 4: The central server adjusts the focal length based on the light spot center obtained in Step 3, and simultaneously adjusts the camera's PT attitude to place the light spot center at the initial position of the principal image point. The specific adjustment method is as follows:

[0109] A Cartesian coordinate system is established with the initial position of the principal point as the origin. The horizontal axis is parallel to the camera's rotation angle, and its positive direction is defined to align with the direction in which the camera's rotation angle increases. The vertical axis is parallel to the camera's pitch angle, and its positive direction is defined to align with the direction in which the camera's pitch angle increases. The image can be divided into upper and lower parts along the horizontal axis, and into left and right parts along the vertical axis.

[0110] When the center of the light spot is on the left side of the image, the central server decreases the camera's rotation angle p, causing the image to move to the left while the light spot center moves to the right. When the center of the light spot is on the right side of the image, the central server increases the camera's rotation angle p, causing the image to move to the right while the light spot center moves to the left. When the center of the light spot is on the top side of the image, the central server increases the camera's pitch angle t, causing the image to move upwards while the light spot center moves downwards. When the center of the light spot is on the bottom side of the image, the central server decreases the camera's pitch angle t, causing the image to move downwards while the light spot center moves upwards.

[0111] Step 5: Determine if the camera's focal length is at its maximum. If not, repeat steps 2-4; otherwise, use the light spot at the starting position of the principal point as the target to be monitored.

[0112] Step 6: After selecting the monitoring target according to Step 5, keep the camera rotation angle p and pitch angle t constant, and gradually zoom in from near to far. During this process, extract the light spot position frame by frame using the same detection and positioning method as in Steps 2 and 3, and combine it with the Kalman filter algorithm to filter out other light spot interference in the image range when the focal length decreases. When the minimum focal length is adjusted, combine the image coordinates of the light spot center determined by the tracking results, and update the initial position of the principal point using the light spot center position. The specific method is as follows:

[0113] As the focal length gradually decreases, the image area gradually increases, introducing interference factors such as other light spots besides the current light spot, affecting the accuracy of the current light spot's positioning. Therefore, the Kalman filter algorithm is introduced. The core idea of ​​this algorithm is to perform state estimation by fusing the system's dynamic model and measurement data. It typically consists of initialization, prediction, and measurement update steps, and uses a recursive approach to effectively and continuously update the estimated value. The position of the light spot closest to the current estimated value in each step is taken as the tracking result for that step, and the prior parameters are dynamically corrected.

[0114] First, we define a state vector for a stochastic discrete-time process, which can be represented by a discrete stochastic difference equation:

[0115] x k =Ax k-1 +Bu k-1 +ω k-1

[0116] Where, x k Let x represent the system state variable at time k. k-1 Let A represent the system state variables at time k-1, A be the state escape matrix (which links the states at time k to k-1), B be the control matrix (which measures how the control input u acts on the current system state), and ω be the system state variables at time k-1. k-1 Indicates noise.

[0117] Then, define the observation variables:

[0118] Z k =Hx k +V k

[0119] Among them, Z k Represents the observed value, x k Let H be the state variable defined in the previous equation, where x is x. k For Z k Gain, V k This indicates observation noise.

[0120] During the prediction phase, the state at the current moment is estimated based on the posterior estimate from the previous moment, thus obtaining the prior estimate for the current moment:

[0121]

[0122] in, Let A be the prior state estimate at time k, which is the result predicted at time k based on the optimal estimate at time k-1; let B be the state transition matrix; and let B be the control matrix, which is how the control variable u acts on the current state. P represents the filtered result of the posterior state estimate at time k-1. k-1The posterior estimated covariance matrix at time k-1 is used to measure state uncertainty and is the result of filtering. Let be the prior estimate covariance matrix at time k; Q is the noise covariance matrix.

[0123] During the measurement update phase, the estimated value from the prediction phase is corrected using the current observation value; that is, the current posterior estimate is obtained from the prior estimate. The update equation for the Kalman filter state is shown below:

[0124]

[0125] in, Let k be the prior state estimate at time k; P is the prior estimate of the covariance matrix at time k; k Let V be the posterior estimated covariance matrix at time k; H is the transformation matrix from state variables to observations; R is the measurement noise covariance; V k Kalman coefficients; Z represents the posterior state estimate at time k; k The observed values ​​are the input values ​​for filtering. The residuals between the actual observations and the predicted values ​​are used to correct the prior using the Kalman coefficients, and finally the posterior is obtained.

[0126] During the actual lens scaling process, the center spot position of the previous frame image is first used as the observation value at time k-1. The estimated value at time k is calculated, and combined with the actual observation value at time k, the Kalman coefficients are calculated to update the posterior estimate and the posterior covariance matrix. Starting from the second frame, the prior state estimate at the current time is calculated first. Then, within the image range, the Euclidean distance between the center positions of all detected spots within a certain range of the current spot center and the prior state estimate is measured. The closest spot is used as the observation value at the current time, and then the filter state is updated. When the focal length is adjusted to the minimum, no further state updates are performed.

[0127] Step 7: Repeat steps 2-6 multiple times, using the coordinates of the center of the light spot after iteration as a set of principal point coordinates of the undetermined intrinsic parameter image (u). i v i The iterative process is illustrated in the diagram below. Figure 5 As shown.

[0128] Step 8: Repeat steps 2-7 to obtain the coordinates of multiple undetermined principal points, calculate the initial cluster centroid, and after filtering out discrete points with large errors based on the distance threshold, use the centroid of the filtered coordinate group as the final principal point coordinates. The specific steps are as follows:

[0129] 8.1. Obtain the principal point coordinates of multiple undetermined intrinsic parameter images {p} according to steps 2-7.i =(u i v i The coordinates of the n principal points (i = 1, 2, ..., n) are considered as a group. Based on the coordinates of the n principal points in the coordinate group, the centroid of this group in the two-dimensional image coordinates is calculated. The centroid coordinates are derived from the mean of the coordinates in each dimension and are expressed as μ. u μ v The calculation formula is:

[0130]

[0131] Where u and v represent the horizontal and vertical coordinates of the principal point, respectively, and j represents the principal point number.

[0132] 8.2 Calculate the Euclidean distance between the centroid coordinates and each principal point of the image to be determined. The formula is:

[0133]

[0134] Where p represents the coordinates of the principal point of the image to be determined, μ represents the centroid coordinates, u and v represent the horizontal and vertical axes of the image coordinate system, respectively, and d represents the Euclidean distance between the centroid coordinates and the principal point p of the image to be determined.

[0135] 8.3 Calculate the average distance from all undetermined principal points of the image within the cluster to the current centroid. The formula is:

[0136]

[0137] In the formula, m represents the number of principal points to be determined, μ is the centroid coordinate, d represents the distance between the j-th principal point and the centroid, and mdist represents the average distance between each principal point to be determined and the centroid.

[0138] 8.4. Compare mdist with a predetermined threshold T. If mdist is greater than T, then start from the principal point {p} of the undetermined image. i =(u i v i In the set {i = 1, 2, ..., n}, the principal image point furthest from the centroid is removed using the formula: del(max{d(p)} j ,μ), j=1,2,...m}), repeat 7.1 to 7.4 until mdist is not greater than the predetermined threshold T.

[0139] 8.5. After iterations from 8.1 to 8.4, the centroid (μ) is obtained. u μ v The principal point coordinates of the image are output as intrinsic parameters of the camera.

[0140] This invention provides a calibration method for the principal point of a camera's intrinsic parameters based on nighttime light tracking. The principal point is the intersection of the optical axis and the imaging plane, and is one of many intrinsic parameters. Although existing algorithms and calibration techniques can estimate the position of the principal point, the calculation process is relatively cumbersome, and accuracy and robustness are difficult to guarantee. This invention utilizes the inherent characteristics of the principal point in a surveillance camera and the characteristics of light spots in outdoor nighttime scenes. Through threshold segmentation, contour fitting, and target tracking techniques, it detects, locates, and tracks light spots in nighttime surveillance video scenes. The intrinsic parameters of the surveillance camera's principal point are determined by the position of the light spot center in the surveillance image. See the effect diagram below. Figure 3 and Figure 4 .

[0141] This invention also provides a rapid camera principal point calibration system based on nighttime light tracking, comprising the following modules:

[0142] The image acquisition module is used to acquire real-time surveillance video images in nighttime scenes by configuring the camera patrol strategy, obtain image information, and use the image center as the initial position of the image principal point.

[0143] The light location information acquisition module is used to control the surveillance camera to acquire image images when the focal length is at its minimum, select a light spot as the object to be monitored, use threshold segmentation to filter the edge and contour information of the light spot in the image, and further extract the location information of the light spot in the night scene.

[0144] The center positioning module is used to perform ellipse fitting operation based on the obtained light spot contour generation result, so as to accurately describe the position and center positioning of the light spot;

[0145] The spot center adjustment module is used to increase the focal length based on the acquired spot center, while adjusting the camera's rotation and pitch angles to place the spot center at the initial position of the principal image point.

[0146] The judgment module is used to determine whether the camera's focal length is at its maximum at this time. If not, the light position information acquisition module, center positioning module, and light spot center adjustment module are executed repeatedly; otherwise, the light spot at the starting position of the main point is taken as the target to be monitored.

[0147] The principal point coordinate acquisition module, based on the selected target to be monitored, keeps the camera rotation angle p and pitch angle t constant, and gradually zooms in from near to far. During this process, the position of the light spot is extracted frame by frame, and combined with the Kalman filter algorithm, other light spot interference in the image range is filtered out when the focal length decreases. When the minimum focal length is adjusted, the image coordinates of the light spot center determined by the tracking results are used to update the initial position of the principal point, and the process is iterated multiple times. The coordinates of the light spot center after the iteration are used as a set of undetermined intrinsic parameter principal point coordinates.

[0148] The calibration module is used to obtain the principal point coordinates of multiple undetermined intrinsic parameter images, calculate the initial cluster centroid, and filter out discrete points with errors greater than a certain threshold based on a distance threshold. The centroid of the filtered coordinate group is then used as the final principal point coordinates of the intrinsic parameter images.

[0149] The specific implementation methods of each module are the same as those of each step, and will not be described in this invention.

[0150] This invention also provides a rapid camera principal point calibration device based on nighttime light tracking, comprising:

[0151] One or more processors;

[0152] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the aforementioned method for fast camera principal point calibration based on nighttime light tracking.

[0153] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the invention.

Claims

1. A method for rapid camera principal point calibration based on nighttime light tracking, characterized in that, Includes the following steps: Step 1: By configuring the camera patrol strategy, real-time monitoring video images in nighttime scenes are collected to obtain image information, and the center of the image is used as the initial value of the main image point; Step 2: Control the monitoring camera to acquire image when the focal length is the smallest, select a light spot as the object to be monitored, use threshold segmentation to filter the edge and contour information of the light spot in the image, and further extract the position information of the light spot in the night scene. Step 3: Based on the light spot contour generation result obtained in Step 2, perform ellipse fitting operation to accurately describe the light spot position and center positioning; Step 4: Increase the focal length based on the light spot center obtained in Step 3, and at the same time adjust the camera's rotation and pitch angles to place the light spot center at the initial position of the principal image point. Step 5: Determine if the camera's focal length is at its maximum. If not, repeat steps 2-4; otherwise, use the light spot at the starting position of the principal point as the target to be monitored. Step 6: After selecting the target to be monitored according to Step 5, keep the camera rotation angle p and pitch angle t constant, and gradually zoom in from near to far. During this process, execute Step 2 and Step 3 frame by frame to extract the position of the light spot, and use the Kalman filter algorithm to filter out other light spot interference in the image range when the focal length decreases. When the minimum focal length is adjusted, update the initial position of the principal point by combining the image coordinates of the light spot center determined by the tracking results. Step 7: Repeat steps 2-6 multiple times, using the coordinates of the center of the light spot after iteration as a set of principal point coordinates of the undetermined memory image (u i , v i ); Step 8: Repeat steps 2-7 to obtain the principal point coordinates of multiple undetermined intrinsic parameter images, calculate the initial cluster centroid, and filter out discrete points with errors greater than a certain threshold based on the distance threshold. Then, use the centroid of the filtered coordinate group as the final principal point coordinates of the intrinsic parameter images.

2. The method for rapid camera principal point calibration based on nighttime light tracking as described in claim 1, characterized in that: In step 2, the spot image is segmented using the Otsu's method (maximum inter-class variance). The specific implementation method is as follows: First, a threshold k is set to divide the image into two regions based on grayscale values. The larger the inter-class variance of the grayscale values ​​between the two regions, the greater the difference between the foreground and background, and the better the threshold. The formula for calculating the inter-class variance is defined as: in, and The average gray level across the two regions. This represents the global average grayscale value. and The proportions of the two regions to the whole are given, and the k value with the largest inter-class variance is obtained by iterative traversal algorithm.

3. The method for rapid camera principal point calibration based on nighttime light tracking as described in claim 1, characterized in that: In step 3, ellipse fitting is performed using the least squares method. The parametric equation of the ellipse is: Where A, B, C, D, and E are coefficients, let... These are N contour points on the elliptical contour. The more contour points, the better the fitting effect. Based on the least squares principle, the objective function for fitting is expressed as: To minimize the objective function F, using the extremum theorem, we can obtain: Solving the above system of linear equations yields the values ​​of A, B, C, D, and E; based on the principles of elliptic geometry, the coordinates of the ellipse's center are further calculated. Major semi-axis a, minor semi-axis b, and major axis inclination angle : 。 4. The method for rapid camera principal point calibration based on nighttime light tracking as described in claim 1, characterized in that: The specific implementation method of step 4 is as follows: A Cartesian coordinate system is established with the initial position of the principal point as the origin. The horizontal direction is one axis, parallel to the direction of the camera rotation angle, and the positive direction of the horizontal axis is defined to be consistent with the direction of the increase of the camera rotation angle. The vertical direction is another axis, parallel to the direction of the camera pitch angle, and the positive direction of the vertical axis is defined to be consistent with the direction of the increase of the camera pitch angle. The image can be divided into upper and lower parts by the horizontal axis and into left and right parts by the vertical axis. When the center of the light spot is located on the left side of the image, the camera rotation angle is reduced. This causes the image to move to the left while simultaneously moving the center of the light spot to the right; when the center of the light spot is located on the right side of the image, the camera's rotation angle is increased. This causes the image to move to the right while simultaneously shifting the center of the light spot to the left; when the center of the light spot is located at the top of the image, the camera's pitch angle is increased. This causes the image to move upwards while simultaneously shifting the center of the light spot downwards; when the center of the light spot is located at the bottom of the image, the camera's pitch angle is reduced. This causes the image to move downwards while the center of the light spot moves upwards.

5. The method for rapid camera principal point calibration based on nighttime light tracking as described in claim 1, characterized in that: The specific implementation method of the Kalman filter algorithm in step 6 is as follows: First, we define a state vector for a stochastic discrete-time process, which can be represented by a discrete stochastic difference equation: in, Represents the system state variables at time k. for The system state variables at time k, where A represents the state escape matrix, i.e., the state transition matrix between time k and time k. Connecting the time-state relationship, B represents the control matrix, used to measure how the control variable u acts on the current state of the system. Indicates noise; Then, define the observation variables: in, Represents the observed value. The state variables defined in the previous equation, for right Gain, Indicates observation noise; During the prediction phase, the state at the current moment is estimated based on the posterior estimate from the previous moment, thus obtaining the prior estimate for the current moment: in, Let A be the prior state estimate at time k, which is the result predicted at time k based on the optimal estimate at time k-1; let B be the state transition matrix; and let B be the control matrix, which is how the control variable u acts on the current state. The posterior state estimate at time k-1 is the filtered result. The posterior estimated covariance matrix at time k-1 is used to measure state uncertainty and is the result of filtering. Let be the prior estimate covariance matrix at time k; Q is the noise covariance matrix. During the measurement update phase, the estimated value from the prediction phase is corrected using the current observation value; that is, the current posterior estimate is obtained from the prior estimate. The update equation for the Kalman filter state is shown below: in, Let k be the prior state estimate at time k; Let be the prior estimate of the covariance matrix at time k; Let H be the posterior estimated covariance matrix at time k; H be the transformation matrix from state variables to observations; and R be the measurement noise covariance. Kalman coefficients; Let be the posterior state estimate at time k; The observed values ​​are the input values ​​for filtering. The residuals between the actual observations and the predicted values ​​are used to correct the prior using the Kalman coefficients, and finally the posterior is obtained.

6. The method for rapid camera principal point calibration based on nighttime light tracking as described in claim 5, characterized in that: Step 6: During the actual lens scaling process, firstly, the center spot position of the previous frame image is used as the observation value at time k-1. The estimated value at time k is calculated, and combined with the actual observation value at time k, the Kalman coefficient is calculated to update the posterior estimate and the posterior covariance matrix. Starting from the second frame, the prior state estimate at the current time is calculated first. Then, within the image range, the Euclidean distance between the detected center positions of all spots within a certain range of the current spot center and the prior state estimate is calculated. The closest spot is taken as the observation value at the current time, and then the state of the filter is updated. When the focal length is adjusted to the minimum, the state is no longer updated.

7. The method for rapid camera principal point calibration based on nighttime light tracking as described in claim 1, characterized in that: The specific implementation method of step 8 is as follows: Step 8.1: Obtain the principal point coordinates of multiple undetermined intrinsic parameter images according to steps 2-7. The coordinates of these multiple principal points are considered as a group. Based on the coordinates of the n principal points in the coordinate group, the centroid of this group of coordinates in the two-dimensional image coordinates is calculated. The centroid coordinates are derived from the mean of the coordinates in each dimension, and are expressed as follows: The calculation formula is: Where u and v represent the horizontal and vertical coordinates of the principal point, respectively, and j represents the principal point number; Step 8.2: Calculate the Euclidean distance between the centroid coordinates and each principal point of the image to be determined. The calculation formula is: Where p represents the coordinates of the principal point of the image to be determined. The centroid coordinates are represented by u and v, which represent the horizontal and vertical axes of the image coordinate system, respectively. The Euclidean distance between the centroid coordinates and the principal point p of the image to be determined is represented by d. Step 8.3: Calculate the average distance from all undetermined principal points of the image within the cluster to the current centroid. The calculation formula is: In the formula, m represents the number of principal points of the image to be determined. Let be the centroid coordinates, d represent the distance between the j-th principal image point and the centroid, and mdist represent the average distance between all undetermined principal image points and the centroid. Step 8.4: Compare mdist with a predetermined threshold T. If mdist is greater than T, then start from the principal point of the undetermined image. The formula for removing the principal image point farthest from the centroid is: Repeat steps 8.1 to 8.4 until mdist is no greater than the predetermined threshold T; Step 8.5, after iterations 8.1 to 8.4, yields the centroid ( Output the principal point coordinates as the camera's intrinsic reference image.

8. The method for rapid camera principal point calibration based on nighttime light tracking as described in claim 1, characterized in that: The image information in step 1 is , in, Indicates the range of horizontal movement of the camera; Indicates the range of movement in the vertical direction; Indicates the zoom range.

9. A rapid camera principal point calibration system based on nighttime light tracking, characterized in that, Includes the following modules: The image acquisition module is used to acquire real-time surveillance video images in nighttime scenes by configuring the camera patrol strategy, obtain image information, and take the center of the image as the initial position of the principal point. The light position information acquisition module is used to control the surveillance camera to acquire image images when the focal length is the smallest, select a light spot as the object to be monitored, use threshold segmentation to filter the edge and contour information of the light spot in the image, and further extract the position information of the light spot in the night scene. The center positioning module is used to perform ellipse fitting operation based on the obtained light spot contour generation result, so as to accurately describe the position and center positioning of the light spot; The spot center adjustment module is used to increase the focal length based on the acquired spot center, while adjusting the camera's rotation and pitch angles to place the spot center at the initial position of the principal image point. The judgment module is used to determine whether the camera's focal length is at its maximum at this time. If not, the light position information acquisition module, center positioning module, and light spot center adjustment module are executed repeatedly; otherwise, the light spot at the starting position of the main point is taken as the target to be monitored. The principal point coordinate acquisition module, based on the selected target to be monitored, keeps the camera rotation angle p and pitch angle t constant, and gradually zooms in from near to far. During this process, the position of the light spot is extracted frame by frame, and the Kalman filter algorithm is used to filter out other light spot interference in the image range when the focal length decreases. When the minimum focal length is adjusted, the initial position of the principal point is updated by combining the image coordinates of the center of the light spot determined by the tracking results, and iterates multiple times. The coordinates of the center of the light spot after iteration are used as a set of undetermined intrinsic parameter principal point coordinates. The calibration module is used to obtain the principal point coordinates of multiple undetermined intrinsic parameter images, calculate the initial cluster centroid, and filter out discrete points with errors greater than a certain threshold based on a distance threshold. The centroid of the filtered coordinate group is then used as the final principal point coordinates of the intrinsic parameter images.

10. A rapid camera principal point calibration device based on nighttime light tracking, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement a method for rapid camera principal point calibration based on nighttime light tracking as described in any one of claims 1 to 8.

Images