A joint calibration and target tracking method for security radar and ball camera

Abstract

The invention relates to intrusion target tracking, and in particular to a joint calibration and target tracking method of a security radar and a ball camera. The method comprises the following steps: installing a radar and a ball camera, and setting a monitoring area; constructing a tracking model between an intrusion target position and a PTZ of the ball camera, and a field of view model between the intrusion target position and a field of view of the ball camera; collecting multiple sets of training data, and constructing a training data set, and solving a tracking model and a field of view model by using the training data set; continuously receiving and parsing radar data, and analyzing whether there is an intrusion target in the monitoring area; inputting the parsed intrusion target position into a tracking model and a field of view model respectively, and a host computer controls the ball camera based on an output result of the tracking model, and superimposing the field of view of the ball camera in the radar field of view based on the output results of the tracking model and the field of view model; the technical solution provided by the invention can effectively overcome the defect that the prior art cannot quickly and accurately track the intrusion target.

Description

Technical Field

[0001] The invention relates to intrusion target tracking, and in particular to a joint calibration and target tracking method of a security radar and a ball camera. Background Art

[0002] In recent years, intelligent video surveillance technology has been widely used, but in actual application, there are problems such as being easily affected by the environment, having a small coverage area, and being difficult to actively detect intruders. With the gradual popularization of millimeter-wave radar in the civil field, its wide detection range and its unaffected by weather can provide security systems with reliable target detection capabilities. In addition, radar cooperates with the security strategy of ball cameras, and expands the monitoring range by rotating the ball cameras, and has been applied in many security scenarios.

[0003] In the existing security systems that link radar and ball cameras, the PTZ value of the ball camera is mostly calculated based on trigonometric functions. This method requires not only the pre-specification of the installation position and height of the radar and ball camera, but also a lot of debugging work, and the target tracking effect is poor. In addition, when facing multi-target tracking, there is a disadvantage that intrusion targets in different positions are easily missed because the field of view of the ball camera cannot cover all targets at once. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a joint calibration and target tracking method of a security radar and a ball camera, which can effectively overcome the defect of the prior art that it is unable to quickly and accurately track the intruding target.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] A joint calibration and target tracking method of a security radar and a ball camera comprises the following steps:

[0009] S1. Install radar and ball camera, and set up monitoring area;

[0010] S2, constructing a tracking model between the intrusion target position and the ball camera PTZ, and a field of view model between the intrusion target position and the field of view of the ball camera;

[0011] S3, collecting multiple sets of training data, and constructing a training data set, and using the training data set to solve the tracking model and the field of view model;

[0012] S4, continuously receiving and analyzing radar data to analyze whether there are intrusion targets in the surveillance area;

[0013] S5, inputting the intrusion target position obtained by the analysis into the tracking model and the field of view model respectively, the upper computer controls the ball camera based on the output result of the tracking model, and superimposes the field of view of the ball camera in the radar field of view based on the output results of the tracking model and the field of view model;

[0014] S6. The ball camera captures the intruding target, and the host computer displays the captured image in the superimposed field of view after identification and analysis.

[0015] Preferably, a radar and a ball camera are installed in S1, and a monitoring area is set, including:

[0016] Install millimeter wave radar and smart ball camera, set the monitoring area, and calibrate the radar, ball camera and monitoring area in the plane coordinate system;

[0017] The radar and the ball camera are on the same vertical line with the origin, and the installation height of the ball camera is H. C .

[0018] Preferably, in S2, a tracking model between the intrusion target position and the ball camera PTZ is constructed, including:

[0019] The P value of the ball machine is calculated using the following formula:

[0020]

[0021] Among them, the P value of the ball camera is the horizontal deflection angle of the ball camera, a p is the coefficient of the ratio between X and Y, b p is the offset, a p 、b p are fixed constants, X and Y are the lateral distance and longitudinal distance of the intrusion target in the radar coordinate system respectively;

[0022] The T value of the ball machine is calculated using the following formula:

[0023]

[0024] Among them, the T value of the ball camera is the pitch and deflection angle of the ball camera, a t H C , the coefficient of the ratio between R, b t is the offset, a t 、b t are all fixed constants, R is the straight-line distance between the intrusion target and the radar in the radar coordinate system;

[0025] Since the zoom ratio of the ball camera is proportional to the straight-line distance R, the Z value of the ball camera is calculated using the following formula:

[0026] Z=a z R+b z

[0027] Among them, the Z value of the ball camera is the zoom multiple of the ball camera, a z b is the coefficient for calculating the linear distance R when the dome camera zooms. z is the offset, a z , b z are all fixed constants.

[0028] Preferably, constructing a field of view model between the intrusion target position and the field of view of the ball camera in S2 includes:

[0029] Since the size of the ball camera's field of view changes with the straight-line distance R, the ball camera's field of view width W and field of view length L are calculated using the following formulas:

[0030] W=a W R+b W

[0031] L = a L R+b L

[0032] Among them, a W b is the coefficient of the straight-line distance R when calculating the field width W of the ball camera. W is the offset, a W , b W All are fixed constants;

[0033] a L b is the coefficient for calculating the straight-line distance R when the field of view width L of the ball camera is used. L is the offset, a L , b L are all fixed constants.

[0034] Preferably, multiple sets of training data are collected in S3, and a training data set is constructed, including:

[0035] In the surveillance area, calculate the intrusion target position x0, y0, r0 of the intrusion target G0 in the radar coordinate system, manually rotate the ball camera so that the intrusion target G0 appears in the center of the ball camera's field of view, and record the ball camera's horizontal deflection angle p0, the ball camera's pitch deflection angle t0, and the ball camera's zoom multiple z0 at this time;

[0036] Keep the ball camera still and measure the position of the upper left corner of the ball camera's field of view in the radar coordinate system (x L ,y L ), and the position of the lower right corner of the ball camera's field of view in the radar coordinate system (x R ,y R ), and calculate the width w0 and length l0 of the field of view of the ball camera at this time;

[0037] For intrusion targets at different locations in the surveillance area, collect N sets of training data and construct a training data set;

[0038] Among them, x0 and y0 are the lateral distance and longitudinal distance of the intrusion target G0 in the radar coordinate system, respectively, and r0 is the straight-line distance between the intrusion target G0 and the radar in the radar coordinate system;

[0039] The training data set includes the target position set (X, Y, R), the ball camera parameter set (P, T, Z) and the ball camera field of view set (W, L).

[0040] Preferably, solving the tracking model and the field of view model using the training data set in S3 includes:

[0041] The training data set is input into the tracking model and the field of view model respectively, and the least square method is used for linear fitting to calculate the parameters in the tracking model and the field of view model.

[0042] Preferably, S4 continuously receives and analyzes radar data to analyze whether there is an intrusion target in the monitoring area, including:

[0043] Continuously receive radar data. The radar sends radar data at a data refresh rate of 10 frames / s. Circularly analyze each frame of radar data to determine whether there is an intrusion target in the monitoring area;

[0044] If there is an intrusion target in the monitoring area, enter S5, otherwise continue to wait and analyze and judge the next frame of radar data.

[0045] Preferably, in S5, the intrusion target position obtained by the analysis is input into the tracking model and the field of view model respectively, the host computer controls the ball camera based on the output result of the tracking model, and superimposes the field of view of the ball camera in the radar field of view based on the output results of the tracking model and the field of view model, including:

[0046] For the intrusion target G' that enters the monitoring area, the radar data is analyzed to obtain the intrusion target position x', y', r' of the intrusion target G', and the intrusion target position x', y', r' is input into the tracking model to obtain the corresponding horizontal deflection angle p', pitch deflection angle t', and zoom multiple z' of the ball camera. The upper computer controls the ball camera based on the horizontal deflection angle p', pitch deflection angle t', and zoom multiple z' of the ball camera.

[0047] The intrusion target position r' is input into the field of view model to obtain the corresponding field of view width w' and field of view length l' of the ball camera. The host computer updates the coverage position and size of the field of view of the ball camera in real time in the display interface based on the horizontal deflection angle p', pitch deflection angle t', zoom factor z', field of view width w' and field of view length l' of the ball camera, and superimposes it with the radar field of view.

[0048] (III) Beneficial effects

[0049] Compared with the prior art, the joint calibration and target tracking method of security radar and ball camera provided by the present invention has the following beneficial effects:

[0050] 1) Use millimeter-wave radar to search for targets over a large range, and convert the position of the intrusion target detected by the radar into the PTZ value of the ball camera, thereby controlling the rotation and zoom of the ball camera, increasing the monitoring range of the ball camera, and improving the flexibility of the ball camera;

[0051] 2) In terms of the method of converting the intrusion target position into the PTZ value of the ball camera, by creating a tracking model and using the least squares method to solve the model, more accurate target tracking can be achieved;

[0052] 3) In the method of calculating the effective field of view of the ball camera, by creating a field of view model, when the ball camera rotates to different angles, the width and length of the effective field of view of the ball camera are synchronously calculated, and the coverage position and size of the field of view of the ball camera are updated in real time in the host computer, so as to more intuitively display the ball camera movement and target tracking results. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 It is a schematic diagram of the process of the present invention;

[0055] Figure 2 A schematic diagram of installing a radar and a ball camera and setting a monitoring area in the present invention;

[0056] Figure 3 For the present invention Figure 2 Plan view of the dome camera's field of view;

[0057] Figure 4 Schematic diagram of the change of the field of view size of the ball camera with the straight-line distance R in the present invention;

[0058] Figure 5 It is a scatter plot of the ratio between the horizontal deflection angle P of the ball camera and X and Y in the present invention;

[0059] Figure 6 The pitch and deflection angles T and H of the ball camera in the present invention are C , a scatter plot of the ratio between R;

[0060] Figure 7 It is a scatter plot of the zoom factor Z and the straight-line distance R of the ball camera in the present invention;

[0061] Figure 8 It is a scatter plot of the field width W and the straight-line distance R of the ball camera in the present invention;

[0062] Fig. 9 It is a scatter plot of the field length L and the straight-line distance R of the ball camera in the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] A joint calibration and target tracking method for security radar and ball camera, such as Figure 1 As shown in the figure, ① install radar and ball camera, and set the monitoring area, such as Figure 2 As shown, specifically including:

[0065] Install millimeter wave radar and smart ball camera, set the monitoring area, and calibrate the radar, ball camera and monitoring area in the plane coordinate system;

[0066] The radar and the ball camera are on the same vertical line with the origin, and the installation height of the ball camera is H. C .

[0067] ② Construct a tracking model between the intrusion target position and the PTZ camera, as well as a field of view model between the intrusion target position and the field of view of the ball camera.

[0068] 1) Build a tracking model between the intrusion target location and the PTZ camera, including:

[0069] The P value of the ball machine is calculated using the following formula:

[0070]

[0071] Among them, the P value of the ball camera is the horizontal deflection angle of the ball camera, a p is the coefficient of the ratio between X and Y, b p is the offset, a p 、b p are fixed constants, X and Y are the lateral distance and longitudinal distance of the intrusion target in the radar coordinate system respectively;

[0072] The T value of the ball machine is calculated using the following formula:

[0073]

[0074] Among them, the T value of the ball camera is the pitch and deflection angle of the ball camera, a t H C , the coefficient of the ratio between R, b t is the offset, a t , b t are all fixed constants, R is the straight-line distance between the intrusion target and the radar in the radar coordinate system;

[0075] Since the zoom ratio of the ball camera is proportional to the straight-line distance R, the Z value of the ball camera is calculated using the following formula:

[0076] Z=a z R+b z

[0077] Among them, the Z value of the ball camera is the zoom multiple of the ball camera, a z b is the coefficient for calculating the linear distance R when the dome camera zooms. z is the offset, a z , b z are all fixed constants.

[0078] 2) Construct a field of view model between the intrusion target location and the field of view of the ball camera, including:

[0079] like Figure 4 As shown, since the field of view of the ball camera changes with the straight-line distance R, the field of view width W and field of view length L of the ball camera are calculated using the following formulas:

[0080] W=a W R+b W

[0081] L = a L R+b L

[0082] Among them, a W b is the coefficient of the straight-line distance R when calculating the field width W of the ball camera. W is the offset, a W , b W All are fixed constants;

[0083] a L b is the coefficient for calculating the straight-line distance R when the field of view width L of the ball camera is used. L is the offset, a L , b L are all fixed constants.

[0084] ③ Collect multiple sets of training data, construct a training data set, and use the training data set to solve the tracking model and field of view model.

[0085] 1) Collect multiple sets of training data and construct a training data set, including:

[0086] In the surveillance area, calculate the intrusion target position x0, y0, r0 of the intrusion target G0 in the radar coordinate system, manually rotate the ball camera so that the intrusion target G0 appears in the center of the ball camera's field of view, and record the ball camera's horizontal deflection angle p0, the ball camera's pitch deflection angle t0, and the ball camera's zoom multiple z0 at this time;

[0087] Keep the ball machine still. Figure 3 As shown in the figure, measure the position of the upper left corner of the ball camera's field of view in the radar coordinate system (x L ,y L ), and the position of the lower right corner of the ball camera's field of view in the radar coordinate system (x R ,y R ), and calculate the width w0 and length l0 of the field of view of the ball camera at this time;

[0088] For intrusion targets at different locations in the surveillance area, collect N sets of training data and construct a training data set;

[0089] Among them, x0 and y0 are the lateral distance and longitudinal distance of the intrusion target G0 in the radar coordinate system, respectively, and r0 is the straight-line distance between the intrusion target G0 and the radar in the radar coordinate system;

[0090] The training data set includes the target position set (X, Y, R), the ball camera parameter set (P, T, Z) and the ball camera field of view set (W, L).

[0091] 2) Use the training data set to solve the tracking model and field of view model, including:

[0092] The training data set is input into the tracking model and the field of view model respectively, and the least square method is used for linear fitting to calculate the parameters in the tracking model and the field of view model.

[0093] Collect multiple sets of training data as shown in the following table:

[0094] Table 1 Multiple sets of training data

[0095]

[0096]

[0097] like Figures 5 to 7 As shown in Figure 1, the training data set constructed by multiple sets of training data is input into the tracking model, and the least squares method is used for linear fitting to obtain the values ​​of each parameter: a p =57.205, b p =0.0009; a t =56.872, b t =0.0076; a z =0.1, bz = 0. During the test, since the zoom factor Z of the ball camera is 32 times at the highest and 1 times at the lowest, when the straight-line distance R between the intrusion target and the radar exceeds 320m, the zoom factor Z of the ball camera will no longer change, and the effective value range of the straight-line distance R is 10m to 320m.

[0098] like Figure 8 and Fig. 9 As shown, the training data set constructed by multiple sets of training data is input into the field of view model, and the least squares method is used for linear fitting to obtain the values ​​of each parameter: a W =-0.04, b W =19; a L =-0.105, b L =91.

[0099] In addition, after solving the parameters in the tracking model and the field of view model, a test target can be set in the monitoring area to test and verify the solved model. When the test target enters the monitoring area, the radar data is parsed to obtain the test target position of the test target, and the test target position is input into the tracking model and the field of view model respectively to obtain the corresponding ball camera PTZ value, ball camera field of view width and ball camera field of view length. The ball camera is controlled to rotate and change magnification based on the ball camera PTZ value, and the model is tested and verified by judging whether the test target is at the center of the ball camera field of view.

[0100] ④Continuously receive and analyze radar data to analyze whether there are intrusion targets in the surveillance area, including:

[0101] Continuously receive radar data. The radar sends radar data at a data refresh rate of 10 frames / s. Circularly analyze each frame of radar data to determine whether there is an intrusion target in the monitoring area;

[0102] If there is an intrusion target in the monitoring area, enter S5, otherwise continue to wait and analyze and judge the next frame of radar data.

[0103] ⑤ Input the intrusion target position obtained by analysis into the tracking model and the field of view model respectively. The host computer controls the ball camera based on the output results of the tracking model, and superimposes the field of view of the ball camera in the radar field of view based on the output results of the tracking model and the field of view model, including:

[0104] For the intrusion target G' that enters the monitoring area, the radar data is analyzed to obtain the intrusion target position x', y', r' of the intrusion target G', and the intrusion target position x', y', r' is input into the tracking model to obtain the corresponding horizontal deflection angle p', pitch deflection angle t', and zoom multiple z' of the ball camera. The upper computer controls the ball camera based on the horizontal deflection angle p', pitch deflection angle t', and zoom multiple z' of the ball camera.

[0105] The intrusion target position r' is input into the field of view model to obtain the corresponding field of view width w' and field of view length l' of the ball camera. The host computer updates the coverage position and size of the field of view of the ball camera in real time in the display interface based on the horizontal deflection angle p', pitch deflection angle t', zoom factor z', field of view width w' and field of view length l' of the ball camera, and superimposes it with the radar field of view.

[0106] ⑥ The ball camera captures the intruding target, and the host computer displays the captured image in the superimposed field of view after identification and analysis.

[0107] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A joint calibration and target tracking method of a security radar and a ball camera, characterized in that: The following steps are involved: S1. Install radar and ball camera, and set up monitoring area; S2, constructing a tracking model between the intrusion target position and the ball camera PTZ, and a field of view model between the intrusion target position and the field of view of the ball camera; S3, collecting multiple sets of training data, and constructing a training data set, and using the training data set to solve the tracking model and the field of view model; S4, continuously receiving and analyzing radar data to analyze whether there are intrusion targets in the surveillance area; S5, inputting the intrusion target position obtained by the analysis into the tracking model and the field of view model respectively, the upper computer controls the ball camera based on the output result of the tracking model, and superimposes the field of view of the ball camera in the radar field of view based on the output results of the tracking model and the field of view model; S6. The ball camera captures the intruder target, and the host computer displays the captured image in the superimposed field of view after identification and analysis; S2 builds a tracking model between the intrusion target location and the ball camera PTZ, including: The P value of the ball machine is calculated using the following formula: Among them, the P value of the ball camera is the horizontal deflection angle of the ball camera, a p is the coefficient of the ratio between X and Y, b p is the offset, a p 、b p are fixed constants, X and Y are the lateral distance and longitudinal distance of the intrusion target in the radar coordinate system respectively; The T value of the ball machine is calculated using the following formula: Among them, the T value of the ball camera is the pitch and deflection angle of the ball camera, a t H C , the coefficient of the ratio between R, b t is the offset, a t 、b t are all fixed constants, R is the straight-line distance between the intrusion target and the radar in the radar coordinate system; Since the zoom ratio of the ball camera is proportional to the straight-line distance R, the Z value of the ball camera is calculated using the following formula: Z=a z R+b z Among them, the Z value of the ball camera is the zoom multiple of the ball camera, a z b is the coefficient for calculating the linear distance R when the dome camera zooms. z is the offset, a z 、b z All are fixed constants; In S2, a field of view model between the intrusion target position and the field of view of the ball camera is constructed, including: Since the size of the ball camera's field of view changes with the straight-line distance R, the ball camera's field of view width W and field of view length L are calculated using the following formulas: W=a W R+b W L=a L R+b L Among them, a W b is the coefficient of the straight-line distance R when calculating the field width W of the ball camera. W is the offset, a W 、b W All are fixed constants; a L b is the coefficient for calculating the straight-line distance R when the field of view width L of the ball camera is used. L is the offset, a L 、b L are all fixed constants.

2. The joint calibration and target tracking method of security radar and ball camera according to claim 1 is characterized in that: Install radar and ball camera in S1 and set monitoring area, including: Install millimeter wave radar and smart ball camera, set the monitoring area, and calibrate the radar, ball camera and monitoring area in the plane coordinate system; The radar and the ball camera are on the same vertical line with the origin, and the installation height of the ball camera is H. C .

3. The joint calibration and target tracking method of security radar and ball camera according to claim 1 is characterized in that: Collect multiple sets of training data in S3 and build a training data set, including: In the surveillance area, calculate the intrusion target position x0, y0, r0 of the intrusion target G0 in the radar coordinate system, manually rotate the ball camera so that the intrusion target G0 appears in the center of the ball camera's field of view, and record the ball camera's horizontal deflection angle p0, the ball camera's pitch deflection angle t0, and the ball camera's zoom multiple z0 at this time; Keep the ball camera still and measure the position of the upper left corner of the ball camera's field of view in the radar coordinate system (x L ,y L ), and the position of the lower right corner of the ball camera's field of view in the radar coordinate system (x R ,y R ), and calculate the width w0 and length l0 of the field of view of the ball camera at this time; For intrusion targets at different locations in the surveillance area, collect N sets of training data and construct a training data set; Among them, x0 and y0 are the lateral distance and longitudinal distance of the intrusion target G0 in the radar coordinate system, respectively, and r0 is the straight-line distance between the intrusion target G0 and the radar in the radar coordinate system; The training data set includes the target position set (X, Y, R), the ball camera parameter set (P, T, Z) and the ball camera field of view set (W, L).

4. The joint calibration and target tracking method of security radar and ball camera according to claim 3 is characterized in that: In S3, the tracking model and the field of view model are solved using the training data set, including: The training data set is input into the tracking model and the field of view model respectively, and the least square method is used for linear fitting to calculate the parameters in the tracking model and the field of view model.

5. The joint calibration and target tracking method of security radar and ball camera according to claim 4 is characterized in that: S4 continuously receives and analyzes radar data to analyze whether there are intruder targets in the surveillance area, including: Continuously receive radar data. The radar sends radar data at a data refresh rate of 10 frames / s. Circularly analyze each frame of radar data to determine whether there is an intrusion target in the monitoring area; If there is an intrusion target in the monitoring area, enter S5, otherwise continue to wait and analyze and judge the next frame of radar data.

6. The joint calibration and target tracking method of security radar and ball camera according to claim 5 is characterized in that: In S5, the intrusion target position obtained by the analysis is input into the tracking model and the field of view model respectively. The host computer controls the ball camera based on the output result of the tracking model, and superimposes the field of view of the ball camera in the radar field of view based on the output results of the tracking model and the field of view model, including: For the intrusion target G' that enters the monitoring area, the radar data is analyzed to obtain the intrusion target position x', y', r' of the intrusion target G', and the intrusion target position x', y', r' is input into the tracking model to obtain the corresponding horizontal deflection angle p', pitch deflection angle t', and zoom multiple z' of the ball camera. The upper computer controls the ball camera based on the horizontal deflection angle p', pitch deflection angle t', and zoom multiple z' of the ball camera. The intrusion target position r' is input into the field of view model to obtain the corresponding field of view width w' and field of view length l' of the ball camera. The host computer updates the coverage position and size of the field of view of the ball camera in real time in the display interface based on the horizontal deflection angle p', pitch deflection angle t', zoom factor z', field of view width w' and field of view length l' of the ball camera, and superimposes it with the radar field of view.

Images