A target tracking method and system based on gun and PTZ device

Abstract

The present invention provides a target tracking method and system based on a gun camera and a PTZ device, which utilizes the target tracking data in the gun camera screen to correct the PTZ device motion control parameters generated by the screen tracking data of the PTZ device, thereby improving the success rate of the PTZ device in tracking the target in the monitoring scene.

Description

Technical Field

[0001] The invention relates to the field of intelligent security monitoring, and in particular to a target tracking method and system based on a gun camera and a PTZ device. Background Art

[0002] In modern intelligent security monitoring, automatic analysis of the target behavior intention in the monitoring scene, such as whether the target is about to enter the restricted area, move quickly, etc., is of great significance for reducing the workload of staff and improving the practical value of the system. To automatically analyze the target behavior intention, it is necessary to first perceive the target in the monitoring scene, especially through PTZ (abbreviation of Pan / Tilt / Zoom, representing the full range (left / right / up / down) movement of the pan / tilt and lens zoom and zoom control) devices such as pan / tilt cameras and PTZ dome cameras. By using their horizontal and vertical rotation and depth of field zoom functions, it can not only lock targets moving in a large range, but also provide close-up images with rich target details and textures. Close-up images with rich textures provide the premise for the subsequent application of intelligent analysis algorithms. Manually controlling the rotation and zoom of the pan / tilt camera not only increases the workload of the staff, but also the obtained monitoring images are not smooth and continuous. More importantly, it is difficult to track fast-moving targets. Therefore, in order to better realize the follow-up and locking of the target in the monitoring scene by the PTZ device, it is necessary to realize automatic control of the rotation and zoom of the PTZ device.

[0003] The existing automatic control of PTZ devices can be roughly divided into active and passive types according to the source of control data. The active type requires the target in the scene to load GPS positioning tracking equipment, electronic compass, or other auxiliary reference components and other equipment. The target actively informs the PTZ device of its location information, which is used to realize the rotation and scaling control of the PTZ device or for system initialization correction. This mode not only increases the complexity of the system and increases the investment cost, but also the target needs to carry auxiliary equipment, which leads to certain limitations in practical application. The passive type does not require the target to actively inform the PTZ device of its location information, but automatically detects and tracks specific targets in the PTZ device screen through image intelligent analysis algorithms, and controls the rotation and scaling of the PTZ device through the target position. This system is relatively simple and does not require additional equipment investment. More importantly, it can handle almost all kinds of objects that break into the surveillance scene, so it has been widely used. However, the control performance of passive PTZ devices depends on whether the image intelligent analysis algorithm can continuously and stably track the target in the picture. In real applications, since the target moves over a large range in the surveillance scene, such as monitoring the aircraft from the parking position to the runway in the airport flight zone, in order to keep tracking the target, the PTZ device also needs to rotate and scale quickly, resulting in large differences in factors such as target posture, picture background, and scene lighting, which in turn easily leads to the inability of the image intelligent analysis algorithm to continuously and stably track the target in the picture.

[0004] However, in the gun camera screen, since the gun camera is stationary, the degree of change of the above factors is relatively low. Therefore, for the target in the scene, the target tracking success rate in the gun camera screen is higher than that in the PTZ device screen. If the target tracking data in the gun camera screen is used to correct the PTZ device motion control parameters generated by the PTZ device screen tracking data, the success rate of the PTZ device tracking the target in the monitoring scene can be improved. Summary of the invention

[0005] In view of the above technical problems, the technical solution adopted by the present invention is:

[0006] The embodiment of the present invention provides a target tracking method based on a gun camera and a PTZ device, comprising the following steps:

[0007] S10, obtaining first initial position information of the same target of interest in the surveillance scene in the PTZ device screen and second initial position information in the gun screen, and starting a first target tracking algorithm according to the obtained first initial position information and starting a second target tracking algorithm according to the obtained second initial position information;

[0008] S20, respectively using the first target tracking algorithm and the second target tracking algorithm to obtain the position of the target of interest in the PTZ device screen and the position in the gun screen, to obtain a first tracking position and a second tracking position;

[0009] S30, based on the first current tracking position corresponding to the current frame obtained by the first target tracking algorithm, determining the motion control parameter value of the PTZ device in the time interval between the current frame and the next frame, including the motion control parameter values ​​in the horizontal, vertical and depth of field directions;

[0010] S40, based on the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device, determine the position of the target of interest in the gun camera screen relative to the coordinate space of the PTZ device;

[0011] S50, based on the current spatial position of the PTZ device, the motion control parameter value, and the position of the target of interest in the gun camera image relative to the coordinate space of the PTZ device, the motion control parameter value is corrected;

[0012] S60, controlling the motion of the PTZ device according to the corrected motion control parameter value.

[0013] The embodiment of the present invention also provides a target tracking system based on a gun camera and a PTZ device, comprising: a front-end monitoring device, a back-end data processing device and a customer interaction terminal connected in communication;

[0014] The front-end monitoring device includes a gun camera and a ptz device arranged in the monitoring scene;

[0015] The client interaction terminal is used to determine the target of interest that needs to be tracked in the monitoring scene, and send the first initial position information of the determined target of interest in the PTZ device screen or the second initial position information in the gun screen to the background data processing device;

[0016] The background data processing device is used to execute a computer program to implement the following steps:

[0017] S1, starting a first target tracking algorithm according to the acquired first initial position information and starting a second target tracking algorithm according to the acquired second initial position information;

[0018] S2, respectively using the first target tracking algorithm and the second target tracking algorithm to obtain the position of the target of interest in the PTZ device screen and the position in the gun screen, to obtain a first tracking position and a second tracking position;

[0019] S3, based on the first current tracking position corresponding to the current frame obtained by the first target tracking algorithm, determining the motion control parameter value of the PTZ device in the time interval between the current frame and the next frame, including the motion control parameter values ​​in the horizontal, vertical and depth of field directions;

[0020] S4, based on the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device, determine the position of the target of interest in the gun camera screen relative to the coordinate space of the PTZ device;

[0021] S5, based on the current spatial position of the PTZ device, the motion control parameter value, and the position of the target of interest in the gun camera image relative to the coordinate space of the PTZ device, the motion control parameter value is corrected;

[0022] S6, controlling the motion of the PTZ device according to the corrected motion control parameter value.

[0023] The target tracking method and system based on a gun camera and a PTZ device provided by the embodiment of the present invention utilize the target tracking data in the gun camera screen to correct the PTZ device motion control parameters generated by the screen tracking data of the PTZ device, thereby improving the success rate of the PTZ device in tracking the target in the monitoring scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flow chart of a target tracking method based on a gun camera and a PTZ device provided in an embodiment of the present invention;

[0026] Figure 2 A structural block diagram of a target tracking system based on a gun and a PTZ device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] Figure 1 The flowchart of the target tracking method based on the gun and PTZ device provided by the embodiment of the present invention is as follows. Figure 1 As shown, an embodiment of the present invention provides a target tracking method based on a gun camera and a PTZ device, which may include the following steps:

[0029] S10, obtaining the first initial position information of the same target of interest in the surveillance scene in the PTZ device screen and the second initial position information in the gun screen, and starting the first target tracking algorithm according to the obtained first initial position information and starting the second target tracking algorithm according to the obtained second initial position information.

[0030] In the embodiment of the present invention, the monitoring scenario and the target of interest may be determined based on actual needs.

[0031] In an embodiment of the present invention, the first target tracking algorithm and the second target tracking algorithm may be existing target tracking algorithms, such as classic target tracking algorithms based on continuous adaptive mean shift (CamShift), kernel correlation filter (KCF), etc. As long as target tracking can be achieved, the present invention does not make any special limitations. The first target tracking algorithm and the second target tracking algorithm may be the same algorithm or different algorithms. The first target tracking algorithm and the second target tracking algorithm are respectively used to obtain the position of the target of interest in the ptz device screen and the gun screen in real time during the subsequent tracking process.

[0032] In an embodiment of the present invention, the first initial position information and the second initial position information can be determined based on a clicked PTZ device screen or a gun camera screen.

[0033] Further, in an exemplary embodiment, the first initial position information and the second initial position information are determined based on the clicked PTZ device screen or the gun screen, and may include:

[0034] S101, if the currently clicked screen is a ptz device screen, obtain the first initial position information.

[0035] In the embodiment of the present invention, the first initial position information includes the coordinates and size of the target in the screen of the ptz device, and can be identified by a minimum rectangular frame containing the target.

[0036] S102, determining the coordinate space position of the object of interest relative to the ptz device according to the current spatial position of the ptz device, the current parameters of the ptz device and the first initial position information of the object of interest in the ptz device screen.

[0037] In an embodiment of the present invention, the spatial position of the current PTZ device can be used as a reference, and the offset of the target relative to the reference can be calculated according to the current parameters of the PTZ device and the first initial position information of the target of interest in the PTZ device screen. The coordinate space position of the target of interest relative to the PTZ device can be determined by the reference and the offset of the target of interest relative to the reference.

[0038] S103, determining the second initial position information of the target of interest in the gun camera screen according to the coordinate space position of the target of interest relative to the PTZ device and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device.

[0039] In the embodiment of the present invention, the spatial correspondence mapping relationship between the gun camera screen and the ptz device is a set spatial correspondence mapping relationship, which can be determined by actual calibration. During the actual calibration, the gun camera screen is divided into a grid of m rows and n columns, and the ptz device is adjusted to the center of each grid to obtain an element unit with the coordinates of the grid center and the spatial coordinates corresponding to the ptz device as the mapping relationship. All element units constitute a mapping matrix of the spatial correspondence mapping relationship of m rows and n columns.

[0040] Further, in S103, the second initial position information may be determined in the following manner:

[0041] S1031, taking the coordinate space position of the target of interest relative to the ptz device as the reference position, obtaining the L nearest neighbor element units closest to the reference position in the mapping matrix representing the spatial corresponding mapping relationship between the gun camera screen and the ptz device; preferably, L=4.

[0042] S1032, performing bilinear interpolation on the data representing the image pixel coordinates in the L nearest neighbor element units to obtain the second initial position information of the target of interest in the gun camera screen.

[0043] Further, in another exemplary embodiment, the first initial position information and the second initial position information are determined based on the clicked PTZ device screen or the gun screen, and may include:

[0044] S104, if the current click is on the gun camera screen, obtain the second initial position information.

[0045] In the embodiment of the present invention, the second initial position information includes the coordinates and size of the target in the gun camera screen, and can be identified by a minimum rectangular frame containing the target.

[0046] S105, determining the spatial position of the target of interest relative to the ptz device according to the second initial position information, the spatial corresponding mapping relationship between the gun camera image and the ptz device.

[0047] Further, in S105, the spatial position of the object of interest relative to the ptz device can be determined by:

[0048] S1051, using the second initial position information as a reference position, obtaining L nearest neighbor element units closest to the reference position in a mapping matrix representing the spatial corresponding mapping relationship between the gun camera screen and the PTZ device; preferably, L=4.

[0049] S1052, bilinear interpolation is performed on the data representing the position of the ptz device in the L nearest neighbor element units to obtain the spatial position of the target of interest relative to the ptz device.

[0050] S106, according to the spatial position of the target of interest relative to the PTZ device, control the PTZ device to rotate to the spatial position of the target of interest relative to the PTZ device, so as to capture the target of interest in the screen of the PTZ device and obtain first initial position information.

[0051] In an embodiment of the present invention, the PTZ device can be controlled to rotate to the spatial position of the target of interest relative to the PTZ device according to the spatial position of the target of interest relative to the PTZ device, and then the identification frame in the second initial position information is offset to a position with reference to the center of the PTZ device screen in the PTZ device screen, and the offset identification frame is used as the first initial position information of the target of interest in the PTZ device screen.

[0052] S20, respectively using the first target tracking algorithm and the second target tracking algorithm to obtain the position of the target of interest in the PTZ device screen and the position in the gun camera screen, to obtain a first tracking position and a second tracking position.

[0053] S30, based on the first current tracking position corresponding to the current frame obtained by the first target tracking algorithm, determine the motion control parameter value of the PTZ device in the time interval between the current frame and the next frame, including the motion control parameter values ​​in the horizontal, vertical and depth of field directions.

[0054] S30 further includes:

[0055] S301, estimating a moving speed of an object of interest according to a first current tracking position and a position change of the object of interest in previous n frames.

[0056] In an embodiment of the present invention, the estimation method for estimating the moving speed of the target of interest can adopt an existing estimation method, for example, the estimation method disclosed in the document with publication number CN105744234B and invention name A motion target linkage method and device based on pixel difference ratio.

[0057] S302, sending a motion status query command to the ptz device to obtain the current motion status of the ptz device.

[0058] S303, determining the motion control parameter values ​​of the PTZ device in the horizontal, vertical and depth directions within the time interval between the current frame and the next frame according to the estimated moving speed of the target of interest and the current motion state of the PTZ device.

[0059] In an embodiment of the present invention, according to the estimated moving speed of the target of interest and the current motion state of the PTZ device, the motion control parameter values ​​of the PTZ device in the horizontal, vertical and depth of field directions within the time interval between the current frame and the next frame can be determined by an existing method, for example, the method disclosed in the document with announcement number CN105744234B and invention name A motion target linkage method and device based on pixel differential ratio.

[0060] S40, based on the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device, determine the position of the target of interest in the gun camera screen relative to the coordinate space of the PTZ device.

[0061] Specifically, in S40, the spatial position of the object of interest relative to the ptz device can be determined in the following manner:

[0062] S401, taking the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm as the reference position, obtaining the L nearest neighbor element units closest to the reference position in the mapping matrix representing the spatial corresponding mapping relationship between the gun camera screen and the ptz device; preferably, L=4.

[0063] S402, bilinear interpolation is performed on the data representing the position of the ptz device in the L nearest element units to obtain the position of the target of interest in the gun camera image in the coordinate space relative to the ptz device.

[0064] S50, based on the current spatial position of the PTZ device and the position of the target of interest in the gun camera image relative to the coordinate space of the PTZ device, modify the motion control parameter value.

[0065] In an embodiment of the present invention, modifying the motion control parameter value may include:

[0066] (1) Send a position status query command to the ptz device to obtain the current spatial position of the ptz device;

[0067] (2) determining the motion control parameter correction coefficients of the PTZ device in the horizontal, vertical and depth directions according to the queried PTZ device spatial position and the position of the target of interest in the gun camera image relative to the PTZ device space;

[0068] (3) Correct the motion control parameter value obtained in step S30 according to the calculated current frame motion control parameter and correction coefficient.

[0069] In a specific example, S50 may further include the following steps:

[0070] S501, obtaining Δv i =vi c -v i ptz ; where v i c ∈(v1 c , v2 c , v3 c ), v i ptz ∈(v1 ptz , v2 ptz ,

[0071] v3 ptz ), Δv i ∈(Δv1, Δv2, Δv3), the value of i ranges from 1 to 3; v1 c , v2 c , v3 c Respectively represent the horizontal coordinates, vertical coordinates and depth coordinates of the target of interest in the gun camera image in the coordinate space of the ptz device; v1 ptz , v2 ptz , v3 ptz They respectively represent the current horizontal coordinates, vertical coordinates and depth-of-field coordinates of the ptz device; Δv1, Δv2 and Δv3 respectively represent the differences of the above corresponding coordinates.

[0072] S502, obtain v i ptz-r =v i ptz-c +(Δv i / Δt)*k;v i ptz-c ∈(v1 ptz-c , v2 ptz-c , v3 ptz-c ),

[0073] v1 ptz-c , v2 ptz-c , v3 ptz-c are the motion control parameter values ​​of the PTZ device in the horizontal, vertical and depth directions in the time interval between the current frame and the next frame determined by estimating the target motion in the PTZ device screen; Δt is the time interval between the current frame and the next frame; v i ptz-r ∈(v1 ptz-r , v2 ptz-r , v3 ptz-r ), v1 ptz-r , v2 ptz-r , v3 ptz-rk are the motion control parameter values ​​of the PTZ device in the horizontal, vertical and depth directions in the time interval between the current frame and the next frame after correction. i ∈(k1, k2, k3), k1, k2, k3 are the weight coefficients of the motion control parameters of the ptz device in the horizontal, vertical and depth of field directions respectively.

[0074] In an illustrative embodiment, k1, k2, and k3 are fixed values, and the values ​​of k1, k2, and k3 can be determined based on actual application scenario debugging.

[0075] In another exemplary embodiment, k1, k2, k3 are variables that can be obtained based on adaptive adjustment. i =(1-exp((Δv i *Δv i ) / μ i ))*α, where μ i ∈(μ1,μ2,μ3) is the set ptz speed change variance in the horizontal, vertical, and depth of field directions, preferably, it can be the ptz speed change variance in the horizontal, vertical, and depth of field directions statistically calculated according to the actual application scenario, α is the set weight coefficient, preferably, it can be the weight coefficient determined by debugging according to the actual application scenario. Compared with k1, k2, k3 being fixed values, k1, k2, k3 obtained based on adaptive adjustment can make the control more accurate.

[0076] S60, controlling the motion of the PTZ device according to the corrected motion control parameter value.

[0077] The PTZ device can be controlled to rotate and scale according to the corrected motion control parameter value to track the target of interest in the surveillance scene. According to the corrected motion control parameters of the PTZ device in the horizontal, vertical and depth of field directions, and the horizontal, vertical and depth of field motion control command definitions of the PTZ devices of different manufacturers, the control command of the PTZ device is generated in the time interval between the current frame and the next frame, and the PTZ device is controlled to rotate and scale smoothly and stably to track the target of interest in the surveillance scene.

[0078] Furthermore, the method provided by the embodiment of the present invention also includes:

[0079] S70, determining whether the tracking process is finished, if not, executing S20; otherwise, executing S80.

[0080] S80, ending the tracking process.

[0081] In S70, it is determined whether the current automatic tracking task of the target of interest in the monitoring scene is finished. If not finished, the target tracking process of the gun and the PTZ device is returned to continue the state change of the subsequent target automatic tracking step. If finished, the above process is exited, and the method state is reset to prepare for the next execution of the target automatic tracking task. In an embodiment of the present invention, there are two situations that can be judged as finished, one is manual control to end the PTZ device target automatic tracking task, and the other is when the gun and the PTZ device fail to automatically track the target of interest and the number of motion estimations of the target of interest after the failure exceeds the maximum estimation threshold set in advance, the number of motion estimations of the target of interest is obtained by the prediction module of the tracking algorithm, and the maximum estimation threshold can be set according to the actual application scenario.

[0082] In an embodiment of the present invention, the ptz device may include a pan / tilt head and a gun camera or a ball camera integrating rotation, zooming and video recording.

[0083] In the embodiment of the present invention, the gun camera includes one or more cameras, which are used to cooperate with PTZ to realize the tracking task of targets in surveillance scenes of different ranges. If there are multiple cameras, multiple pictures can be spliced ​​into a panoramic picture to calculate the spatial correspondence between the gun camera picture and the PTZ device.

[0084] Figure 2 The structure block diagram of the target tracking system based on the gun and PTZ device provided by the embodiment of the present invention. Figure 2 As shown, the target tracking system based on the gun camera and the PTZ device provided by the embodiment of the present invention may include three parts: a front-end monitoring device, a background data processing device, and a customer interaction terminal connected in communication. The switches of the three parts are connected through optical fiber links, and the devices are connected to the corresponding switches through a gigabit network cable.

[0085] In an embodiment of the present invention, the front-end monitoring device is used to track the target of interest in the monitoring scene to provide a close-up picture with rich target details and textures, including a gun camera, a PTZ device and a front-end switch set in the monitoring scene, and the PTZ device and the gun camera are fixed on a mounting bracket. The PTZ device can be a combination of a pan / tilt and a gun camera or a ball camera integrating rotation, zooming and video recording. According to the size of the field of view of the monitoring scene, the gun camera can be a single or multiple cameras, which are used to cooperate with the PTZ device to achieve the tracking task of targets in monitoring scenes of different ranges. If there are multiple cameras, multiple pictures can be spliced ​​into a panoramic picture to facilitate the calculation of the spatial correspondence between the gun camera picture and the PTZ device. The front-end switch can be an ordinary switch. Considering that the front-end monitoring equipment is placed outdoors for a long time, an industrial switch with better performance can also be selected. The number of ports of the front-end switch is determined according to the needs of the front-end equipment. The mounting bracket is used to fix the gun camera and the PTZ device of the front-end equipment to the installation position. At the same time, the mounting bracket is also equipped with a small outdoor cabinet for placing related modules such as switches and equipment power supply.

[0086] The client interactive terminal is used to run the client program of the system, and helps the user control the system to realize the automatic tracking function of the key focus target in the monitoring scene through convenient interactive operations, including determining the target of interest that needs to be tracked in the monitoring scene, and sending the first initial position information of the determined target of interest in the PTZ device screen or the second initial position information in the gun screen to the background data processing device. The client interactive terminal is mainly composed of a client computer, a display, and a client switch. If the display is an ordinary display, the interactive operation is realized by a mouse, and if it is a touch display, the interactive operation is realized directly on the display screen by touch.

[0087] The background data processing device is used to execute the target tracking method of the combined gun camera and PTZ device. The background data processing device is mainly composed of a data processing workstation and a background switch. The data processing workstation is used to execute the target tracking method of the combined gun camera and PTZ device when capturing the video stream and decoding the video from the front-end monitoring device, and to connect and process the control commands of the customer interaction terminal to realize the control of the front-end monitoring device to automatically track the key focus targets in the monitoring scene. The background switch is used for data exchange between the data processing workstation, the front-end monitoring device and the customer interaction terminal. Specifically, the background data processing device is used to execute a computer program to implement the following steps:

[0088] S1, starting a first target tracking algorithm according to the acquired first initial position information and starting a second target tracking algorithm according to the acquired second initial position information;

[0089] S2, respectively using the first target tracking algorithm and the second target tracking algorithm to obtain the position of the target of interest in the PTZ device screen and the position in the gun screen, to obtain a first tracking position and a second tracking position;

[0090] S3, based on the first current tracking position corresponding to the current frame obtained by the first target tracking algorithm, determining the motion control parameter value of the PTZ device in the time interval between the current frame and the next frame, including the motion control parameter values ​​in the horizontal, vertical and depth of field directions;

[0091] S4, based on the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device, determine the position of the target of interest in the gun camera screen relative to the coordinate space of the PTZ device;

[0092] S5, based on the current spatial position of the PTZ device, the motion control parameter value, and the position of the target of interest in the gun camera image relative to the coordinate space of the PTZ device, the motion control parameter value is corrected;

[0093] S6, controlling the motion of the PTZ device according to the corrected motion control parameter value;

[0094] S7, determine whether the tracking process is completed, if not, execute S2; otherwise, execute S8;

[0095] S8, ending the tracking process.

[0096] The specific implementation of the above steps S1 to S8 may be the same as steps S10 to S80 of the aforementioned embodiment, and the specific description thereof is omitted to avoid redundancy.

[0097] In addition, for complex surveillance scenarios, multiple front-end surveillance devices can be deployed to cover the surveillance field of view, and each front-end surveillance device is equipped with a background data processing unit. As needed, the background data processing device can also be configured with a streaming media server to store historical videos of gun cameras and PTZ devices, which is convenient for users to review events afterwards. According to user needs, multiple customer interactive terminals can be configured. The interactive terminals can control multiple front-end surveillance devices through channel switching, and different role permissions are assigned to each interactive terminal to ensure orderly control of the same front-end surveillance device by different interactive terminals.

[0098] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A target tracking method based on a gun and a PTZ device, characterized in that: The following steps are involved: S10, obtaining first initial position information of the same target of interest in the surveillance scene in the PTZ device screen and second initial position information in the gun screen, and starting a first target tracking algorithm according to the obtained first initial position information and starting a second target tracking algorithm according to the obtained second initial position information; S20, respectively using the first target tracking algorithm and the second target tracking algorithm to obtain the position of the target of interest in the PTZ device screen and the position in the gun screen, to obtain a first tracking position and a second tracking position; S30, based on the first current tracking position corresponding to the current frame obtained by the first target tracking algorithm, determining the motion control parameter value of the PTZ device in the time interval between the current frame and the next frame, including the motion control parameter values ​​in the horizontal, vertical and depth directions; S40, based on the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device, determine the position of the target of interest in the gun camera screen relative to the coordinate space of the PTZ device; S50, based on the current spatial position of the PTZ device, the motion control parameter value, and the position of the target of interest in the gun camera image relative to the coordinate space of the PTZ device, the motion control parameter value is corrected; S60, controlling the motion of the PTZ device according to the corrected motion control parameter value.

2. The method according to claim 1, characterized in that The first initial position information and the second initial position information are determined based on a clicked PTZ device screen or a gun camera screen.

3. The method according to claim 2, characterized in that The first initial position information and the second initial position information are determined based on the clicked PTZ device screen or the box camera screen, including: S101, if the currently clicked screen is a ptz device screen, obtain the first initial position information; S102, determining the coordinate space position of the object of interest relative to the ptz device according to the current spatial position of the ptz device, the current parameters of the ptz device and the first initial position information of the object of interest in the ptz device screen; S103, determining the second initial position information of the target of interest in the gun camera screen according to the coordinate space position of the target of interest relative to the PTZ device and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device.

4. The method according to claim 3, characterized in that In S103, the second initial position information is determined in the following manner: S1031, taking the coordinate space position of the target of interest relative to the PTZ device as a reference position, obtaining L nearest neighbor element units closest to the reference position in a mapping matrix representing the spatial corresponding mapping relationship between the gun screen and the PTZ device; S1032, performing bilinear interpolation on the data representing the image pixel coordinates in the L nearest neighbor element units to obtain the second initial position information of the target of interest in the gun camera screen.

5. The method according to claim 2, characterized in that: The first initial position information and the second initial position information are determined based on the clicked PTZ device screen or the box camera screen, including: S104, if the current click is on the gun camera screen, obtain the second initial position information; S105, determining the spatial position of the target of interest relative to the ptz device according to the second initial position information, the spatial corresponding mapping relationship between the gun camera image and the ptz device; S106, according to the spatial position of the target of interest relative to the PTZ device, control the PTZ device to rotate to the spatial position of the target of interest relative to the PTZ device, so as to capture the target of interest in the screen of the PTZ device and obtain first initial position information.

6. The method according to claim 5, characterized in that In S105, the spatial position of the object of interest relative to the ptz device is determined by: S1051, using the second initial position information as a reference position, obtaining L nearest neighbor element units closest to the reference position in a mapping matrix representing a spatial corresponding mapping relationship between the gun camera screen and the PTZ device; S1052, bilinear interpolation is performed on the data representing the position of the ptz device in the L nearest neighbor element units to obtain the spatial position of the target of interest relative to the ptz device.

7. The method according to claim 1, characterized in that S50 further includes: S501, obtaining Δv i =v i c -v i ptz ; where v i c ∈(v1 c , v2 c , v3 c ), v i ptz ∈(v1 ptz , v2 ptz , v3 ptz ), Δv i ∈(Δv1, Δv2, Δv3), i ranges from 1 to 3; v1 c , v2 c , v3 c Respectively represent the horizontal, vertical and depth coordinates of the target of interest in the gun screen relative to the coordinate space of the ptz device; v1 ptz , v2 ptz , v3 ptz Respectively represent the current coordinates of the ptz device in the horizontal direction, vertical direction and depth of field direction; Δv1 is the difference of the coordinates in the horizontal direction, Δv2 is the difference of the coordinates in the vertical direction, and Δv3 is the difference of the coordinates in the depth of field direction; S502, obtain v i ptz-r =v i ptz-c +(Δv i / Δt)*k i ;v i ptz-c ∈(v1 ptz-c , v2 ptz-c , v3 ptz-c ), v1 ptz-c , v2 ptz -c , v3 ptz-c are the motion control parameter values ​​of the PTZ device in the horizontal, vertical and depth directions in the time interval between the current frame and the next frame determined by estimating the motion of the target of interest in the PTZ device screen; Δt is the time interval between the current frame and the next frame; v i ptz-r ∈(v1 ptz-r , v2 ptz-r , v3 ptz-r ), v1 ptz-r , v2 ptz-r , v3 ptz-r are the motion control parameter values ​​of the PTZ device in the horizontal, vertical and depth directions in the time interval between the current frame and the next frame after correction, respectively. i ∈(k1, k2, k3), k1, k2, k3 are the weight coefficients of the motion control parameters of the ptz device in the horizontal, vertical and depth of field directions respectively.

8. The method according to claim 7, characterized in that k i =(1-exp((Δv i ·Δv i ) / m i ))*a, Among them, μ i ∈(μ1,μ2,μ3) is the velocity change variance of the set PTZ device in the horizontal, vertical and depth of field directions, and α is the set weight coefficient.

9. The method according to claim 1, characterized in that: In S40, the spatial position of the object of interest relative to the ptz device is determined by: S401, taking the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm as the reference position, obtaining L nearest neighbor element units closest to the reference position in the mapping matrix representing the spatial corresponding mapping relationship between the gun camera screen and the PTZ device; S402, bilinear interpolation is performed on the data representing the position of the ptz device in the L nearest element units to obtain the position of the target of interest in the gun camera image in the coordinate space relative to the ptz device.

10. A target tracking system based on a gun and a PTZ device, characterized in that: include: Communication-connected front-end monitoring equipment, back-end data processing equipment and customer interaction terminals; The front-end monitoring equipment includes a gun camera and a ptz device arranged in the monitoring scene; The client interaction terminal is used to determine the target of interest that needs to be tracked in the monitoring scene, and send the first initial position information of the determined target of interest in the PTZ device screen or the second initial position information in the gun screen to the background data processing device; The background data processing device is used to execute a computer program to implement the following steps: S1, starting a first target tracking algorithm according to the acquired first initial position information and starting a second target tracking algorithm according to the acquired second initial position information; S2, respectively using the first target tracking algorithm and the second target tracking algorithm to obtain the position of the target of interest in the PTZ device screen and the position in the gun screen, to obtain a first tracking position and a second tracking position; S3, based on the first current tracking position corresponding to the current frame obtained by the first target tracking algorithm, determining the motion control parameter value of the PTZ device in the time interval between the current frame and the next frame, including the motion control parameter values ​​in the horizontal, vertical and depth of field directions; S4, based on the second current tracking position corresponding to the current frame obtained by the second target tracking algorithm and the spatial corresponding mapping relationship between the gun camera screen and the PTZ device, determine the position of the target of interest in the gun camera screen relative to the coordinate space of the PTZ device; S5, based on the current spatial position of the PTZ device, the motion control parameter value, and the position of the target of interest in the gun camera image relative to the coordinate space of the PTZ device, the motion control parameter value is corrected; S6, controlling the motion of the PTZ device according to the corrected motion control parameter value.

Images