Target tracking method and device of gun-ball linkage equipment, terminal and storage medium

Abstract

The application provides a target tracking method and device of a gun-ball linkage equipment, a terminal and a storage medium. The target tracking method comprises the following steps: acquiring a plurality of historical gun monitoring images in sequence, identifying the plurality of historical gun monitoring images in sequence, and acquiring a gun monitoring position of a target; mapping the gun monitoring position of the target to a plurality of historical ball monitoring images in sequence to obtain a ball monitoring position of the target; acquiring a ball running track of the target based on the ball monitoring position of the target, and predicting a first current ball monitoring position of the target based on the ball running track of the target; acquiring a current gun monitoring image, and acquiring a second current ball monitoring position based on the current gun monitoring image; and controlling the ball to track the target based on the first current ball monitoring position and the second current ball monitoring position. In this way, the comparison and confirmation are increased in the linkage of the gun and the ball, and the ability of the gun and the ball to stably hand over the target is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the security video technical field, in particular to a target tracking method and device of a gun-ball linkage device, a terminal and a storage medium. BACKGROUND

[0002] The gun-ball linkage monitoring mode is to monitor a certain area by using a wide-view gun camera, identify targets of interest in the monitored area such as pedestrians, vehicles, non-motor vehicles, etc., and the ball camera identifies, tracks and enlarges the targets of interest in the gun camera monitoring area. The gun-ball linkage monitoring mode needs to obtain the coordinate correlation between the gun camera, the ball camera and the on-site environment, the gun camera identifies and tracks the target in the collected image, determines the coordinate relationship of the target, and sends corresponding control instructions to the ball camera according to the coordinate correlation with the ball camera, so as to control the movement of the ball camera holder, and realize tracking of the target.

[0003] The current gun-ball linkage monitoring mode mainly tracks the target based on the matching degree of the feature information of the target in the gun camera image and the ball camera image, and lacks support for scenes where there are certain differences in target features in different pictures of the gun camera and the ball camera or scenes where there is target occlusion. SUMMARY

[0004] The present application provides a target tracking method and device of a gun-ball linkage device, a terminal and a storage medium.

[0005] One of the technical solutions adopted by the present application is to provide a target tracking method, which is applied to a gun-ball linkage device, wherein the target tracking method comprises:

[0006] Obtaining a plurality of continuous historical gun camera monitoring images, identifying the plurality of continuous historical gun camera monitoring images, and obtaining a gun camera monitoring position of a target;

[0007] Mapping the gun camera monitoring position of the target to a plurality of continuous historical ball camera monitoring images to obtain a ball camera monitoring position of the target;

[0008] Based on the ball camera monitoring position of the target, obtaining a ball camera running track of the target, and predicting a first current ball camera monitoring position of the target based on the ball camera running track of the target;

[0009] Obtaining a current gun camera monitoring image, and obtaining a second current ball camera monitoring position based on the current gun camera monitoring image;

[0010] Based on the first current ball camera monitoring position and the second current ball camera monitoring position, controlling the ball camera to track the target.

[0011] Prior to the step of mapping the target's camera position to a series of historical PTZ camera images, the procedure includes:

[0012] The bullet camera and the PTZ camera are calibrated to obtain the cross-camera association relationship between the bullet camera and the PTZ camera.

[0013] The step of mapping the target's camera position onto a series of historical PTZ camera images includes:

[0014] Based on the cross-camera association, the bullet camera monitoring position of the target is mapped onto historical PTZ camera monitoring images of multiple consecutive frames.

[0015] The process of mapping the target's camera position onto a series of historical PTZ camera images to obtain the target's PTZ camera position, and obtaining the target's PTZ camera trajectory based on the target's PTZ camera position, includes:

[0016] Based on the target's bolt monitoring position, the bolt movement trajectory of the target in the historical bolt monitoring images of the target in the consecutive frames is obtained;

[0017] The target's camera movement trajectory is mapped onto the continuous multi-frame historical PTZ camera monitoring images to obtain the target's PTZ camera movement trajectory, wherein the target's PTZ camera movement trajectory includes the target's PTZ camera monitoring position.

[0018] The step of predicting the first current PTZ camera monitoring position of the target based on the PTZ camera's trajectory includes:

[0019] In response to a preset target tracking algorithm, the position of the target on the historical PTZ camera monitoring images of the target in the continuous multi-frame PTZ camera running trajectory is tracked, and the first current PTZ camera monitoring position of the target is predicted on the current PTZ camera monitoring image.

[0020] The process of acquiring the current bullet camera's monitoring image and acquiring the second current PTZ camera's monitoring position based on the current bullet camera's monitoring image includes:

[0021] Obtain the current gun monitoring position of the target in the current gun monitoring image;

[0022] The current bullet camera monitoring position is mapped onto the current PTZ camera monitoring image to obtain the second current PTZ camera monitoring position of the target.

[0023] The step of controlling the PTZ camera to track the target based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position includes:

[0024] In response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being less than a set threshold, the PTZ camera is controlled to autonomously track the target.

[0025] The method of controlling the PTZ camera to track the target based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position includes:

[0026] In response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being greater than a set threshold, the PTZ camera's running trajectory is updated based on the second current PTZ camera monitoring position, and the PTZ camera is controlled to track the target according to the updated PTZ camera running trajectory.

[0027] Another technical solution adopted in this application is to provide a target tracking device based on a gun-ball linkage device, wherein the target tracking device includes an identification module, a mapping module, a prediction module, a monitoring module, and a tracking module; wherein...

[0028] The identification module is used to acquire multiple consecutive frames of historical gun monitoring images, identify the multiple consecutive frames of historical gun monitoring images, and obtain the gun monitoring position of the target.

[0029] The mapping module is used to map the target's bullet camera monitoring position onto a series of historical PTZ camera monitoring images to obtain the target's PTZ camera monitoring position.

[0030] The prediction module is used to obtain the PTZ camera's running trajectory based on the PTZ camera's monitoring position of the target, and predict the first current PTZ camera monitoring position of the target based on the PTZ camera's running trajectory.

[0031] The monitoring module is used to acquire the current bullet camera monitoring image and acquire the second current PTZ camera monitoring position based on the current bullet camera monitoring image;

[0032] The tracking module is used to control the PTZ camera to track the target based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position.

[0033] Another technical solution adopted in this application is to provide a terminal device, the terminal device including a memory and a processor coupled to the memory;

[0034] The memory is used to store program data, and the processor is used to execute the program data to implement the data transmission method described above.

[0035] Another technical solution adopted in this application is to provide a computer storage medium for storing program data, which, when executed by a computer, is used to implement the data transmission method described above.

[0036] The beneficial effects of this application are: adding a comparison and confirmation step in the association process between the bullet camera and the PTZ camera, realizing a stable handover process between the bullet camera and the PTZ camera, and at the same time, the target features and target movement trajectory acquired by the bullet camera during the linkage process provide more information for the active tracking of the PTZ camera, reducing the phenomenon of discontinuous or failed tracking of the PTZ camera due to target obstruction. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating an embodiment of the target tracking method for the gun-ball linkage device provided in this application;

[0039] Figure 2 yes Figure 1 A flowchart illustrating the S4 sub-step.

[0040] Figure 3 This is a flowchart illustrating another embodiment of the target tracking method for the gun-ball linkage device provided in this application;

[0041] Figure 4 This is a schematic diagram of an embodiment of the target tracking device provided in this application;

[0042] Figure 5 This is a schematic diagram of the structure of an embodiment of the terminal device provided in this application;

[0043] Figure 6 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0045] In traditional target tracking methods for camera-tube cameras, the camera and the PTZ camera acquire images containing target features respectively. By comparing the feature differences of the target between different images, the PTZ camera can achieve autonomous tracking of the target. However, this method may result in poor tracking performance when there are certain differences in the target image features acquired by the camera and the PTZ camera or when the target is partially occluded by the environment.

[0046] This application mainly designs a target tracking method for a camera-tube camera linkage device. To address the shortcomings of traditional methods, this application adds target motion trajectory information analysis and prediction to the original method. A comparison and confirmation step is added during the association process between the camera and the tube camera to reduce the target tracking loss of the tube camera when there are certain differences in the target image features acquired by the camera and the tube camera. This achieves stable handover between the camera and the tube camera. The acquired target features and target trajectory information provide more information for the tube camera's tracking. The tube camera still has accurate tracking capability even when the target is partially occluded by the environment.

[0047] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the target tracking method for the gun-ball linkage device provided in this application.

[0048] like Figure 1 As shown, the target tracking method of the gun-ball linkage device provided in this application may include the following steps:

[0049] S1, acquire multiple consecutive frames of historical gun monitoring images, identify multiple consecutive frames of historical gun monitoring images, and obtain the gun monitoring position of the target.

[0050] In one embodiment, historical images of the camera module are acquired, and target features and target location information are extracted based on the historical images of the camera module.

[0051] Optionally, the target may include, but is not limited to, one or more of vehicles, pedestrians, and non-motorized vehicles.

[0052] Optionally, the target feature information can be the convolutional features obtained by inputting the image region corresponding to the target into a pre-trained convolutional neural network.

[0053] In one embodiment, the camera acquires continuous monitoring images from frame 2 to frame N, and extracts target features and target location information from the aforementioned continuous N-1 frames of historical monitoring images.

[0054] Optionally, the target location information can be the pixel location information of the target in the gun image.

[0055] Prior to S1, the target tracking method in this application embodiment may further include the following steps: calibrating the bullet camera and the PTZ camera to obtain the cross-camera association relationship between the bullet camera and the PTZ camera.

[0056] In one embodiment, the target tracking device calibrates the bullet camera and the PTZ camera respectively, and obtains the camera intrinsic parameters, extrinsic parameters and distortion coefficients of the bullet camera and the PTZ camera respectively, thereby establishing the correlation between the pixel position of the target in the image obtained by the bullet camera and the PTZ camera and the real position of the target in the real environment.

[0057] Optionally, the bullet camera and the PTZ camera are calibrated using a checkerboard.

[0058] In one embodiment, the target tracking device can acquire monitoring images corresponding to bullet cameras and PTZ cameras respectively, select one or more fixed targets in the bullet camera monitoring image and the PTZ camera monitoring image as comparison targets, construct a coordinate transformation algorithm, and obtain the cross-camera association relationship between the bullet camera and the PTZ camera.

[0059] Optionally, the comparison target can be a building in the monitored scene.

[0060] Optionally, the comparison target can be a stationary vehicle in the monitored scene.

[0061] S2 maps the target's camera position onto a series of historical PTZ camera images to obtain the target's PTZ camera position.

[0062] In some embodiments, when the target can be obtained from both the bullet camera monitoring image and the PTZ camera monitoring image, the step of mapping the target's bullet camera monitoring position to a series of historical PTZ camera monitoring images is performed.

[0063] In some embodiments, if at least one of the bullet camera monitoring image and the PTZ camera monitoring image fails to acquire the target, the step of mapping the target's bullet camera monitoring position to a series of historical PTZ camera monitoring images is not performed, and the process jumps back to S1.

[0064] This includes mapping the target's camera position to multiple consecutive frames of historical PTZ camera images, including:

[0065] Based on cross-camera correlation, the target's bullet camera monitoring position is mapped onto historical PTZ camera monitoring images across multiple consecutive frames.

[0066] The cross-camera association between the bullet camera and the PTZ camera can be calculated from the calibration parameters of the bullet camera and the PTZ camera. Specifically, the coordinate position of the calibration object in the calibration image of the bullet camera and the coordinate position of the calibration object in the calibration image of the PTZ camera can be calculated from the same calibration object in the real environment. Then, the cross-camera association between the bullet camera and the PTZ camera can be calculated based on the two coordinate positions.

[0067] Specifically, the target tracking device acquires the target's position information in the continuous monitoring images from frame 2 to frame N from the bullet camera. Through the cross-camera association established between the bullet camera and the PTZ camera, it converts the target's monitoring position in the bullet camera for N-1 consecutive frames to the target's monitoring position in the PTZ camera for N-1 consecutive frames.

[0068] Through the above steps, the target monitoring location information acquired by the bullet camera is mapped to the target monitoring location in the PTZ camera image.

[0069] S3, based on the target's PTZ camera monitoring position, obtain the target's PTZ camera running trajectory, and predict the target's first current PTZ camera monitoring position based on the target's PTZ camera running trajectory.

[0070] The target's PTZ camera trajectory includes the PTZ camera's monitoring position on multiple consecutive frames of PTZ camera monitoring images. One method for obtaining the target's PTZ camera trajectory is to sequentially map the monitoring positions on the bullet camera images to the PTZ camera images, and then generate the target's PTZ camera trajectory by arranging the monitoring positions on multiple consecutive frames of PTZ camera images in chronological order. Another method is to generate the target's bullet camera trajectory by arranging the monitoring positions on multiple consecutive bullet camera images in chronological order, and then map the target's bullet camera trajectory onto the PTZ camera images to generate the target's PTZ camera trajectory.

[0071] The prediction of the target's first current PTZ camera monitoring position based on the target's PTZ camera trajectory includes the following steps:

[0072] In response to a preset target tracking algorithm, the target's position on historical PTZ camera monitoring images in multiple consecutive frames is tracked based on the target's PTZ camera trajectory, and the first current PTZ camera monitoring position of the target is predicted on the current PTZ camera monitoring image.

[0073] In one embodiment, the preset target tracking algorithm can be a single target tracking algorithm, which obtains the target's position information in multiple consecutive frames and then obtains the target's trajectory.

[0074] Optionally, the single-target tracking algorithm may include a model-matching-based target tracking algorithm, such as the mean-shift method or the particle filter method.

[0075] Optionally, the single-target tracking algorithm may include deep learning-based target tracking algorithms, such as target tracking algorithms based on convolutional neural networks, target tracking algorithms based on Siamese neural networks + Hungarian algorithm, and target tracking algorithms combining Siamese neural networks and region recommendation networks.

[0076] In one embodiment, the target tracking device predicts the target's position information in the (N+1)th frame of the PTZ camera monitoring image based on the acquired target trajectory, i.e., the first current PTZ camera monitoring position.

[0077] Optionally, the method for predicting the target position information in the N+1th frame of the PTZ camera monitoring image can be a trajectory prediction method such as Kalman filtering or velocity averaging.

[0078] Optionally, the target's location information may include the pixel value at the center of the target's bounding box.

[0079] S4, acquire the current bullet camera monitoring image, and acquire the second current PTZ camera monitoring position based on the current bullet camera monitoring image.

[0080] In one embodiment, the target tracking device converts the acquired current N+1 frame bullet camera monitoring image into a second current PTZ camera monitoring position through the established cross-camera association relationship between the bullet camera and the PTZ camera.

[0081] Please see Figure 2 , Figure 2 yes Figure 1 A flowchart illustrating the S4 sub-step.

[0082] like Figure 2 As shown, the sub-steps of S4 may include:

[0083] S41, obtain the current gun monitoring position of the target in the current gun monitoring image.

[0084] Specifically, the camera follows the target and acquires the target's position information in the (N+1)th frame of the monitored image.

[0085] S42, map the current bullet camera monitoring position onto the current PTZ camera monitoring image to obtain the target's second current PTZ camera monitoring position.

[0086] Specifically, the target tracking device converts the pixel position value of the target in the currently acquired bullet camera monitoring image into the position information of the target in the N+1th frame monitoring image of the PTZ camera through the cross-camera relationship established between the bullet camera and the PTZ camera, which is the second current PTZ camera monitoring position.

[0087] Optionally, the target's location information may include the pixel value at the center of the target's bounding box.

[0088] Please continue reading. Figure 1 .

[0089] S5, based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position, controls the PTZ camera to track the target.

[0090] Specifically, the target tracking device controls whether the PTZ camera adjusts its pan-tilt head to track the target by comparing information from the first current PTZ camera monitoring position and the second current PTZ camera monitoring position, and obtains the PTZ camera monitoring image of the tracked target in real time.

[0091] Specifically, in response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being less than a set threshold, the PTZ camera is controlled to autonomously track the target.

[0092] In one embodiment, if the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position is less than a set threshold, the handover between the bullet camera and the PTZ camera is realized, and the image features and motion trajectory information of the target are updated, and the PTZ camera is controlled to autonomously track the target.

[0093] Optionally, the positional difference can be the Manhattan distance between the center of the target frame in the first current PTZ camera monitoring position and the center of the target frame in the second current PTZ camera monitoring position.

[0094] Optionally, the positional difference can be the Euclidean distance between the center of the target frame in the first current PTZ camera monitoring position and the center of the target frame in the second current PTZ camera monitoring position.

[0095] Specifically, in response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being greater than a set threshold, the PTZ camera's running trajectory is updated based on the second current PTZ camera monitoring position, and the PTZ camera is controlled to track the target according to the updated PTZ camera running trajectory.

[0096] In one embodiment, if the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position is greater than a set threshold, the target tracking device uses the second current PTZ camera monitoring position as the updated content of the PTZ camera's motion trajectory to re-control the PTZ camera to track the target.

[0097] Optionally, the positional difference can be the Manhattan distance between the center of the target frame in the first current PTZ camera monitoring position and the center of the target frame in the second current PTZ camera monitoring position.

[0098] Optionally, the positional difference can be the Euclidean distance between the center of the target frame in the first current PTZ camera monitoring position and the center of the target frame in the second current PTZ camera monitoring position.

[0099] Please see Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of the target tracking method for the gun-ball linkage device provided in this application.

[0100] like Figure 3 As shown, another embodiment of the target tracking method for the gun-ball linkage device may include the following steps:

[0101] S100: Acquire the first frame of the gun's monitoring image, and obtain target features and target location information.

[0102] In one embodiment, a first frame of gun monitoring image is acquired, and target features and target location information are extracted based on the first frame of gun monitoring image.

[0103] Optionally, the target may include, but is not limited to, one or more of vehicles, pedestrians, and non-motorized vehicles.

[0104] Optionally, the target feature information can be the convolutional features obtained by inputting the image region corresponding to the target into a pre-trained convolutional neural network.

[0105] Optionally, the target location information can be the target's location information in the gun image.

[0106] Before S100, the following steps may also be included: calibrating the bullet camera and PTZ camera to obtain the cross-camera association relationship between the bullet camera and PTZ camera.

[0107] In one embodiment, the target tracking device calibrates the bullet camera and the PTZ camera respectively, and obtains the camera intrinsic parameters, extrinsic parameters and distortion coefficients of the bullet camera and the PTZ camera respectively, thereby establishing the correlation between the pixel position of the target in the image obtained by the bullet camera and the PTZ camera and the real position of the target in the real environment.

[0108] Optionally, the bullet camera and the PTZ camera are calibrated using a checkerboard.

[0109] In one embodiment, the bullet camera and the PTZ camera can acquire corresponding monitoring images respectively. One or more fixed targets in the bullet camera monitoring image and the PTZ camera monitoring image are selected as comparison targets. A coordinate transformation algorithm is constructed to obtain the cross-camera association relationship between the bullet camera and the PTZ camera.

[0110] S200: Acquire the 2nd to k-1th frame monitoring images of the PTZ camera within multiple consecutive time periods, obtain the target position information of the target through the tracking algorithm, and obtain the corresponding PTZ camera monitoring position based on the target position information in different frames through coordinate transformation of the cross-camera relationship between the PTZ camera and the PTZ camera.

[0111] In one embodiment, the target tracking device acquires the target's position information in the continuous monitoring images from frame 2 to frame k-1 obtained by the bullet camera, and through the cross-camera association established between the bullet camera and the PTZ camera, converts the target's monitoring position in the bullet camera for consecutive frames k-2 to the target's monitoring position in the PTZ camera for consecutive frames k-2.

[0112] Through the above steps, the target monitoring location information acquired by the bullet camera is mapped to the target monitoring location in the PTZ camera image.

[0113] The S300, based on the PTZ camera coordinate transformation relationship, transforms the target's coordinates within the PTZ camera's view to the same coordinate system according to the PTZ camera's monitoring position information, and updates the corresponding motion trajectory information.

[0114] Since the PTZ coordinate information of the target differs under different PTZ camera monitoring positions, coordinate transformation can convert the target's PTZ information into PTZ camera coordinate information.

[0115] Specifically, when the PTZ camera is in different monitoring positions, the monitoring direction changes, resulting in inconsistent coordinate systems for multiple frames of PTZ monitoring images. In this case, the target tracking device can transform the target coordinates in multiple consecutive frames of PTZ monitoring images to unify them into the same coordinate system, facilitating the generation of the target's PTZ camera trajectory.

[0116] In one embodiment, target motion trajectory information can be obtained through a single-target tracking algorithm.

[0117] Optionally, the single-target tracking algorithm may include a model-matching-based target tracking algorithm, such as the mean-shift method or the particle filter method.

[0118] Optionally, the single-target tracking algorithm may include deep learning-based target tracking algorithms, such as target tracking algorithms based on convolutional neural networks, target tracking algorithms based on Siamese neural networks + Hungarian algorithm, and target tracking algorithms combining Siamese neural networks and region recommendation networks.

[0119] S400 predicts the target position information corresponding to the kth frame based on motion trajectory information, and obtains the first PTZ camera monitoring position through the PTZ camera coordinate transformation relationship.

[0120] In one embodiment, the target tracking device predicts the target's position information in the k-th frame PTZ camera monitoring image, i.e., the first current PTZ camera monitoring position, based on the target's running trajectory obtained from the 2nd frame to the (k-1)th frame of the PTZ camera.

[0121] Optionally, the method for predicting the target position information in the k-th frame PTZ camera monitoring image can be a trajectory prediction method such as Kalman filtering or velocity averaging.

[0122] Optionally, the target's location information may include the pixel value at the center of the target's bounding box.

[0123] S500: Obtain the k-th frame of the PTZ camera's monitoring image, acquire the target's location information, and obtain the second PTZ camera's monitoring position through the coordinate transformation relationship between the PTZ camera and the PTZ camera.

[0124] In one embodiment, the camera follows the target and acquires the target's position information in the current camera's k-th frame monitoring image.

[0125] The target tracking device uses the cross-camera relationship established between the bullet camera and the PTZ camera to convert the pixel position value of the target in the currently acquired bullet camera monitoring image into the position information of the target in the k-th frame monitoring image of the PTZ camera, which is the second current PTZ camera monitoring position.

[0126] Optionally, the target's location information may include the pixel value at the center of the target's bounding box.

[0127] S600, is the difference between the monitoring position of the first PTZ camera and the monitoring position of the second PTZ camera within a preset threshold?

[0128] In one embodiment, if the difference between the monitoring position of the first PTZ camera and the second PTZ camera is within a preset threshold, the process jumps to S800; if the difference between the monitoring position of the first PTZ camera and the second PTZ camera is not within the preset threshold, the process jumps to S700.

[0129] Optionally, the positional difference can be the Manhattan distance between the center of the target frame in the first current PTZ camera monitoring position and the center of the target frame in the second current PTZ camera monitoring position.

[0130] Optionally, the positional difference can be the Euclidean distance between the center of the target frame in the first current PTZ camera monitoring position and the center of the target frame in the second current PTZ camera monitoring position.

[0131] S700, k = k + 1.

[0132] In one embodiment, the target tracking device updates the PTZ camera's trajectory based on the second current PTZ camera monitoring position, and controls the PTZ camera to track the target according to the updated PTZ camera trajectory.

[0133] Jump to S300.

[0134] S800, PTZ camera with autonomous tracking.

[0135] In one embodiment, the target tracking device performs the handover between the bullet camera and the PTZ camera according to the target confirmation process described above, updates the target's image features and motion trajectory information, and controls the PTZ camera to autonomously track the target.

[0136] The above embodiments are merely one common example of this application and do not constitute any limitation on the technical scope of this application. Therefore, any minor modifications, equivalent changes, or alterations made to the above content based on the substance of the solution of this application shall still fall within the scope of the technical solution of this application.

[0137] Please continue reading Figure 4 , Figure 4 This is a schematic diagram of the structure of an embodiment of the target tracking device provided in this application. The target tracking device 500 of this application embodiment includes an identification module 51, a mapping module 52, a prediction module 53, a monitoring module 54, and a tracking module 55.

[0138] The identification module 51 is used to acquire multiple consecutive frames of historical gun monitoring images, identify multiple consecutive frames of historical gun monitoring images, and obtain the gun monitoring position of the target.

[0139] The mapping module 52 is used to map the target's bullet camera monitoring position onto a series of historical PTZ camera monitoring images to obtain the target's PTZ camera monitoring position.

[0140] The prediction module 53 is used to obtain the target's PTZ camera trajectory based on the target's PTZ camera monitoring position, and predict the target's first current PTZ camera monitoring position based on the target's PTZ camera trajectory.

[0141] The monitoring module 54 is used to acquire the current bullet camera monitoring image and acquire the second current PTZ camera monitoring position based on the current bullet camera monitoring image.

[0142] The tracking module 55 is used to control the PTZ camera to track the target based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position.

[0143] Please continue reading Figure 5 , Figure 5 This is a schematic diagram of the structure of an embodiment of the terminal device provided in this application. The terminal device 600 of this application embodiment includes a processor 61 and a memory 62.

[0144] The processor 61 and the memory 62 are connected to the bus. The memory 62 stores program data, and the processor 61 is used to execute the program data to implement the target tracking method described in the above embodiments.

[0145] In this embodiment, processor 61 can also be referred to as a CPU (Central Processing Unit). Processor 61 may be an integrated circuit chip with signal processing capabilities. Processor 61 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 61 can be any conventional processor.

[0146] This application also provides a computer storage medium; please refer to the following: Figure 6 , Figure 6This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 700 stores program data 71, which is used to implement the target tracking method of the above embodiment when executed by the processor.

[0147] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A target tracking method based on a gun-ball linkage device, wherein the gun-ball linkage device includes a gun camera and a ball camera, characterized in that, The target tracking method includes: Acquire consecutive frames of historical gun monitoring images, identify the consecutive frames of historical gun monitoring images, and obtain the gun monitoring position of the target; The target's camera monitoring position is mapped onto a series of historical PTZ camera monitoring images to obtain the target's PTZ camera monitoring position. Based on the PTZ camera monitoring location of the target, the PTZ camera running trajectory of the target is obtained, and the first current PTZ camera monitoring location of the target is predicted based on the PTZ camera running trajectory of the target; Acquire the current bullet camera monitoring image, and obtain the second current PTZ camera monitoring position based on the current bullet camera monitoring image; Based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position, the PTZ camera is controlled to track the target; The step of controlling the PTZ camera to track the target based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position includes: In response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being less than a set threshold, the PTZ camera is controlled to autonomously track the target; In response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being greater than a set threshold, the PTZ camera's running trajectory is updated based on the second current PTZ camera monitoring position, and the PTZ camera is controlled to track the target according to the updated PTZ camera running trajectory.

2. The method according to claim 1, characterized in that, Before the step of mapping the target's camera position to a series of historical PTZ camera images, the following steps are included: The bullet camera and the PTZ camera are calibrated to obtain the cross-camera association relationship between the bullet camera and the PTZ camera; The step of mapping the target's camera position to a series of historical PTZ camera images includes: Based on the cross-camera association, the bullet camera monitoring position of the target is mapped onto historical PTZ camera monitoring images of multiple consecutive frames.

3. The method according to claim 1 or 2, characterized in that, The target's camera monitoring position is obtained by mapping the target's bullet camera monitoring position onto a series of historical PTZ camera monitoring images. And, based on the PTZ camera's monitoring location of the target, obtain the PTZ camera's trajectory, including: Based on the target's bolt monitoring position, the bolt movement trajectory of the target in the historical bolt monitoring images of the target in the consecutive frames is obtained; The target's camera movement trajectory is mapped onto the continuous multi-frame historical PTZ camera monitoring images to obtain the target's PTZ camera movement trajectory, wherein the target's PTZ camera movement trajectory includes the target's PTZ camera monitoring position.

4. The method according to claim 1, characterized in that, The prediction of the first current PTZ camera monitoring position of the target based on the PTZ camera's trajectory includes: In response to a preset target tracking algorithm, the position of the target on the historical PTZ camera monitoring images of the target in the continuous multi-frame PTZ camera running trajectory is tracked, and the first current PTZ camera monitoring position of the target is predicted on the current PTZ camera monitoring image.

5. The method according to claim 1 or 4, characterized in that, Acquire the current bullet camera monitoring image, and based on the current bullet camera monitoring image, obtain the second current PTZ camera monitoring position, including: Obtain the current gun monitoring position of the target in the current gun monitoring image; The current bullet camera monitoring position is mapped onto the current PTZ camera monitoring image to obtain the second current PTZ camera monitoring position of the target.

6. A target tracking device based on a gun-ball linkage device, characterized in that, The target tracking device includes an identification module, a mapping module, a prediction module, a monitoring module, and a tracking module; wherein, The identification module is used to acquire multiple consecutive frames of historical gun monitoring images, identify the multiple consecutive frames of historical gun monitoring images, and obtain the gun monitoring position of the target. The mapping module is used to map the target's bullet camera monitoring position onto a series of historical PTZ camera monitoring images to obtain the target's PTZ camera monitoring position. The prediction module is used to obtain the PTZ camera's running trajectory based on the PTZ camera's monitoring position of the target, and predict the first current PTZ camera monitoring position of the target based on the PTZ camera's running trajectory. The monitoring module is used to acquire the current bullet camera monitoring image and acquire the second current PTZ camera monitoring position based on the current bullet camera monitoring image; The tracking module is used to control the PTZ camera to track the target based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position; The step of controlling the PTZ camera to track the target based on the first current PTZ camera monitoring position and the second current PTZ camera monitoring position includes: In response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being less than a set threshold, the PTZ camera is controlled to autonomously track the target; In response to the positional difference between the first current PTZ camera monitoring position and the second current PTZ camera monitoring position being greater than a set threshold, the PTZ camera's running trajectory is updated based on the second current PTZ camera monitoring position, and the PTZ camera is controlled to track the target according to the updated PTZ camera running trajectory.

7. A terminal device, characterized in that, The terminal device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the target tracking method as described in any one of claims 1 to 5.

8. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the target tracking method as described in any one of claims 1 to 5.

Images