Target Tracking Method, Device, Computer Equipment, and Storage Medium

Abstract

The present application relates to a target tracking method, apparatus, computer device, and storage medium. By fusing the predicted trajectories of the same target to be tracked in multiple current frame images collected by camera devices with different perspectives, a fused predicted trajectory of each target to be tracked is obtained. Then, the first features of each target to be tracked detected in the current frame image are matched with the features of the corresponding fused predicted trajectory, and each target to be tracked is tracked based on the successfully matched first features. This method saves resources and time and improves the tracking efficiency when performing target tracking based on multiple images.

Description

Technical Field

[0001] This application relates to the technical field of target tracking, and particularly to a target tracking method, apparatus, computer device, and storage medium. Background Art

[0002] With the development of technology, target tracking has been widely applied in more and more fields, such as intelligent monitoring, human-computer interaction, video retrieval, etc.

[0003] Generally, when performing target tracking, after obtaining the entire image in the target scene, target tracking is performed based on the entire image. Usually, the acquisition range of ordinary RGB camera devices is limited, and only images within a limited range can be acquired for the same scene. Then, these multiple images need to be stitched together to form a single image, and then the target in the target scene is tracked according to the synthesized image.

[0004] However, the above target tracking process is relatively complex, resulting in low efficiency when performing target tracking based on multiple images. Summary of the Invention

[0005] Based on this, it is necessary to provide a target tracking method, apparatus, computer device, and storage medium that can improve the tracking efficiency when performing target tracking based on multiple images for the above technical problems.

[0006] In a first aspect, an embodiment of this application provides a target tracking method, which includes:

[0007] Fuse the predicted trajectories of the same target to be tracked in multiple current-frame images to obtain the fused predicted trajectories of each target to be tracked; the multiple current-frame images are images collected by camera devices from different perspectives;

[0008] Match the first feature of each target to be tracked with the feature of the corresponding fused predicted trajectory; the first feature is the feature of the target to be tracked obtained by performing target detection on the current-frame image;

[0009] Track the target to be tracked according to the successfully matched first feature.

[0010] In one of the embodiments, the method further includes:

[0011] Detect each target to be tracked from multiple current-frame images collected by camera devices from different perspectives to obtain the detection frames of each target to be tracked;

[0012] Extract the image features in the detection frames of each target to be tracked as the first feature.

[0013] In one embodiment, before fusing the predicted trajectories of the same target to be tracked in multiple current frame images, the method further includes:

[0014] Predicting the predicted trajectory of each target to be tracked in each current frame image through a preset tracking algorithm model, where the tracking algorithm model is constructed based on a preset state space equation.

[0015] In one embodiment, the above-mentioned preset state space equation includes the space equation of uniformly variable motion state and the space equation of uniform motion state;

[0016] If the tracking algorithm model is constructed based on the space equation of uniformly variable motion state, the tracking algorithm model is used to predict the trajectory of the target in the image that has not been completely tracked; if the tracking algorithm model is constructed based on the space equation of uniform motion state, the tracking algorithm model is used to update the trajectory of the target in the image that has been completely tracked.

[0017] In one embodiment, the above-mentioned matching of the first feature of each target to be tracked with the feature of the corresponding fused predicted trajectory includes:

[0018] Obtaining the current number of consecutive tracking failures of each target to be tracked; where the current consecutive tracking failure means that the first feature of the target to be tracked and the feature of the fused predicted trajectory have continuously failed to match before the current frame;

[0019] Sequentially matching the first feature of each target to be tracked with the feature of the corresponding fused predicted trajectory in ascending order of the current number of consecutive tracking failures.

[0020] In one embodiment, the above-mentioned sequential matching of the first feature of each target to be tracked with the feature of the corresponding fused predicted trajectory includes:

[0021] Sequentially calculating the similarity between the feature of the fused predicted trajectory of each target to be tracked and all the first features;

[0022] Obtaining the first feature corresponding to the minimum similarity of each target to be tracked;

[0023] If the first features corresponding to the minimum similarity of all targets to be tracked are all different, the first feature corresponding to the minimum similarity of each target to be tracked successfully matches the feature of the fused predicted trajectory;

[0024] If there are at least two first features corresponding to the minimum similarity of the targets to be tracked that are the same, obtain the second feature, and based on the intersection over union between the second feature and the fused predicted trajectory of each target to be tracked, match the second feature with the fused predicted trajectory of each target to be tracked; where the second feature is the first feature that has not been successfully matched.

[0025] In one embodiment, the second feature includes other first features other than the first feature corresponding to the minimum similarity, and / or the same first features among the first features corresponding to the minimum similarity of all the targets to be tracked.

[0026] In one embodiment, matching the second feature with the fusion prediction trajectories of each target to be tracked based on the intersection over union between the second feature and the fusion prediction trajectories of each target to be tracked includes:

[0027] Calculating the intersection over union between the fusion prediction trajectory of each target to be tracked and each first feature in the second feature in sequence;

[0028] Obtaining the first feature corresponding to the minimum intersection over union of each target to be tracked;

[0029] If the first features corresponding to the minimum intersection over union of all the targets to be tracked are all different, the first features corresponding to the minimum intersection over union of each target to be tracked are successfully matched with the fusion prediction trajectory.

[0030] In one embodiment, before tracking the target to be tracked according to the successfully matched first feature, the method further includes:

[0031] Storing the successfully matched first feature into the prediction trajectory list corresponding to the first feature in the feature pool, where the feature pool includes the corresponding relationship between the first feature of each target to be tracked successfully matched in each frame of image and the fusion prediction trajectory.

[0032] In one embodiment, after storing the successfully matched first feature into the prediction trajectory list corresponding to the first feature in the feature pool, the method further includes:

[0033] Managing the first feature of each target to be tracked in the feature pool according to a preset storage condition; the storage condition is used to control the number of the first features of each target to be tracked in the feature pool.

[0034] In a second aspect, an embodiment of the present application provides a target tracking device, which includes:

[0035] A fusion module, configured to fuse the prediction trajectories of the same target to be tracked in multiple current frame images to obtain the fusion prediction trajectory of each target to be tracked; the multiple current frame images are images collected by camera devices with different perspectives;

[0036] A matching module, configured to match the first feature of each target to be tracked with the feature of the corresponding fusion prediction trajectory; the first feature is the feature of the target to be tracked obtained by performing target detection on the current frame image;

[0037] A tracking module for tracking a target to be tracked according to the successfully matched first feature.

[0038] In one embodiment, the apparatus includes:

[0039] A detection box acquisition module for detecting each target to be tracked from multiple current frame images collected by camera devices from different perspectives, and obtaining detection boxes of each target to be tracked;

[0040] A feature extraction module for extracting image features in the detection boxes of each target to be tracked as the first feature.

[0041] In one embodiment, the apparatus includes:

[0042] A prediction module for predicting the prediction trajectory of each target to be tracked in each current frame image through a preset tracking algorithm model, where the tracking algorithm model is constructed based on a preset state space equation.

[0043] In one embodiment, the above-mentioned preset state space equation includes a space equation for a uniformly variable motion state and a space equation for a uniform motion state; if the tracking algorithm model is constructed based on the space equation for a uniformly variable motion state, the tracking algorithm model is used to predict the trajectory of the target in the image that has not been completely tracked; if the tracking algorithm model is constructed based on the space equation for a uniform motion state, the tracking algorithm model is used to update the trajectory of the target in the image that has been completely tracked.

[0044] In one embodiment, the above-mentioned matching module includes:

[0045] A failure times unit for obtaining the current consecutive tracking failure times of each target to be tracked; where the current consecutive tracking failure means that the first feature of the target to be tracked and the feature of the fusion prediction trajectory have continuously failed to match before the current frame;

[0046] An order matching unit for sequentially matching the first feature of each target to be tracked with the feature of the corresponding fusion prediction trajectory in ascending order of the current consecutive tracking failure times.

[0047] In one embodiment, the above-mentioned sequential matching unit is specifically configured to calculate the similarity between the features of the fusion prediction trajectories of each target to be tracked and all the first features in sequence; obtain the first feature corresponding to the minimum similarity of each target to be tracked; if the first features corresponding to the minimum similarity of all the targets to be tracked are all different, then the first feature corresponding to the minimum similarity of each target to be tracked successfully matches the feature of the fusion prediction trajectory; if there are at least two first features corresponding to the minimum similarity of the targets to be tracked that are the same, then obtain the second feature, and based on the intersection-over-union ratio between the second feature and the fusion prediction trajectories of each target to be tracked, match the second feature with the fusion prediction trajectories of each target to be tracked; wherein, the second feature is the first feature that has not been successfully matched.

[0048] In one embodiment, the above-mentioned second feature includes other first features except the first feature corresponding to the minimum similarity, and / or, the same first features among the first features corresponding to the minimum similarity of all the targets to be tracked.

[0049] In one embodiment, the above-mentioned sequential matching unit is further specifically configured to calculate the intersection-over-union ratio between the fusion prediction trajectory of each target to be tracked and each first feature in the second feature in sequence; obtain the first feature corresponding to the minimum intersection-over-union ratio of each target to be tracked; if the first features corresponding to the minimum intersection-over-union ratio of all the targets to be tracked are all different, then the first feature corresponding to the minimum intersection-over-union ratio of each target to be tracked successfully matches the fusion prediction trajectory.

[0050] In one embodiment, the device further includes: a storage module, configured to store the successfully matched first feature into the prediction trajectory list corresponding to the first feature in the feature pool, and the feature pool includes the corresponding relationship between the first feature of each target to be tracked successfully matched in each frame of image and the fusion prediction trajectory.

[0051] In one embodiment, the device further includes: a management module, configured to manage the first feature of each target to be tracked in the feature pool according to a preset storage condition; the storage condition is used to control the number of the first features of each target to be tracked in the feature pool.

[0052] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of any method provided in the first aspect embodiment are implemented.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any method provided in the first aspect embodiment are implemented.

[0054] A target tracking method, device, computer device, and storage medium according to an embodiment of the present application fuse the predicted trajectories of the same target to be tracked in multiple current frame images collected by camera devices with different perspectives to obtain the fused predicted trajectories of each target to be tracked, and then match the first features of each target to be tracked detected in the current frame image with the features of the corresponding fused predicted trajectories, and track each target to be tracked according to the successfully matched first features. In this method, for each target to be tracked in multiple current frame images collected by camera devices with different perspectives, after predicting the predicted trajectory separately, the fused predicted trajectories of each target to be tracked are obtained through a fusion method, and the fused predicted trajectories are used to match the first features of each target to be tracked in the multiple current frame images, so as to realize the tracking of the targets to be tracked in the multiple current frame images. In the whole process, whether predicting the predicted trajectory of the target in the current frame image or detecting the first feature of the target in the current frame image, it is based on the individual image of the current frame, without splicing multiple images of the current frame, saving the splicing process, thus saving resources and time and improving the tracking efficiency when tracking targets based on multiple images. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is an application environment diagram of the target tracking method in an embodiment;

[0056] Figure 1a It is an application environment diagram of the target tracking method in an embodiment;

[0057] Figure 2 It is a schematic flowchart of the target tracking method in an embodiment;

[0058] Figure 3 It is a schematic flowchart of the target tracking method in another embodiment;

[0059] Figure 4 It is a schematic flowchart of the target tracking method in another embodiment; <y

[0060] Figure 5 It is a schematic flowchart of the target tracking method in another embodiment;

[0061] Figure 6 It is a schematic flowchart of the target tracking method in another embodiment;

[0062] Figure 7 It is a schematic diagram of the target tracking method in another embodiment;

[0063] Figure 8 It is a structural block diagram of the target tracking device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] The target tracking method provided by the present application can be applied to an application environment as Figure 1 shown. Among them, the computer device communicates with each camera device through a network. Among them, the computer device can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The camera devices include, but are not limited to, professional cameras, CCD cameras, network cameras, broadcast cameras, business cameras, home cameras, studio / live pedestal cameras, portable cameras, black and white cameras, color cameras, infrared cameras, X-ray cameras, surveillance cameras, undercover cameras, button cameras for reading words, etc. Among them, multiple camera devices can be the same or different, and no limitation is made in this regard.

[0066] Among them, for the internal structure diagram of the computer device, please refer to Figure 1a . The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data for target tracking. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a target tracking method.

[0067] The embodiments of the present application provide a target tracking method, device, computer device and storage medium, which can improve the tracking efficiency when tracking a target based on multiple images. The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below through embodiments and with reference to the accompanying drawings. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. It should be noted that a target tracking method provided by the present application, Figures 2 - 7 the execution subject of which is a computer device. Among them, the Figures 2 - 7 execution subject can also be a target tracking device, and the device can be implemented as part or all of a computer device in a software, hardware, or software-hardware combination manner.

[0068] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application.

[0069] In one embodiment, as Figure 2 shown, a target tracking method is provided. This embodiment involves the specific process in which a computer device predicts the trajectories of targets in multiple images respectively based on multiple images, then fuses the predicted trajectories of the same target to be tracked to obtain the fused predicted trajectory of the target to be tracked, and then matches the first feature of each target to be tracked in the multiple images according to the fused predicted trajectory, and tracks the target to be tracked with successful matching. This embodiment includes the following steps:

[0070] S101, fuse the predicted trajectories of the same target to be tracked in multiple current-frame images to obtain the fused predicted trajectories of each target to be tracked; the multiple current-frame images are images collected by camera devices with different perspectives.

[0071] Among them, the current-frame image refers to the image collected by the camera device at the current moment. The combination of each frame of image collected by the camera device at different moments is the video stream collected by the camera device. In practical applications, when tracking a target in a target scene, in some cases, the target scene can be monitored by camera devices with different perspectives relative to the target scene, that is, multiple camera devices can collect the video stream of the target scene in a certain time period from different perspectives. The target to be tracked is the target that needs to be tracked in the image.

[0072] Among them, the predicted trajectory refers to the predicted trajectory of each target in the current-frame image. For example, the trajectories of each target in the current-frame image are predicted according to the trajectories that have occurred for each target in the previous frame image. It should be noted here that the form of the trajectory in each frame of image is in the form of a rectangular frame, and the predicted trajectory can be called a predicted tracking frame. Since each camera device collects the video stream from a different perspective, for each frame of image in the video stream, the targets to be tracked in the target scene are repeated. For example, target No. 1 may appear in the video stream of each perspective, and target No. 2 may only appear in the video stream of several of the perspectives; therefore, after predicting the trajectories of the targets in all the current-frame images collected by each camera device, for the same target to be tracked (for example, there are multiple predicted trajectories for target No. 1), the predicted trajectories of the same target to be tracked need to be fused to obtain the final fused predicted trajectory of the target to be tracked. It can be understood that the fused predicted trajectory of each target to be tracked can reflect the complete trajectory of the target in the current video in the target scene.

[0073] S102. Match the first feature of each target to be tracked with the feature of the corresponding fused prediction trajectory. The first feature is the feature of the target to be tracked obtained by performing target detection on the current frame image.

[0074] Among them, the first feature is the feature of the target to be tracked obtained by performing target detection on multiple current frame images. For example, detection is performed through a preset target detection model, which can be a model based on a neural network, etc. This embodiment does not limit this. It can be understood that the first feature detects the targets existing in the current frame image, but it only detects the regions of interest and does not know which target each detected region of interest belongs to, that is, it does not know the specific identifiers of the detected targets. Therefore, if we want to achieve the tracking of each target in the current frame image, we must first determine which target each detected first feature belongs to and determine the identifier of the target to be tracked corresponding to each first feature.

[0075] Specifically, match the first feature of each target to be tracked with the feature of the corresponding fused prediction trajectory. Among them, when matching, it can be matched by similarity, or by intersection over union, etc., and other methods can also be used for matching. This embodiment does not limit this.

[0076] S103. Track the target to be tracked according to the successfully matched first feature.

[0077] In the above process, the identifiers of the targets to be tracked in the fused prediction trajectory have been determined. Therefore, if the first feature matches successfully with a certain fused prediction trajectory, it is considered that the identifier of the target to be tracked corresponding to this first feature is the identifier of the target to be tracked corresponding to this fused prediction trajectory. By matching in turn, the identifiers of the targets to be tracked corresponding to the detected first features are determined. Equivalently, after determining the identifier of each target to be tracked in the current frame image, update the first feature detected for each target to be tracked to the true trajectory of the target to be tracked in the current frame image, and complete the tracking of each target to be tracked in the current frame image.

[0078] The target tracking method provided in this embodiment obtains the fused prediction trajectories of each target to be tracked by fusing the prediction trajectories of the same target to be tracked in multiple current frame images collected by camera devices with different perspectives, and then matches the first features of each target to be tracked detected in the current frame image with the features of the corresponding fused prediction trajectories, and tracks each target to be tracked according to the successfully matched first features. In this method, for each target to be tracked in multiple current frame images collected by camera devices with different perspectives, after separately predicting the prediction trajectories, the fused prediction trajectories of each target to be tracked are obtained through a fusion method, and the fused prediction trajectories are used to match the first features of each target to be tracked in the multiple current frame images, so as to realize the tracking of the targets to be tracked in the multiple current frame images. In the whole process, whether predicting the prediction trajectories of the targets in the current frame image or detecting the first features of the targets in the current frame image, it is based on the individual images of the current frame, without splicing multiple images of the current frame, saving the splicing process, thereby saving resources and time and improving the tracking efficiency when tracking targets based on multiple images.

[0079] An embodiment of obtaining the first features of each target to be tracked in the current frame image is provided, as Figure 3 shown. In one embodiment, this embodiment includes:

[0080] S201. Detect each target to be tracked from multiple current frame images collected by camera devices with different perspectives, and obtain the detection frames of each target to be tracked.

[0081] In practical applications, the first feature may be the feature of the detection frame of the target to be tracked detected from each frame image. Therefore, each target to be tracked is first detected from multiple current frame images to obtain the detection frames of each target to be tracked. For example, the multiple current frame images are input into the target detection model to obtain the detection frames of each target to be tracked. Among them, the detection frames obtained in this embodiment can at least represent the position and size information of each target to be tracked, but cannot determine the identifier of each target to be tracked.

[0082] S202. Extract the image features in the detection frames of each target to be tracked as the first features.

[0083] After obtaining the detection boxes of each target to be tracked in multiple current frame images, the image features in each detection box are extracted as the first features of each target to be tracked. Among them, the extraction algorithm of the image features includes, but is not limited to, the Histogram of Oriented Gradient (HOG) formed by calculating and statistically analyzing the histogram of gradient directions in local regions of the image, Speeded Up Robust Features (SURF), which is an improvement of SIFT, and its main features are fast, Oriented FAST and Rotated BRIEF (ORB), which is an algorithm for fast feature point extraction and description, and Scale-invariant feature transform (SIFT), which is a description used in the field of image processing. Preferably, in this embodiment, the image feature is the HOG feature. Since the target tracking method provided by the embodiments of the present application can be applied to real-time videos, it is desired that the time for extracting the first features of each target to be tracked in multiple current frame images is as fast as possible. Therefore, it is necessary to minimize the GPU occupancy, and the HOG feature does not occupy GPU resources, so that the time for extracting the first features in this embodiment can meet the requirements of real-time videos.

[0084] In the target tracking method provided by the embodiments of the present application, by using the image features in the detection boxes of the targets in the detected images as the first features of the targets, on the one hand, the detection boxes can at least reflect the position and size information of each target to be tracked, and using the features in the detection boxes as the first features enables the first features to accurately represent each target to be tracked; on the other hand, when using the HOG feature as the image feature, it does not occupy GPU resources, so that the time for extracting the first features in this embodiment can meet the requirements of real-time videos.

[0085] For the prediction process of the predicted trajectories of each target to be tracked mentioned in the above embodiments, several embodiments will be described below. In one embodiment, the embodiment of obtaining the predicted trajectory of each target to be tracked in the current frame image includes: predicting the predicted trajectory of each target to be tracked in each current frame image through a preset tracking algorithm model, where the tracking algorithm model is constructed based on a preset state space equation.

[0086] Among them, the state-space equation is an expression that can reflect the changes in the target's trajectory at different times, motion information, etc. based on different motion states of the target in space; a tracking algorithm model constructed based on this state-space equation, such as a Kalman filter, can be closer to the real information when the target moves in space. After constructing the tracking algorithm model, predicting the predicted trajectory of each target to be tracked in each current frame image through this tracking algorithm model can make the predicted trajectory of each target to be tracked more accurate.

[0087] Optionally, in one embodiment, the preset state-space equation includes the space equation for uniformly variable motion state and the space equation for uniform motion state; if the tracking algorithm model is constructed based on the space equation for uniformly variable motion state, the tracking algorithm model is used to predict the trajectory of the target in the image that has not been completely tracked; if the tracking algorithm model is constructed based on the space equation for uniform motion state, the tracking algorithm model is used to update the trajectory of the target in the image that has been completely tracked.

[0088] Among them, when the tracking algorithm model is constructed based on the space equation for uniformly variable motion state, the space equation for uniformly variable motion state couples the motion of the target in space with time, takes into account the influence of acceleration, can make the trajectory prediction error smaller, and improves the tracking effect with large speed changes.

[0089] Exemplarily, taking the trajectories of each target at each moment as a rectangular box, in the first case, regarding the motion of the target in the image as uniformly variable motion, the following state-space equation (1) is constructed to reflect the information change when the target moves uniformly variably in the image.

[0090]

[0091] Among them, in the above formula, x′ and y′ represent the coordinates of the center point of the target's trajectory in the current frame image on the x-axis and y-axis of the image, x and y represent the coordinates of the center point of the target's trajectory on the x-axis and y-axis of the image before time t, represents the velocity of the same target in the x-axis and y-axis directions of the image before time t, represents the acceleration of the same target in the x-axis and y-axis directions of the image before time t; α′ and h′ represent the aspect ratio and height of the target's trajectory in the current frame image, α and h represent the aspect ratio and height of the target's trajectory before time t, represents the aspect ratio change rate and height change rate of the same target before time t.

[0092] Among them, the above-mentioned t represents the moment of tracking failure. Then, the trajectory coordinates before t are based on the last image that has been successfully tracked in the video stream, and the speed and acceleration refer to the average speed and average acceleration during a certain period before the moment t. Then, based on the above state space equation (1), the trajectory of the target in each image (each frame of the video stream) can be predicted. Since the state space equation (1) is coupled with time and the acceleration is considered, the influence of acceleration can be taken into account during the trajectory prediction stage, which can make the trajectory prediction error smaller and improve the tracking effect for large speed changes.

[0093] Among them, when the tracking algorithm model is constructed based on the space equation of the uniform motion state, the space equation of the uniform motion state simplifies the motion state of the target, which can save computing resources and improve the update efficiency.

[0094] Exemplarily, the target tracking algorithm provided by the embodiments of the present application needs to be applied to real-time video. To avoid the problem of excessive time consumption, when updating the trajectory of the target in each frame of the image, the state space equation (1) can be simplified to the form of uniform motion, as shown in the following formula (2):

[0095]

[0096] Among them, in the above formula, x′ and y′ represent the coordinates of the center point of the current trajectory on the x-axis and y-axis of the image, and x and y represent the coordinates of the center point of the previous frame trajectory on the x-axis and y-axis of the image. represents the speed of the same target in the previous frame in the x-axis and y-axis directions of the image; α′ and h′ represent the aspect ratio and height of the current trajectory, and α and h represent the aspect ratio and height of the previous frame trajectory. represents the aspect ratio change rate and height change rate of the same target in adjacent frames.

[0097] In this way, through the simplified state space equation (2) and using the Kalman filter, after determining the first features of each target to be tracked, the real-time state of each target to be tracked in the image can be updated. Since the motion of the target is simplified to the form of uniform motion, the established tracking algorithm model can improve the update efficiency when updating the trajectory of the target, meeting the requirements of real-time video.

[0098] As Figure 4 shown, in one embodiment, the process of matching the first features of each target to be tracked with the features of the corresponding fusion prediction trajectory in S102 includes the following steps:

[0099] S301, obtaining the current consecutive tracking failure times of each target to be tracked; where the current consecutive tracking failure means that the first features of the target to be tracked and the features of the fusion prediction trajectory have continuously failed to match before the current frame.

[0100] Among them, the current consecutive tracking failure indicates the situation where the first feature of the target to be tracked before the current frame fails to continuously match the feature of the fused prediction trajectory. For each target to be tracked, it is necessary to determine the trajectory of each target to be tracked frame by frame in the video stream. Each target to be tracked has a situation of tracking failure. After determining the target trajectory in each frame of the image, record the tracking failure or success situation of each target to be tracked. For example, a feature pool can be established. In this feature pool, a table is established for each target to be tracked respectively, which is used to store all information of each target to be tracked, including: in which frame of the image the target to be tracked fails or succeeds in tracking, the detection box of the target to be tracked, the predicted trajectory, etc. In this way, when information is needed, it can be directly obtained from the feature pool. Of course, other methods can also be used to obtain the number of current consecutive tracking failures of each target to be tracked, and this embodiment does not limit this.

[0101] S302. Match the first feature of each target to be tracked with the feature of the corresponding fused prediction trajectory in ascending order of the number of current consecutive tracking failures.

[0102] Among them, the more consecutive tracking failures, the lower the matching success rate of the target to be tracked. After obtaining the number of current consecutive tracking failures of each target to be tracked, match the first feature of each target to be tracked with the feature of the corresponding fused prediction trajectory in ascending order of the number of current consecutive tracking failures, and start the matching with the target to be tracked with a high matching success rate.

[0103] In this embodiment, starting from the smallest number of current consecutive tracking failures of each target to be tracked for matching, and preferentially matching the target to be tracked with a high matching success rate can improve the matching efficiency.

[0104] When matching, a hybrid matching method can be used for feature matching. Different matching methods can complement each other, match the first feature and the corresponding fused prediction trajectory from different dimensions, and can effectively determine the identifiers of each target to be tracked, improving the accuracy and effectiveness of target tracking.

[0105] Then, as Figure 5 shown, in one embodiment, the above S302 includes the following steps:

[0106] S401. Calculate the similarity between the feature of the fused prediction trajectory of each target to be tracked and all the first features in turn.

[0107] If the steps in this embodiment are executed one by one in sequence, taking one of the targets to be tracked, Target No. 1 as an example: Calculate the similarity between the fused prediction trajectory of this Target No. 1 and all the first features. Here, all the first features refer to all the first features in multiple current-frame images. For example, if there are 3 current-frame images and 10 first features are detected in these 3 images, then the similarity is calculated between the fused prediction trajectory of Target No. 1 and these 10 first features respectively, that is, there are 10 similarities.

[0108] Among them, the way to calculate the similarity can be through distance metrics, such as Euclidean distance, etc. This embodiment does not limit the way to calculate the similarity.

[0109] S402, Obtain the first feature corresponding to the minimum similarity of each target to be tracked.

[0110] After obtaining the similarities between the features of the fused prediction trajectories of each target to be tracked and all the first features, obtain the first feature corresponding to the minimum similarity of each target to be tracked. Since for each target to be tracked, there are multiple similarities, select the first feature corresponding to the minimum similarity from the multiple similarities of this target to be tracked. For example: The target to be tracked is Target No. 1 and there are a total of 10 first features. Then Target No. 1 has 10 similarities after step S401. Select the minimum similarity from these 10 similarities, and the first feature corresponding to this minimum similarity is the first feature that Target No. 1 needs to obtain; following this process, obtain the first feature corresponding to the minimum similarity of each target to be tracked in turn.

[0111] S403, If the first features corresponding to the minimum similarities of all the targets to be tracked are all different, then the first features corresponding to the minimum similarities of each target to be tracked match successfully with the features of the fused prediction trajectory.

[0112] After obtaining the first feature corresponding to the minimum similarity of each target to be tracked in the above steps, for each target to be tracked, there is a corresponding relationship: Target No. N to be tracked - fused prediction trajectory - minimum similarity - first feature; If there are no duplicate first features in this corresponding relationship, that is, one first feature cannot appear in more than two corresponding relationships; If there are no duplicates in the first features in this corresponding relationship, it is determined that this set of first features corresponding to the minimum similarity of this target to be tracked matches successfully with the fused prediction trajectory.

[0113] It should be noted that since the corresponding relationships of all the targets to be tracked obtained in this step are determined by the minimum similarity of each target to be tracked, for each target to be tracked, the set of first features corresponding to its minimum similarity is the most similar to the fusion prediction trajectory. On this premise, if there are no duplicate first features in the corresponding relationships, it means that each first feature has been matched to the correct fusion prediction trajectory, and then the first features and the fusion prediction trajectories in the corresponding relationships are successfully matched.

[0114] S404. If there are at least two first features corresponding to the minimum similarities of the targets to be tracked that are the same, obtain the second features, and based on the intersection-over-union ratio between the second features and the fusion prediction trajectories of the targets to be tracked, match the second features with the fusion prediction trajectories of the targets to be tracked; where the second features are the first features that have not been successfully matched.

[0115] However, if the corresponding relationships of all the targets to be tracked obtained above are such that as long as there are more than two identical first features, then the first features and the fusion prediction trajectories in the corresponding relationships where the identical first feature is located are not successfully matched. For this situation, two processing methods can be adopted. One is to determine that all the corresponding relationships where the identical first feature is located do not hold, that is, the first feature and the fusion prediction trajectory are not successfully matched. Another processing method is to compare the magnitude relationships of the similarities in all the corresponding relationships where the identical first feature is located, and determine the first feature to the smallest similarity value among these corresponding relationships, and the other corresponding relationships can be adjusted in ascending order of their respective similarity values until there are no duplicate first features in all the corresponding relationships. Among them, in this embodiment, the first method is used for processing. If there are at least two first features corresponding to the minimum similarities of the targets to be tracked that are the same, obtain the second features, where the second features are the first features that have not been successfully matched.

[0116] Optionally, the second features include other first features other than the first features corresponding to the minimum similarities, and / or the identical first features among the first features corresponding to the minimum similarities of all the targets to be tracked.

[0117] That is, in addition to including the same first features in the first features corresponding to the minimum similarity of the above-mentioned target to be tracked, the second features may also include other first features outside the first features corresponding to the minimum similarity. Specifically, since the corresponding relationship of each of the above-mentioned targets to be tracked is determined by the minimum similarity, there will be some remaining first features during the process of determining the corresponding relationship of the minimum similarity value. These remaining first features also belong to the first features that have not been successfully matched. For example, there are 3 targets to be tracked: Target No. 1, Target No. 2, and Target No. 3; there are 4 first features: First Feature A, First Feature B, First Feature C, and First Feature D; the similarity between Target No. 1 and First Feature A is the smallest. Suppose the corresponding relationship of Target No. 1 is: Target No. 1--First Feature A--Minimum Similarity--Target 1 Fusion Prediction Trajectory; the corresponding relationship of Target No. 2 is: Target No. 2--First Feature A--Minimum Similarity--Target 2 Fusion Prediction Trajectory; the corresponding relationship of Target No. 3 is: Target No. 3--First Feature B--Minimum Similarity--Target 3 Fusion Prediction Trajectory; the corresponding relationship of Target No. 4 is: Target No. 4--First Feature C--Minimum Similarity--Target 4 Fusion Prediction Trajectory; in this case, First Feature A appears in two corresponding relationships, and it is determined that First Feature A fails to match. Since First Feature D does not appear in the corresponding relationship, it is left over and also fails to match.

[0118] For the first features (i.e., the second features) that have not been successfully matched, it is possible to determine whether they match the corresponding fusion prediction trajectory by calculating the intersection over union (IoU), that is, based on the IoU between the second features and the fusion prediction trajectories of each target to be tracked, the second features and the fusion prediction trajectories of each target to be tracked are matched. As Figure 6 shown, in one embodiment, "matching the second features and the fusion prediction trajectories of each target to be tracked based on the IoU between the second features and the fusion prediction trajectories of each target to be tracked" in S404 includes the following steps:

[0119] S501, calculate the IoU between the fusion prediction trajectory of each target to be tracked and each first feature in the second features in turn.

[0120] Similarly, when calculating step by step, each step in this embodiment is executed one by one. Taking one of the targets to be tracked, Target No. 1, as an example: Calculate the intersection over union (IoU) between the fused prediction trajectory of this Target No. 1 and each first feature in the second feature. Here, the first feature is the first feature that did not match successfully in the above-mentioned second feature. When calculating the IoU, still calculate the IoU between each first feature in the calculated second feature and the fused prediction trajectory of Target No. 1. For example, if there are 4 first features in the second feature, then calculate the IoU between the fused prediction trajectory of Target No. 1 and these 4 first features respectively, that is, Target No. 1 has 4 IoUs. Calculated according to this process, each target to be tracked has 4 IoUs.

[0121] Among them, the way to calculate the IoU is the intersection of the detection box corresponding to the first feature and the rectangular box of the fused prediction trajectory divided by the union of the two.

[0122] S502, Obtain the first feature corresponding to the minimum IoU of each target to be tracked.

[0123] After obtaining the IoU between the features of the fused prediction trajectory of each target to be tracked and each first feature in the second feature, obtain the first feature corresponding to the minimum IoU of each target to be tracked. The selection method is similar to the above similarity. For example: If the target to be tracked is Target No. 1 and there are a total of 4 first features in the second feature, then Target No. 1 obtains 4 IoUs after step S501. Select the minimum IoU from these 4 IoUs, and the first feature corresponding to this minimum IoU is the first feature corresponding to the minimum IoU of Target No. 1; According to this process, obtain the first feature corresponding to the minimum IoU of each target to be tracked in turn.

[0124] S503, If the first features corresponding to the minimum IoU of all targets to be tracked are all different, then the first features corresponding to the minimum IoU of each target to be tracked are successfully matched with the fused prediction trajectory.

[0125] After obtaining the first feature corresponding to the minimum IoU of each target to be tracked in the above steps, for each target to be tracked, the corresponding relationship is: Target No. N to be tracked - fused prediction trajectory - minimum IoU - first feature; If there is no repetition in the first features in this corresponding relationship, it is determined that this set of first features corresponding to the minimum IoU of the target to be tracked is successfully matched with the fused prediction trajectory.

[0126] It should be understood that since in this embodiment, the IoU is calculated for the first features that did not match successfully after similarity, in the case where the number of first features that did not match successfully is small, it is not necessarily possible to determine the corresponding relationship for each target to be tracked. Therefore, in the case where the number of first features is small, as long as it is ensured that in the corresponding relationship where each first feature is located, the similarity value is the smallest.

[0127] The target tracking method provided in this embodiment first matches the fusion prediction trajectories of each target to be tracked with the acquired first features based on similarity. For the first features that fail to match successfully, the intersection over union (IoU) method is used for matching. In this way, matching from different dimensions can comprehensively and effectively ensure that each first feature can be matched, enabling each target to be tracked accurately.

[0128] For the successfully matched first features and fusion prediction trajectories, while tracking the corresponding targets to be tracked according to the first features, the first features of the successfully matched targets to be tracked can also be stored. In one embodiment, the method further includes: storing the successfully matched first features into the prediction trajectory list corresponding to the first features in the feature pool. The feature pool includes the corresponding relationships between the first features of the successfully matched targets to be tracked and the fusion prediction trajectories in each frame of the image. Optionally, after storing the successfully matched first features into the prediction trajectory list corresponding to the first features in the feature pool, the method further includes: managing the first features of each target to be tracked in the feature pool according to a preset storage condition; the storage condition is used to control the number of the first features of each target to be tracked in the feature pool.

[0129] Among them, for each frame of the image, the successfully matched first features in all images of this frame are stored into the prediction trajectory list corresponding to the first features in the feature pool. The feature pool includes the corresponding relationships between the first features of the successfully matched targets to be tracked and the fusion prediction trajectories in each frame of the image, and may also include other information. For example, the number of tracking failures of the target to be tracked, or the similarity and IoU values in the matching process, and the identifiers of each target to be tracked determined after successful matching, etc. This embodiment does not limit this.

[0130] To avoid the large amount of data in the feature pool occupying memory and affecting the tracking efficiency, it is necessary to manage the data in the feature pool. For example, limit the storage quantity, only retain a certain amount of data for each target to be tracked, and delete the features with too long retention time. Taking the HOG feature as an example, although the HOG feature can be used to distinguish targets, it is not necessary to distinguish all targets at all times. For those that are relatively long ago in time, its effect is not very good. Therefore, only the latest 100 frames of HOG features of each trajectory can be retained in the feature pool. In this way, the feature pool is effectively managed, which can not only save data for subsequent use but also does not affect the tracking efficiency.

[0131] In one embodiment, as Figure 7 shown, an embodiment of a target tracking method is provided. This embodiment includes:

[0132] S601, obtaining the current frame image in each video stream collected from different perspectives;

[0133] S602, Detect the detection boxes of the targets to be tracked from multiple current frame images;

[0134] S603, Extract the HOG features of the detection boxes of each target to be tracked, and execute S606;

[0135] S604, Construct the tracking algorithm according to the model of the state space equation;

[0136] S605, Predict the predicted trajectories of the targets to be tracked in the current frame image;

[0137] S606, Store them in the feature pool;

[0138] S607, Calculate the similarity between the features of the predicted trajectories and all HOG features for matching;

[0139] S608, Calculate the intersection over union between the predicted trajectories and the remaining unmatched HOG features for matching;

[0140] S609, Update the trajectories for successful matching;

[0141] S610, End.

[0142] For each step in the target tracking method provided in this embodiment, its implementation principle and technical effect are similar to those in the previous target tracking method embodiments, and will not be elaborated here. Figure 7 The implementation methods of the steps in the embodiment are only examples, and the implementation methods are not limited. The order of the steps can be adjusted in actual applications as long as the purposes of the steps can be achieved.

[0143] It should be understood that although Figures 2 - 7 the steps in the flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figures 2 - 7 at least a part of the steps in

[0144] In one embodiment, as Figure 8 shown, a target tracking device is provided, and the device includes:

[0145] A fusion module 10, configured to fuse predicted trajectories of the same target to be tracked in multiple current-frame images, so as to obtain a fused predicted trajectory of each target to be tracked; the multiple current-frame images are images collected by camera devices with different perspectives.

[0146] A matching module 11, configured to match the first feature of each target to be tracked with the feature of the corresponding fused predicted trajectory; the first feature is the feature of the target to be tracked obtained by performing target detection on the current-frame image.

[0147] A tracking module 12, configured to track the target to be tracked according to the successfully matched first feature.

[0148] In one embodiment, the device includes:

[0149] A detection box acquisition module, configured to detect each target to be tracked from multiple current-frame images collected by camera devices with different perspectives, so as to obtain a detection box of each target to be tracked.

[0150] A feature extraction module, configured to extract the image feature in the detection box of each target to be tracked as the first feature.

[0151] In one embodiment, the device includes: a prediction module, configured to predict the predicted trajectory of each target to be tracked in each current-frame image through a preset tracking algorithm model, where the tracking algorithm model is constructed based on a preset state space equation.

[0152] In one embodiment, the above-mentioned preset state space equation includes a space equation in a uniformly variable motion state and a space equation in a uniform motion state; if the tracking algorithm model is constructed based on the space equation in a uniformly variable motion state, the tracking algorithm model is used to predict the trajectory of the target in the image that has not been completely tracked; if the tracking algorithm model is constructed based on the space equation in a uniform motion state, the tracking algorithm model is used to update the trajectory of the target in the image that has been completely tracked.

[0153] In one embodiment, the above-mentioned matching module 11 includes:

[0154] A failure times unit, configured to obtain the current consecutive tracking failure times of each target to be tracked; where the current consecutive tracking failure indicates that the first feature of the target to be tracked and the feature of the fused predicted trajectory have continuously failed to match before the current frame.

[0155] An order matching unit, configured to match the first feature of each target to be tracked with the feature of the corresponding fused predicted trajectory in ascending order of the current consecutive tracking failure times.

[0156] In one embodiment, the above-mentioned sequential matching unit is specifically configured to sequentially calculate the similarity between the features of the fusion prediction trajectories of each target to be tracked and all the first features; obtain the first feature corresponding to the minimum similarity of each target to be tracked; if the first features corresponding to the minimum similarity of all the targets to be tracked are all different, then the first feature corresponding to the minimum similarity of each target to be tracked is successfully matched with the feature of the fusion prediction trajectory; if there are at least two first features corresponding to the minimum similarity of the targets to be tracked that are the same, then obtain the second feature, and based on the intersection-over-union ratio between the second feature and the fusion prediction trajectories of each target to be tracked, match the second feature with the fusion prediction trajectories of each target to be tracked; wherein, the second feature is the first feature that has not been successfully matched.

[0157] In one embodiment, the above-mentioned second feature includes other first features other than the first feature corresponding to the minimum similarity, and / or the same first features among the first features corresponding to the minimum similarity of all the targets to be tracked.

[0158] In one embodiment, the above-mentioned sequential matching unit is further specifically configured to sequentially calculate the intersection-over-union ratio between the fusion prediction trajectory of each target to be tracked and each first feature in the second feature; obtain the first feature corresponding to the minimum intersection-over-union ratio of each target to be tracked; if the first features corresponding to the minimum intersection-over-union ratio of all the targets to be tracked are all different, then the first feature corresponding to the minimum intersection-over-union ratio of each target to be tracked is successfully matched with the fusion prediction trajectory.

[0159] In one embodiment, the device further includes: a storage module, configured to store the successfully matched first feature into the prediction trajectory list corresponding to the first feature in the feature pool, and the feature pool includes the corresponding relationship between the first feature of each successfully tracked target in each frame image and the fusion prediction trajectory.

[0160] In one embodiment, the device further includes: a management module, configured to manage the first feature of each target to be tracked in the feature pool according to a preset storage condition; the storage condition is used to control the number of the first features of each target to be tracked in the feature pool.

[0161] For the specific limitations of the target tracking device, reference may be made to the limitations on the target tracking method in the above text, which will not be elaborated here. Each module in the above target tracking device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in the form of hardware or independent thereof, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0162] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 1aAs shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be achieved through WIFI, carrier network, NFC (Near Field Communication) or other technologies. When the computer program is executed by the processor, it realizes a target tracking method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, a trackball or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0163] Those skilled in the art can understand that Figure 1a the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0164] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are realized:

[0165] Fuse the predicted trajectories of the same target to be tracked in multiple current frame images to obtain the fused predicted trajectories of each target to be tracked; the multiple current frame images are images collected by camera devices with different perspectives;

[0166] Match the first feature of each target to be tracked with the feature of the corresponding fused predicted trajectory; the first feature is the feature of the target to be tracked obtained by detecting the target in the current frame image;

[0167] Track the target to be tracked according to the successfully matched first feature.

[0168] In one embodiment, when the processor executes the computer program, the following steps are also realized:

[0169] Detect each target to be tracked from multiple current frame images collected by camera devices with different perspectives to obtain the detection frames of each target to be tracked;

[0170] Extract the image features in the detection frames of each target to be tracked as the first feature.

[0171] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0172] Predict the predicted trajectory of each target to be tracked in each current frame image through a preset tracking algorithm model, where the tracking algorithm model is constructed based on a preset state space equation.

[0173] In one embodiment, the above-mentioned preset state space equation includes the space equation of uniformly variable motion state and the space equation of uniform motion state;

[0174] If the tracking algorithm model is constructed based on the space equation of uniformly variable motion state, the tracking algorithm model is used to predict the trajectory of the target in the image where the tracking is not completed; if the tracking algorithm model is constructed based on the space equation of uniform motion state, the tracking algorithm model is used to update the trajectory of the target in the image where the tracking has been completed.

[0175] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0176] Obtain the current number of consecutive tracking failures of each target to be tracked; where the current consecutive tracking failure means that the first feature of the target to be tracked before the current frame fails to match the feature of the fusion predicted trajectory continuously.

[0177] Match the first feature of each target to be tracked with the feature of the corresponding fusion predicted trajectory in ascending order of the current number of consecutive tracking failures.

[0178] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0179] Calculate the similarity between the feature of the fusion predicted trajectory of each target to be tracked and all the first features in turn;

[0180] Obtain the first feature corresponding to the minimum similarity of each target to be tracked;

[0181] If the first features corresponding to the minimum similarity of all the targets to be tracked are all different, the first feature corresponding to the minimum similarity of each target to be tracked matches the feature of the fusion predicted trajectory successfully;

[0182] If there are at least two first features corresponding to the minimum similarity of the targets to be tracked that are the same, obtain the second feature, and match the second feature with the fusion predicted trajectory of each target based on the intersection-over-union ratio between the second feature and the fusion predicted trajectory of each target; where the second feature is the first feature that fails to match successfully.

[0183] In one embodiment, the second feature includes other first features except the first feature corresponding to the minimum similarity, and / or the same first features among the first features corresponding to the minimum similarity of all the targets to be tracked.

[0184] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0185] Calculate the intersection over union (IoU) between the fused predicted trajectory of each target to be tracked and each first feature in the second feature in sequence;

[0186] Obtain the first feature corresponding to the minimum IoU of each target to be tracked;

[0187] If the first features corresponding to the minimum IoU of all the targets to be tracked are all different, the first features corresponding to the minimum IoU of each target to be tracked are successfully matched with the fused predicted trajectory.

[0188] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0189] Store the successfully matched first features into the predicted trajectory list corresponding to the first feature in the feature pool, where the feature pool includes the corresponding relationships between the first features of each successfully matched target to be tracked in each frame of image and the fused predicted trajectory.

[0190] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0191] Manage the first features of each target to be tracked in the feature pool according to a preset storage condition; the storage condition is used to control the number of the first features of each target to be tracked in the feature pool.

[0192] For the computer device provided in the above embodiment, its implementation principle and technical effects are similar to those of the above method embodiment, and will not be elaborated here.

[0193] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0194] Fuse the predicted trajectories of the same target to be tracked in multiple current frame images to obtain the fused predicted trajectory of each target to be tracked; the multiple current frame images are images collected by camera devices with different perspectives;

[0195] Match the features of the first feature of each target to be tracked and the corresponding fused predicted trajectory; the first feature is the feature of the target to be tracked obtained by performing target detection on the current frame image;

[0196] Track the target to be tracked according to the successfully matched first feature.

[0197] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0198] Detect each target to be tracked from multiple current frame images collected by camera devices from different perspectives, and obtain the detection frames of each target to be tracked;

[0199] Extract the image features in the detection frames of each target to be tracked as the first features.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0201] Predict the prediction trajectories of each target to be tracked in each current frame image through a preset tracking algorithm model, where the tracking algorithm model is constructed based on a preset state space equation.

[0202] In one embodiment, the above-mentioned preset state space equation includes the space equation of uniformly variable motion state and the space equation of uniform motion state;

[0203] If the tracking algorithm model is constructed based on the space equation of uniformly variable motion state, the tracking algorithm model is used to predict the trajectories of the targets in the images that have not been completely tracked; if the tracking algorithm model is constructed based on the space equation of uniform motion state, the tracking algorithm model is used to update the trajectories of the targets in the images that have been completely tracked.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0205] Obtain the current consecutive tracking failure times of each target to be tracked; where the current consecutive tracking failure means that the first feature of the target to be tracked and the feature of the fusion prediction trajectory have continuously failed to match before the current frame;

[0206] Match the first feature of each target to be tracked and the feature of the corresponding fusion prediction trajectory in ascending order of the current consecutive tracking failure times.

[0207] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0208] Calculate the similarity between the feature of the fusion prediction trajectory of each target to be tracked and all the first features in turn;

[0209] Obtain the first feature corresponding to the minimum similarity of each target to be tracked;

[0210] If the first features corresponding to the minimum similarities of all the targets to be tracked are all different, the first feature corresponding to the minimum similarity of each target to be tracked and the feature of the fusion prediction trajectory are successfully matched;

[0211] If there are at least two targets to be tracked whose corresponding first features of the minimum similarity are the same, then a second feature is obtained, and based on the intersection-over-union ratio between the second feature and the fused predicted trajectories of each target to be tracked, the second feature is matched with the fused predicted trajectories of each target to be tracked; wherein the second feature is the first feature that was not successfully matched.

[0212] In one embodiment, the second feature includes other first features other than the first feature corresponding to the minimum similarity, and / or the same first feature among the first features corresponding to the minimum similarity of all the targets to be tracked.

[0213] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0214] Calculate the intersection-over-union ratio between the fused predicted trajectory of each target to be tracked and each first feature in the second feature in sequence;

[0215] Obtain the first feature corresponding to the minimum intersection-over-union ratio of each target to be tracked;

[0216] If the first features corresponding to the minimum intersection-over-union ratios of all the targets to be tracked are different, the first features corresponding to the minimum intersection-over-union ratios of the targets to be tracked are successfully matched with the fusion predicted trajectory.

[0217] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0218] The first feature that is successfully matched is stored in a predicted trajectory list corresponding to the first feature in a feature pool. The feature pool includes the correspondence between the first feature of each target to be tracked that is successfully matched in each frame image and the fused predicted trajectory.

[0219] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0220] The first feature of each target to be tracked in the feature pool is managed according to a preset storage condition; the storage condition is used to control the number of first features of each target to be tracked in the feature pool.

[0221] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effects are similar to those of the above method embodiment, and will not be repeated here.

[0222] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0223] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0224] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A target tracking method, characterized in that, The method includes: Fusing the predicted trajectories of the same target to be tracked in multiple current frame images to obtain the fused predicted trajectories of the targets to be tracked; the multiple current frame images are images collected by camera devices with different perspectives; Calculating the similarity between the features of the fused predicted trajectories of the targets to be tracked and the first features of all the targets to be tracked in sequence; Obtaining the first feature corresponding to the minimum similarity of each target to be tracked; If the first features corresponding to the minimum similarities of all the targets to be tracked are all different, the first features corresponding to the minimum similarities of the targets to be tracked match successfully with the features of the fused predicted trajectories; If there are at least two first features corresponding to the minimum similarities of the targets to be tracked that are the same, obtaining second features, and matching the second features with the fused predicted trajectories of the targets to be tracked based on the intersection over union between the second features and the fused predicted trajectories of the targets to be tracked; the first feature is the feature of the target to be tracked obtained by performing target detection on the current frame image; the second feature is the first feature that fails to match successfully; Tracking the target to be tracked according to the successfully matched first feature.

2. The method according to claim 1, wherein, The method further includes: Detecting the targets to be tracked from multiple current frame images collected by camera devices with different perspectives to obtain the detection frames of the targets to be tracked; Extracting the image features in the detection frames of the targets to be tracked as the first features.

3. The method according to claim 1 or 2, characterized in that, Before fusing the predicted trajectories of the same target to be tracked in multiple current frame images, the method further includes: Predicting the predicted trajectory of each target to be tracked in each of the current frame images through a preset tracking algorithm model, where the tracking algorithm model is constructed based on a preset state space equation.

4. The method according to claim 3, wherein The preset state space equation includes the space equation of a uniformly variable motion state and the space equation of a uniform motion state; If the tracking algorithm model is constructed based on the space equation of the uniformly variable motion state, the tracking algorithm model is used to predict the trajectory of the target in the image that has not been completely tracked; If the tracking algorithm model is constructed based on the space equation of the uniform motion state, the tracking algorithm model is used to update the trajectory of the target in the image that has been completely tracked.

5. The method according to claim 1 or 2, characterized in that, The method further includes: Obtaining the current consecutive tracking failure times of each target to be tracked; where the current consecutive tracking failure means that the first feature of the target to be tracked and the feature of the fused predicted trajectory have continuously failed to match before the current frame; Calculating the similarity between the features of the fused predicted trajectories of the targets to be tracked and the first features of all the targets to be tracked in sequence in ascending order of the current consecutive tracking failure times.

6. The method according to claim 1 or 2, characterized in that, The second feature includes other first features except the first feature corresponding to the minimum similarity, and / or, the same first features among the first features corresponding to the minimum similarities of all the targets to be tracked.

7. The method according to claim 1 or 2, characterized in that, The matching of the second features with the fused predicted trajectories of the targets to be tracked based on the intersection over union between the second features and the fused predicted trajectories of the targets to be tracked includes: Calculate the intersection over union (IoU) between the fused prediction trajectories of the to-be-tracked targets and each first feature in the second feature in sequence; Obtain the first feature corresponding to the minimum IoU of each to-be-tracked target; If the first features corresponding to the minimum IoU of all to-be-tracked targets are different from each other, the first features corresponding to the minimum IoU of each to-be-tracked target are successfully matched with the fused prediction trajectories.

8. The method according to claim 1 or 2, characterized in that Before tracking the to-be-tracked targets according to the successfully matched first features, the method further includes: Store the successfully matched first features into the prediction trajectory list corresponding to the first feature in the feature pool, where the feature pool includes the corresponding relationships between the first features of each successfully tracked to-be-tracked target in each frame image and the fused prediction trajectories.

9. The method according to claim 8, wherein After storing the successfully matched first features into the prediction trajectory list corresponding to the first feature in the feature pool, the method further includes: Manage the first features of each to-be-tracked target in the feature pool according to a preset storage condition; the storage condition is used to control the number of the first features of each to-be-tracked target in the feature pool.

10. A target tracking device, characterized in that, The apparatus includes: A fusion module, configured to fuse the prediction trajectories of the same to-be-tracked target in multiple current frame images to obtain the fused prediction trajectories of the to-be-tracked targets; the multiple current frame images are images collected by camera devices with different perspectives; A matching module, configured to calculate the similarity between the features of the fused prediction trajectories of the to-be-tracked targets and the first features of all to-be-tracked targets in sequence; obtain the first feature corresponding to the minimum similarity of each to-be-tracked target; if the first features corresponding to the minimum similarity of all to-be-tracked targets are different from each other, the first features corresponding to the minimum similarity of each to-be-tracked target are successfully matched with the features of the fused prediction trajectories; if there are at least two first features corresponding to the minimum similarity of the to-be-tracked targets that are the same, obtain a second feature, and perform matching between the second feature and the fused prediction trajectories of the to-be-tracked targets based on the IoU between the second feature and the fused prediction trajectories of the to-be-tracked targets; the first feature is the feature of the to-be-tracked target obtained by performing target detection on the current frame image; the second feature is the first feature that has not been successfully matched; A tracking module, configured to track the to-be-tracked targets according to the successfully matched first features.

11. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images