Method and device for multi-target trajectory tracking in video

Abstract

The invention relates to the field of computer vision, in particular to a multi-target trajectory tracking method and device in a video. The method mainly comprises the steps of generating at least one short track of a target based on spatial similarity of target related images in a video; capturing a salient region in each short trajectory, and performing feature interaction on the salient regions to obtain trajectory features; measuring a space-time relationship between the trajectory features, and carrying out short trajectory matching according to the space-time relationship to generate a long trajectory; and recovering the shielded target between different long tracks, and generating a smooth track of the target. According to the invention, the problem of identity jump caused by target occlusion can be relieved, the occluded target can be recovered, and the tracking precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and device for tracking multi-target trajectories in a video. Background Art

[0002] With the widespread application of drones, image acquisition and processing technologies based on drones have developed rapidly in many fields such as transportation, logistics, and photography. Visual target tracking, as an important technology for visual data analysis in drone applications, is the most direct embodiment of the intelligence and automation level of drones. Through the visual multiple object tracking (MOT) technology, the video images taken by drones can be analyzed to locate the position of the target, providing a basis for flight decision-making for the automated application of drones. This technology plays a very important role in the fields of drone emergency rescue in disaster areas, geological surveys, target search, and ground target positioning.

[0003] Currently, multi-target tracking technology based on drone vision has become an important research topic. However, in the videos collected by drones, there is often a problem of target occlusion and identity jump, which leads to interruption of target trajectory tracking, confusion of multiple target trajectories, or identity jump when targets are intertwined.

[0004] In view of this, how to overcome the defects of the prior art and solve the problem of inaccurate multi-target trajectory tracking in videos is a problem to be solved in this technical field. Summary of the invention

[0005] In view of the above defects or improvement requirements of the prior art, the present invention solves the problem of inaccurate multi-target trajectory tracking.

[0006] The embodiment of the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for tracking multiple target trajectories in a video, specifically: generating at least one short trajectory of the target based on the spatial similarity of target-related images in the video; capturing the salient areas within each short trajectory, and performing feature interaction on the salient areas to obtain trajectory features; measuring the spatiotemporal relationship between trajectory features, matching short trajectories according to the spatiotemporal relationship to generate long trajectories; restoring occluded targets between different long trajectories to generate a smooth trajectory of the target.

[0008] Preferably, the method of generating at least one short trajectory of the target based on the spatial similarity of target-related images in the video specifically includes: using a Kalman filter to learn the historical trajectory of each target and outputting the predicted position of the target in the current frame; calculating the spatial similarity between the predicted position of the current frame and the detection result of the current frame, and measuring the spatial similarity by an intersection-over-union ratio to obtain a similarity matrix; performing data association on the similarity matrix until the data association of each frame is completed, and obtaining the short trajectory of the target according to the data association result.

[0009] Preferably, the capturing of the salient areas within each short trajectory and performing feature interaction on the salient areas to obtain trajectory features specifically includes: dividing each short trajectory into trajectory segments of fixed length, inputting the trajectory segments into a graph convolutional spatiotemporal attention network, and obtaining node features of the graph convolutional neural network; screening neighbor nodes of each node between each layer of the graph convolutional neural network, and aggregating node features of each node and neighbor nodes to complete feature interaction between different targets; based on the feature interaction results, calculating the similarity distance between the trajectory segments, calculating the feature differences between different nodes according to the similarity distance, and obtaining discriminative trajectory features according to the feature differences.

[0010] Preferably, the neighbor nodes of each node are screened between each layer of the graph convolutional neural network, and the node features of each node and the neighbor nodes are aggregated to complete the feature interaction between different targets, specifically including: according to the spatial distance between the nodes, the neighbor nodes of each node are screened between each layer of the graph convolutional network; the nodes are connected with their neighbor nodes to construct edges, and the edges are connected to construct a graph structure; the graph topology mechanism is used to transfer information through node feature aggregation to achieve feature interaction between targets.

[0011] Preferably, the measuring the spatiotemporal relationship between trajectory features and matching short trajectories according to the spatiotemporal relationship to generate a long trajectory specifically includes: obtaining the spatiotemporal relationship between short trajectories according to the trajectory features, obtaining a measurement function between the short trajectories according to the spatiotemporal relationship; obtaining a spatiotemporal relationship measurement value between the short trajectories according to the measurement function, matching the short trajectories based on the spatiotemporal relationship measurement value to generate a long trajectory.

[0012] Preferably, the metric function between the short trajectories is obtained according to the spatiotemporal relationship, wherein obtaining the spatial similarity in the metric function specifically includes: using a single target tracker to search for the coordinates of the historical trajectory corresponding to each short trajectory at the current moment, and calculating the spatial distance between the short trajectories according to the coordinates; performing maximum and minimum normalization on the spatial distance, and obtaining the spatial similarity between the short trajectories according to the normalization result.

[0013] Preferably, the metric function between the short trajectories is obtained according to the spatiotemporal relationship, wherein obtaining the time similarity in the metric function specifically includes: obtaining the time domain interval between different short trajectories, and obtaining the time similarity between the short trajectories according to the time domain interval.

[0014] Preferably, the metric function between the short trajectories is obtained according to the spatiotemporal relationship, wherein obtaining the apparent similarity in the metric function specifically comprises: obtaining the apparent similarity between the short trajectories according to the cosine similarity between the trajectory features of different short trajectories.

[0015] In a second aspect, the present invention provides a device for tracking multiple target trajectories in a video, specifically comprising: comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for tracking multiple target trajectories in a video in the first aspect.

[0016] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method described in the first aspect.

[0017] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method provided in the first aspect.

[0018] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer or a processor, causes the computer or the processor to execute the method provided in the first aspect.

[0019] Compared with the prior art, the beneficial effects of the present invention are: fully mining the spatiotemporal context information of the trajectory through feature interaction, extracting more discriminative spatiotemporal features of the trajectory, which can alleviate the problem of identity jump caused by occlusion of the target, improve the robustness of the tracker in the occluded environment, and at the same time restore the occluded target and improve the tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flow chart of a method for tracking multiple target trajectories in a video provided by an embodiment of the present invention;

[0022] Figure 2 A flow chart of another method for tracking multiple target trajectories in a video provided by an embodiment of the present invention;

[0023] Figure 3 A flow chart of another method for tracking multiple target trajectories in a video provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the process of calculating trajectory features using a graph convolutional spatiotemporal attention network in the method provided in an embodiment of the present invention;

[0025] Figure 5 A flow chart of another method for tracking multiple target trajectories in a video provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of the structure of a device for tracking multiple target trajectories in a video provided by an embodiment of the present invention;

[0027] The reference numerals are as follows:

[0028] 11: processor; 12: memory. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0031] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0032] Embodiment 1:

[0033] At present, multi-target tracking based on UAV vision has become an important technology. However, in the videos collected by UAVs, there is often a problem of target occlusion and identity jump. Therefore, studying an efficient and robust UAV multi-target tracking algorithm is of great significance to the application of UAVs.

[0034] like Figure 1 As shown, the method for tracking multiple target trajectories in a video provided by an embodiment of the present invention has the following specific steps:

[0035] Step 101: Generate at least one short trajectory of the target based on the spatial similarity of target-related images in the video.

[0036] When tracking multiple targets in a video, the image area that may contain the target in each frame of the video is called the target-related image. Image analysis is performed on each frame of the video. The motion trajectory of the same target should be a continuous and smooth spatial movement on a continuous timeline. Therefore, the images in the continuous frames containing the target can be analyzed, and the short trajectory of the target between a set of continuous adjacent frames can be obtained through the spatial similarity of the target-related images. In specific implementation, a Kalman filter can be used to generate the short trajectory of the target based on spatial similarity.

[0037] Step 102: Capture the salient regions within each short trajectory, and perform feature interaction on the salient regions to obtain trajectory features.

[0038] When tracking a target trajectory, each target trajectory contains multiple short trajectories. In order to distinguish the short trajectories of different targets, it is also necessary to obtain the trajectory features of each short trajectory. In the specific implementation, the graph convolutional spatiotemporal attention network can be used to capture the salient areas inside each short trajectory, obtain discriminative trajectory features through feature interaction, and enhance the description of trajectory features in the current frame based on the trajectory features of the previous frame.

[0039] Step 103: Measure the spatiotemporal relationship between trajectory features, and match short trajectories according to the spatiotemporal relationship to generate a long trajectory.

[0040] After obtaining the trajectory features of each short trajectory, the short trajectory of each target is combined into a long trajectory by measuring the spatiotemporal relationship between the short trajectories and performing short trajectory matching.

[0041] Step 104: restore the occluded targets between different long trajectories and generate a smooth trajectory of the target.

[0042] When there is no occlusion on the target trajectory, the long trajectory of each target is the target trajectory; when there is occlusion, the motion trajectory of each target is divided into multiple long trajectories by the occluded part.

[0043] Since the motion trajectory of the same target should be a continuous and smooth spatial movement on a continuous timeline, the motion trajectory of the occluded part can also be regarded as a smooth trajectory. In specific implementation, a Gaussian smoothing interpolation algorithm can be used to restore the occluded target between adjacent long trajectories to generate a smooth trajectory of the target.

[0044] After steps 101 to 104 provided in this embodiment, multi-target trajectory tracking in the drone video can be achieved based on graph convolution and trajectory matching.

[0045] like Figure 2As shown, in step 102, the following method can be used to capture the salient areas inside each short trajectory using a graph convolutional spatiotemporal attention network, obtain discriminative trajectory features through feature interaction, and enhance the trajectory description based on the features of the previous frame.

[0046] Step 201: Use the Kalman filter to learn the historical trajectory of each target and output the predicted position of the target in the current frame.

[0047] Kalman filtering can predict the coordinate position and speed of a target from a finite sequence of target positions. In this embodiment, the Kalman filter is used to learn the historical trajectory of each target and output its predicted position in the current frame.

[0048] Step 202: Calculate the spatial similarity between the predicted position of the current frame and the detection result of the current frame, and measure the spatial similarity by intersection-over-union ratio to obtain a similarity matrix.

[0049] Perform target detection on the current frame and obtain the detection result of the current frame. Calculate the spatial similarity between the predicted position of the current frame and the detection result of the current frame. If the predicted position and the detection result match, it indicates that the prediction is correct. In this embodiment, there are multiple detection targets, and the spatial similarity of all targets can be expressed as a similarity matrix.

[0050] In a specific implementation, the spatial similarity can be measured by the intersection over union (IoU), which is the overlap ratio of the candidate bound generated in target detection and the original marked bound, that is, the ratio of the intersection and union of the candidate bound and the original marked bound.

[0051] Step 203: perform data association on the similarity matrix until the data association of each frame is completed, and obtain the short trajectory of the target according to the data association result.

[0052] In actual implementation, various data matching and combination algorithms can be used for data association. In one example, the Hungarian Algorithm is used for data association. The Hungarian Algorithm is a combinatorial optimization algorithm that solves the task allocation problem in polynomial time. It can find the maximum matching node in the graph and input the similarity matrix into the Hungarian Algorithm for data association to obtain the short trajectory between the current frame and the previous frame.

[0053] After step 201 to step 203 provided in this embodiment, a short track between a group of adjacent frames can be obtained. Step 201 to step 203 are repeated for each group of adjacent frames until the video ends, and all short tracks of all targets can be obtained.

[0054] like Figure 3 As shown, in step 103, the following method can be referred to, by measuring the spatiotemporal relationship between short trajectories and performing short trajectory matching to generate a high-quality long trajectory.

[0055] Step 301: Cut each short trajectory into trajectory segments of fixed length, input the trajectory segments into the graph convolutional spatiotemporal attention network, and obtain the node features of the graph convolutional neural network.

[0056] First, each short trajectory needs to be divided into trajectory segments of fixed length. In actual implementation, the length of the trajectory segment can be determined according to the actual scene requirements and the amount of calculation. In general scenes, the length can be set to 5 frames.

[0057] In a specific scenario, such as Figure 4 As shown in the figure, the trajectory fragment is input into the graph convolutional spatiotemporal attention network, which includes the attention network and the graph convolutional neural network. First, the apparent features of the trajectory fragment are extracted through the backbone network (ResNet101) with shared parameters to obtain a 128-dimensional apparent feature vector. The spatial features of the trajectory fragment are extracted through a two-layer gated recurrent unit (Gated Recurrent Unit, abbreviated as GRU) network to obtain a 128-dimensional spatial feature vector. Then, the apparent features and spatial features are concatenated into a 256-dimensional fused feature vector as the node features of the graph convolutional neural network.

[0058] Step 302: Filter the neighbor nodes of each node between each layer of the graph convolutional neural network, aggregate the node features of each node and the neighbor nodes, so as to complete the feature interaction between different targets.

[0059] According to the spatial distance between nodes, the neighbor nodes of each node are screened between each layer of the graph convolutional network; the nodes are connected to their neighbor nodes to build edges, and the edges are connected to build a graph structure; using the graph topology mechanism, information is transmitted through node feature aggregation to achieve feature interaction between targets.

[0060] In a specific scenario, such as Figure 4 As shown in the figure, it contains 3 layers of graph convolutional networks. Each layer of graph convolutional networks first selects the neighbor nodes of each node based on the spatial distance to avoid full connection between nodes and save computation. The nodes with a center distance of no more than 200 are considered neighbor nodes. Connect the nodes with each neighbor node to build edges, connect the edges to build a graph structure, and then use the graph topology mechanism to transfer information through node feature aggregation to achieve feature interaction between targets. The aggregation of node features is shown in the following formula:

[0061]

[0062] in, is the i-th node feature of the t-th frame, is the jth node feature of the (t+1)th frame; σ is the linear layer, σ1 is the first linear layer, and σ2 is the second linear layer; N(i) is the neighbor set of node i; A ij is the edge weight of the current graph convolution layer, that is, the trajectory feature of the edge.

[0063] Step 303: Based on the feature interaction result, the similarity distance between the trajectory segments is calculated, the feature difference between different nodes is calculated according to the similarity distance, and the discriminative trajectory features are obtained according to the feature difference.

[0064] The edge regression of the graph convolutional network is calculated, that is, the similarity matrix between the trajectory segments is calculated, and the feature differences between the nodes are calculated through a two-layer multi-layer perceptron (MLP), so that the features of the same nodes are close to each other and the features of different nodes are far away from each other. The trajectory features are shown in the following formula:

[0065]

[0066] Among them, A ij Represents the trajectory feature of the edge, which is a discriminative 256-dimensional trajectory feature; Sigmoid(·) is the activation function of the graph convolutional network, σ is the linear layer, σ3 is the third linear layer, σ4 is the fourth linear layer, ReLU(·) means using the ReLU activation function, is the i-th node feature of the t-th frame.

[0067] In this graph convolutional network, the cross entropy loss function of positive and negative edge weights can be used:

[0068]

[0069] Among them, A ij is the edge weight of the current graph convolution layer, that is, the trajectory feature of the edge; is the ground truth between node i and node j, is the true value between nodes in the tth frame.

[0070] After steps 301 to 303 provided in this embodiment, the trajectory features of the short trajectory can be obtained.

[0071] like Figure 5 As shown, in step 103, the spatiotemporal relationship between trajectory features may be measured in the following manner, and short trajectory matching may be performed according to the spatiotemporal relationship to generate a long trajectory.

[0072] Step 401: obtaining the spatiotemporal relationship between short trajectories according to the trajectory features, and obtaining the metric function between the short trajectories according to the spatiotemporal relationship.

[0073] Short track T i and T j The spatiotemporal relationship measure W between i,j It can be calculated by the following metric function, where the metric function includes spatial similarity, temporal similarity and apparent similarity:

[0074] W i,j =1-S s (T i ,T j )*S t (T i ,T j )*S v (T i ,T j );

[0075] Among them, W i,j is the spatiotemporal relationship metric, S s (T i ,T j ) is the spatial similarity, S t (T i ,T j ) is the time similarity, S v (T i ,T j ) is the apparent similarity.

[0076] Step 402: Obtain the spatiotemporal relationship measurement value between the short trajectories according to the measurement function, and match the short trajectories based on the spatiotemporal relationship measurement value to generate a long trajectory.

[0077] According to the metric function W between short trajectories i,j , match short trajectories and generate long trajectories.

[0078] After step 401 and step 402 provided in this embodiment, the long trajectory of the target can be obtained.

[0079] In a specific implementation, the spatial similarity, temporal similarity and apparent similarity may be calculated in the following manner.

[0080] (1) Spatial similarity.

[0081] A single target tracker is used to search the coordinates of the historical trajectory corresponding to each short trajectory at the current moment, and the spatial distance between the short trajectories is calculated based on the coordinates; the spatial distance is normalized to the maximum and minimum, and the spatial similarity between the short trajectories is obtained based on the normalized result.

[0082] Spatial similarity S s (T i ,T j ) represents the short trajectory T i and T j The spatial similarity between them. Due to the short trajectory T i and T j There is a time difference between them, and the target may be moving. Therefore, the spatial similarity between the two cannot be directly calculated based on their coordinates. Therefore, we first use the Discriminative Model Prediction (DIMP) to search the historical trajectory T i exist The coordinates of the moment, and then use the coordinates To calculate the spatial distance S between trajectories d (T i ,T j ):

[0083]

[0084] Among them, S d (T i ,T j ) is the trajectory T i and T j The spatial distance between Represents the candidate trajectory T j The starting coordinates of Representative trajectory T j The starting frame of T is the historical trajectory i exist The coordinates of the moment.

[0085] Get the spatial distance S d (T i ,T j ), and then use the maximum and minimum normalization to normalize the spatial distance S d (T i ,T j ) is normalized to obtain the spatial similarity S s (T i ,T j ). s (T i ,T j ) is higher, the shorter the trajectory T i and T j The greater the probability of belonging to the same target.

[0086] (2) Temporal similarity.

[0087] The time domain intervals between different short trajectories are obtained, and the time similarity between the short trajectories is obtained according to the time domain intervals.

[0088] After the target is blocked, it will still reappear in the continuous time domain, and the short trajectory T i and T j The smaller the time domain interval between them, the greater the probability that the two short trajectories belong to the same target, as shown in the following formula:

[0089]

[0090] Among them, S t (T i ,T j ) is the time similarity, Δt is the short trajectory T i and T j The time interval between .

[0091] (3) Apparent similarity.

[0092] According to the cosine similarity between the trajectory features of different short trajectories, the apparent similarity between the short trajectories is obtained according to the cosine similarity.

[0093] Calculate the cosine similarity of the short trajectory features generated by the graph convolutional spatiotemporal attention network. The higher the cosine similarity, the higher the trajectory T i and T j The higher the similarity between them, the higher the cosine similarity is. The cosine similarity is used as the apparent similarity, as shown in the following formula:

[0094]

[0095] Among them, S v (T i ,T j ) is the apparent similarity, f i T and Represent the short trajectory T i and T j apparent characteristics.

[0096] The method for tracking multiple target trajectories in a video provided by this embodiment has the following beneficial effects.

[0097] (1) It can fully explore the spatiotemporal context information of the trajectory through feature interaction and extract more discriminative spatiotemporal features of the trajectory.

[0098] (2) It can alleviate the problem of identity jump caused by target occlusion and improve the robustness of the tracker in occluded environments.

[0099] (3) It can restore occluded targets and improve tracking accuracy.

[0100] Embodiment 2:

[0101] Based on the method for tracking multiple target trajectories in a video provided in the above embodiment 1, the present invention also provides a device for tracking multiple target trajectories in a video that can be used to implement the above method, such as Figure 6 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The device for tracking multiple target trajectories in a video of this embodiment includes one or more processors 11 and a memory 12. Figure 6 A processor 11 is taken as an example.

[0102] The processor 11 and the memory 12 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0103] The memory 12 is a non-volatile computer-readable storage medium as a method for tracking multiple target trajectories in a video, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the method for tracking multiple target trajectories in a video in Example 1. The processor 11 executes various functional applications and data processing of the device for tracking multiple target trajectories in a video by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, implementing the method for tracking multiple target trajectories in a video in Example 1.

[0104] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11, and these remote memories may be connected to the processor 11 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] The program instructions / modules are stored in the memory 12, and when executed by one or more processors 11, the method for tracking multiple targets in a video in the above embodiment 1 is executed, for example, the above described Figures 1 to 3 ,as well as Figure 5 The steps shown.

[0106] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Figure 6 A processor 11 in the embodiment may enable the one or more processors to execute the method for tracking multiple target trajectories in a video in embodiment 1, for example, executing the method described above. Figures 1 to 3 ,as well as Figure 5 The steps shown can also be implemented Figure 4The various modules and units described.

[0107] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0108] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0109] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for tracking multiple target trajectories in a video, characterized in that: include: generating at least one short trajectory of the target based on spatial similarity of images related to the target in the video; Capture the salient areas within each short trajectory, and perform feature interaction on the salient areas to obtain trajectory features; Measure the spatiotemporal relationship between trajectory features, match short trajectories based on the spatiotemporal relationship to generate long trajectories; Recover the occluded objects between different long trajectories and generate smooth trajectories of the objects.

2. The method for tracking multiple target trajectories in a video according to claim 1, characterized in that: The step of generating at least one short trajectory of the target based on the spatial similarity of target-related images in the video specifically includes: Use the Kalman filter to learn the historical trajectory of each target and output the predicted position of the target in the current frame; Calculate the spatial similarity between the predicted position of the current frame and the detection result of the current frame, and measure the spatial similarity by intersection and union ratio to obtain a similarity matrix; The similarity matrix is ​​data associated until the data association of each frame is completed, and the short trajectory of the target is obtained according to the data association result.

3. The method for tracking multiple target trajectories in a video according to claim 1, characterized in that: The capturing of the salient regions within each short track and performing feature interaction on the salient regions to obtain track features specifically includes: Each short trajectory is divided into trajectory segments of fixed length, and the trajectory segments are input into the graph convolutional spatiotemporal attention network to obtain the node features of the graph convolutional neural network; In each layer of graph convolutional neural network, the neighbor nodes of each node are screened, and the node features of each node and the neighbor nodes are aggregated to complete the feature interaction between different targets; Based on the feature interaction results, the similarity distance between trajectory segments is calculated, and the feature differences between different nodes are calculated according to the similarity distance. Based on the feature differences, discriminative trajectory features are obtained.

4. The method for tracking multiple target trajectories in a video according to claim 3, characterized in that: The neighbor nodes of each node are screened between each layer of the graph convolutional neural network, and node features of each node and neighbor nodes are aggregated to complete feature interaction between different targets, specifically including: According to the spatial distance between nodes, the neighbor nodes of each node are screened between each layer of graph convolutional network; Nodes are connected to their neighboring nodes to build edges, and each edge is connected to build a graph structure; By utilizing the graph topology mechanism and node feature aggregation, information is transmitted to achieve feature interaction between targets.

5. The method for tracking multiple target trajectories in a video according to claim 1, characterized in that: The measuring the spatiotemporal relationship between the trajectory features and matching the short trajectories according to the spatiotemporal relationship to generate the long trajectory specifically include: According to the trajectory features, the spatiotemporal relationship between short trajectories is obtained, and the metric function between short trajectories is obtained according to the spatiotemporal relationship; The spatiotemporal relationship measurement value between the short trajectories is obtained according to the measurement function, and the short trajectories are matched based on the spatiotemporal relationship measurement value to generate a long trajectory.

6. The method for tracking multiple target trajectories in a video according to claim 5, characterized in that: The step of obtaining a metric function between short trajectories according to the spatiotemporal relationship, wherein obtaining the spatial similarity in the metric function specifically includes: The single target tracker is used to search the coordinates of the historical trajectory corresponding to each short trajectory at the current moment, and the spatial distance between the short trajectories is calculated based on the coordinates; The spatial distance is normalized to the maximum and minimum, and the spatial similarity between short trajectories is obtained based on the normalized result.

7. The method for tracking multiple target trajectories in a video according to claim 5, characterized in that: The step of obtaining a metric function between short trajectories according to the spatiotemporal relationship, wherein obtaining the temporal similarity in the metric function specifically includes: The time domain intervals between different short trajectories are obtained, and the time similarity between the short trajectories is obtained according to the time domain intervals.

8. The method for tracking multiple target trajectories in a video according to claim 5, characterized in that: The step of obtaining a metric function between short trajectories according to the spatiotemporal relationship, wherein obtaining the apparent similarity in the metric function specifically includes: According to the cosine similarity between the trajectory features of different short trajectories, the apparent similarity between the short trajectories is obtained according to the cosine similarity.

9. A device for tracking multiple target trajectories in a video, characterized in that: The method comprises at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for multi-target trajectory tracking in a video according to any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the method for tracking multiple target trajectories in a video as described in any one of claims 1-8.

Images