A multi-target tracking method and system based on a space-time topology graph

Abstract

The application discloses a multi-target tracking method and system based on a space-time topology graph, and relates to the field of image target tracking.The method comprises the following steps: a target detection algorithm is used to detect an input image at time t, and state data of all detection targets at time t is obtained; the state data of each detection target at time t is taken as a one-type node, the trajectory of each tracked target at time t is taken as a two-type node, and a space-time topology graph is constructed by using appearance features of all detection targets at time t and trajectory features of all tracked targets at time t; a graph neural network model is used to calculate the similarity of nodes in the space-time topology graph; and based on the node similarity, the state data of all detection targets at time t is matched with the trajectory of all tracked targets at time t by using a Hungarian algorithm, and the tracking of all detection targets is completed.The application realizes long-term stable tracking of multi-targets.

Description

Technical Field

[0001] This application relates to the field of image target tracking technology, and in particular to a multi-target tracking method and system based on spatiotemporal topology graphs. Background Technology

[0002] In recent years, with the rapid development of intelligent transportation and monitoring technologies, perception has played a crucial role as an indispensable component. Existing intelligent transportation and monitoring equipment often use RGB cameras as input to detect and locate targets in the surrounding environment.

[0003] Multi-object tracking is a decision-making auxiliary function built upon object detection and localization. Multi-object tracking algorithms predict the state of each target based on the state detected by the detector, and then make decisions for intelligent tasks. By predicting the state of the targets, the risks associated with them can be analyzed, providing more precise control for intelligent decision-making, thereby improving the safety and comfort of intelligent devices. As one of the decision-making auxiliary functions of intelligent systems, multi-object tracking algorithms assign a state to each target in the detected scene and continuously track that target over time. This not only improves the information provided by the detector but also greatly assists in subsequent autonomous driving decisions.

[0004] For RGB cameras, images, as a dense 2D representation, express spatial information through pixel positions and pixel values. Therefore, effective texture information greatly enhances the accuracy of correlation. To address this issue, existing multi-object tracking algorithms often use Kalman filters as input trackers, calculating the intersection-union ratio (IUU) based on predicted positions and bounding boxes provided by detectors to achieve multi-object tracking. To avoid losing the target's texture features by relying on the Kalman filter results, a re-identification network is constructed on top of this, using the target's texture as another key clue. By dividing the detection confidence, the algorithm's accuracy is effectively improved. Although the above algorithms can achieve multi-object tracking, their tracking model is based on the correlation between data from two frames, ignoring the influence of historical data on the current state. Therefore, the results are often difficult to apply to long-term tracking and may lead to tracking loss. Summary of the Invention

[0005] The purpose of this application is to provide a multi-target tracking method and system based on spatiotemporal topology graphs, which can achieve long-term stable tracking of multiple targets.

[0006] To achieve the above objectives, this application provides the following solution:

[0007] Firstly, this application provides a multi-target tracking method based on a spatiotemporal topology graph, the multi-target tracking method based on a spatiotemporal topology graph comprising:

[0008] An object detection algorithm is used to detect the input image at time t, and the state data of all detected objects at time t is obtained; the state data includes: the coordinates of the bounding box, the category of the bounding box, and the confidence score of the bounding box;

[0009] Based on the state data of all detected targets at time t, the appearance features of all detected targets at time t are determined using a re-identification network;

[0010] Obtain the trajectories of all tracked targets at time t and the trajectory features of all tracked targets at time t;

[0011] The state data of each detected target at time t is taken as a first-class node, and the trajectory of each tracked target at time t is taken as a second-class node. Using the appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t, a spatiotemporal topology map is constructed.

[0012] The similarity of nodes in the spatiotemporal topology graph is calculated using a graph neural network model; the node similarity is the similarity between the first type of nodes and the second type of nodes; the graph neural network model includes a multilayer perceptron, a spatiotemporal feature integration network, and a message passing neural network connected in sequence;

[0013] Based on the node similarity, the Hungarian algorithm is used to match the state data of all detected targets at time t with the trajectories of all tracked targets at time t, thereby completing the tracking of all detected targets.

[0014] In a second aspect, this application also provides a computer system, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-target tracking method based on spatiotemporal topology graphs as described in the first aspect.

[0015] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0016] In addition to detecting the input image at time t and determining the state data of all detected targets at time t, this application also acquires the trajectories of all tracked targets at time t, fully considering the influence of historical data on the current state. To effectively match detected targets with tracked targets, this application constructs a spatiotemporal topology graph by treating the state data of detected targets as a first-class node and the trajectories of tracked targets as a second-class node. A graph neural network model is then used to capture the similarity between the first-class and second-class nodes in the spatiotemporal topology graph, and the state data of detected targets is effectively matched with the trajectories of tracked targets based on node similarity. Furthermore, this application implements feature updates of the spatiotemporal topology graph through a graph neural network, ensuring the continuity of detection and trajectory tracking in the time dimension, making this application more adaptable to long-term stable tracking of multiple targets. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the multi-target tracking method based on spatiotemporal topology graphs provided in this application embodiment;

[0019] Figure 2 This is a schematic diagram of the spatiotemporal feature integration network provided in the embodiments of this application;

[0020] Figure 3 A diagram illustrating the computational relationships within the spatiotemporal converter layer provided in this embodiment of the application;

[0021] Figure 4 This is an internal structure diagram of a computer system provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The purpose of this application is to provide a multi-target tracking method and system based on spatiotemporal topology graphs, which can achieve long-term stable tracking of multiple targets.

[0024] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Example 1

[0026] This embodiment provides a multi-target tracking method based on spatiotemporal topology graphs, such as... Figure 1 As shown, this multi-target tracking method based on spatiotemporal topology graphs includes:

[0027] Step S1: Use a target detection algorithm to detect the input image at time t, and obtain the state data of all detected targets at time t; where t > 0.

[0028] In this embodiment, an image sequence I = {I1, I2, ..., I...} is acquired using an RGB camera.t}, I t Let represent the input image at time t. For image sequence I, the YOLOX algorithm outputs the corresponding detection results D = {D1, D2, ..., D...}. t}, D t This represents the detection result of the input image at time t. Specifically, D t ={o1,o2,...,o M}, o M Let represent the state data of the Mth detected target in the input image at time t. In reality, there may be one or more detected targets in the input image at time t, i.e., M ≥ 1. For each detected target's state data, it includes the coordinates of the bounding box corresponding to the detected target, its category, and its confidence score, i.e., o M = {x1,y1,x2,y2,c,s}, where (x1,y1) and (x2,y2) are the coordinates of the diagonal vertices of the bounding box, c is the category of the bounding box, and s is the confidence score of the bounding box.

[0029] Step S2: Based on the state data of all detected targets at time t, use the re-identification network to determine the appearance features of all detected targets at time t.

[0030] In this embodiment, in order to determine the appearance features of the detected targets, for the state data of each detected target, it is necessary to use the OSNet network to obtain its corresponding appearance features. Then, the appearance features of all detected targets in the input image at time t can be represented as F. t ={f1,f2,...,f M} represents the appearance features of the Mth detected target in the input image at time t.

[0031] Step S3: Obtain the trajectories of all tracked targets at time t and the trajectory features of all tracked targets at time t.

[0032] In this embodiment, to facilitate the matching of the detected target with the tracked target, it is also necessary to obtain the trajectories ζ={τ1,τ2,...,τ1,τ2,τ3,τ4,τ5,τ6,τ7,τ8,τ9,τ1,τ1,τ2,τ1,τ2,τ3,τ4,τ5,τ6 ...7,τ8,τ9,τ1,τ C}, τ C Let be the trajectory of the Cth tracked target at time t. The trajectory features of each tracked target at time t are actually composed of multiple appearance features of the tracked target.

[0033] Step S4: Treat the state data of each detected target at time t as a first-class node and the trajectory of each tracked target at time t as a second-class node. Construct a spatiotemporal topology map using the appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t.

[0034] In this embodiment, step S4 specifically includes:

[0035] Step S41: Take the state data of each detected target at time t as a first-class node and the trajectory of each tracked target at time t as a second-class node to construct a node set.

[0036] The state data of each detected target, and the trajectory of each tracked target, will be stored as a node in a node set V, where V = {o1,...,o...} M ,τ1,...,τ C The nodes in the node set V are further divided into Class I nodes and Class II nodes. All Class I nodes form Class I node set F, and all Class II nodes form Class II node set H.

[0037] Step S42: Use the appearance features of each detected target at time t as the features of each Class I node, and use the trajectory features of each tracked target at time t as the features of each Class II node.

[0038] Each node in the node set V has its own features. The features of the first type of node correspond to the appearance features of the detected target, and the features of the second type of node correspond to the trajectory features of the tracked target. The features of the second type of node are essentially still the appearance features of the tracked target. Therefore, the features of the first type of node and the second type of node are consistent in terms of attributes.

[0039] Step S43: Based on the features of all type I nodes and all type II nodes, model the features of the edges between type I nodes and type II nodes in the node set to obtain the spatiotemporal topology graph.

[0040] Not every two nodes in the node set V need to have an edge constructed. Because this embodiment searches for the similarity between the state data of the detected target and the trajectory of the tracked target, edges need to be constructed between first-class nodes and second-class nodes. Edges between first-class nodes and between second-class nodes do not need to be constructed. The final edge set is E = {e...} 1,1 ,e 1,2 ...e i,j The spatiotemporal topological graph is represented as G = (V, E).

[0041] The above modeling process is as follows:

[0042]

[0043] e i,j Let x be the feature of the edge between the i-th type 1 node and the j-th type 2 node. i y i Let x and y be the x and y coordinates of the center coordinates of the i-th type I node, respectively. j y jv represents the x-coordinate and y-coordinate of the center coordinates of the j-th binary node, respectively. x v is the lateral velocity in the pixel coordinate system. y Let t be the vertical velocity in the pixel coordinate system. i Let t be the node time of the i-th type 1 node. j Let h be the node time of the j-th binary node. i w i Let h be the height and width of the bounding box of the i-th type 1 node (calculated from the bounding box coordinates). j w j Let f be the height and width of the bounding box of the j-th binary node (calculated from the bounding box coordinates), respectively. i Let f be the feature of the i-th first-class node. j Let be the feature of the j-th binary node.

[0044] Step S5: Calculate the node similarity in the spatiotemporal topology graph using a graph neural network model; the node similarity is the similarity between Class I nodes and Class II nodes.

[0045] In this embodiment, the graph neural network model includes a multilayer perceptron, a spatiotemporal feature fusion network, and a message-passing neural network connected in sequence. The multilayer perceptron encodes the features of nodes and edges in the spatiotemporal topology graph; the spatiotemporal feature fusion network fuses the trajectories of each tracked target in the spatiotemporal topology graph across multiple time points; and the message-passing neural network updates the features of the spatiotemporal topology graph. Figure 2 As shown, the spatiotemporal feature integration network includes sequentially connected spatiotemporal transformer layers (i.e., Temporal-SpatialTransformer layers; see [link to internal computational relationships]). Figure 3 ( ), linear layer, first normalization layer, feedforward layer and second normalization layer.

[0046] Specifically, in addition to obtaining the node representations corresponding to the detection and trajectory based on the spatiotemporal topology graph G=(V,E), it is also necessary to use the multilayer perceptron in the graph neural network model to transform the data dimension of E into d. t Meanwhile, since the tracked target's trajectory contains trajectory data from multiple historical moments, a spatiotemporal feature fusion network is needed to fuse the tracked target's trajectory data across multiple moments in order to represent the overall trajectory state. After the fusion is completed, a multilayer perceptron is required to process the features of nodes and edges in the spatiotemporal topology graph to generate... and Then, a message-passing neural network is used to update the state of other nodes, thereby ensuring that the update of each graph node is fed back to other nodes in the spatiotemporal topology graph.

[0047] like Figure 3As shown, the trajectory of each tracked target in the spatiotemporal topology map is fused over multiple time points, specifically as follows:

[0048]

[0049] In the formula, Z represents the trajectory of the tracked target after multi-time fusion, ε represents the attention score of the second-class node at multiple time steps, and a i,j F is the result after softmax row normalization of ε. c For the trajectory data of all tracked targets at time t, F p For all tracked targets within a time period tL, where L is the set time duration and d is the trajectory data. t W represents the data dimension after feature processing of the edges at time t by the multilayer perceptron. Q For F c The query mapping matrix, W K For F p The key-value mapping matrix, W V For F p The final value mapping matrix.

[0050] Furthermore, the spatiotemporal topology graph is updated in terms of features, specifically as follows:

[0051]

[0052] In the formula, (ν→e) represents the feature transfer from the node to the edge, and (e→v) represents the feature transfer from the edge to the node. These are the learning parameters for the features of the edges in the message-passing neural network. These are the learning parameters for the features of nodes in the message-passing neural network. The features of the edge between the i-th type 1 node and the j-th type 2 node in the l-th iteration are given. The features of the edge between the i-th type 1 node and the j-th type 2 node in the (l-1)-th iteration are given. For the features of the i-th class 1 node in the l-th iteration, For the features of the i-th class 1 node in the (l-1)th iteration, The features of the j-th binary node in the (l-1)th iteration are... for arrive The transmission of Φ(·) is a network whose features are encoded based on the node connection relationships of the spatiotemporal topology graph.

[0053] In a preferred implementation, the graph neural network model needs to be trained before application to better achieve subsequent tracking tasks. To achieve effective training of the graph neural network model, the node similarity loss is divided into two categories: the first is the node similarity loss calculated based on node features, and the second is the node similarity loss calculated based on edge features. The final total node similarity loss consists of these two parts, and the graph neural network model is trained with the goal of minimizing the total node similarity loss. The loss function of the graph neural network model during the training phase is:

[0054]

[0055] In the formula, The total loss is the node similarity. This is the first type of node similarity loss. For the second type of node similarity loss, A ij This refers to the similarity between the i-th class 1 node and the j-th class 2 node, output by the graph neural network model based on the node features. For A ij The result of performing softmax row normalization, For A ij The result of performing softmax column normalization, Softmax(·) r To normalize the softmax rows, use Softmax(·). c For softmax column normalization, M represents the number of first-class nodes, and C represents the number of second-class nodes. Let p be the label value of the true similarity between the i-th type 1 node and the j-th type 2 node, and let p be the node similarity output by the graph neural network model based on the edge features. t Let y be the probability that the detected target and the tracked target belong to the same target, and let y be the true label of whether the detected target and the tracked target belong to the same target.

[0056] Step S6: Based on node similarity, use the Hungarian algorithm to match the state data of all detected targets at time t with the trajectories of all tracked targets at time t, and complete the tracking of all detected targets.

[0057] In this embodiment, the graph neural network model outputs node similarity d(i,j), and then the Hungarian algorithm is used to calculate the matching relationship B = {b} based on d(i,j). i,j} M×C b i,j Let b be the correlation coefficient between the i-th type I node and the j-th type II node. i,j=1 indicates that the two targets have been paired; otherwise, the pairing is considered unsuccessful. This method enables continuous tracking of image sequences. If d(i,j) is greater than the set threshold 0.5, the state data of the detected target at that moment is saved in the trajectory of its corresponding tracked target; otherwise, it is considered a new tracked target. For a matched detected target, its state data will be used to update the state of the corresponding trajectory, while the old state will be retained. For online multi-target tracking, due to limited storage space, it is necessary to delete and add the trajectories of tracked targets in a timely manner. For a trajectory, if its existence time is less than the first threshold A... a If it exceeds the second threshold A, it will be deleted; max If a detection is less than the third threshold τ, it will be considered unassociated and the trajectory will be deleted. Detections less than the third threshold τ will also not be constructed as new trajectories.

[0058] Before executing the multi-target tracking method based on the spatiotemporal topology graph, there are no tracks of tracked targets. When processing the second frame image, the state data of the detected targets in the previous frame image is used as the initial track and added to the track pool. The purpose of tracking is to calculate the similarity between the state data of the detected targets in the second frame image and the initial track determined in the first frame image. Then, by continuously repeating the above steps S1 to S5, new tracks will be generated and stored in the track pool. Detection is always the output of the target detection algorithm at the latest time step, thus achieving tracking.

[0059] Based on the above analysis, the multi-target tracking method based on spatiotemporal topology graph integrates spatiotemporal features based on historical data, updates the state of nodes based on message passing neural network, designs a loss function corresponding to node similarity, constructs effective trajectory management, realizes stable multi-target tracking, and solves the multi-target tracking problem under long-term clues.

[0060] Example 2

[0061] This embodiment provides a computer system, which can be a server or a terminal, and its internal structure diagram can be as follows. Figure 4As shown, the computer system includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores input image data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned multi-target tracking method based on a spatiotemporal topology map.

[0062] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer system to which the present application is applied. A specific computer system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0063] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0064] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0065] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0066] All actions involving the acquisition of signals, information, or data in this application are carried out in accordance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the owner of the relevant device.

[0067] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0068] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multi-target tracking method based on spatiotemporal topology graphs, characterized in that, The multi-target tracking method based on spatiotemporal topology graphs includes: An object detection algorithm is used to detect the input image at time t, and the state data of all detected objects at time t is obtained; the state data includes: the coordinates of the bounding box, the category of the bounding box, and the confidence score of the bounding box; Based on the state data of all detected targets at time t, the appearance features of all detected targets at time t are determined using a re-identification network; Obtain the trajectories of all tracked targets at time t and the trajectory features of all tracked targets at time t; The state data of each detected target at time t is taken as a first-class node, and the trajectory of each tracked target at time t is taken as a second-class node. Using the appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t, a spatiotemporal topology map is constructed. A graph neural network model is used to calculate the node similarity in the spatiotemporal topology graph; the node similarity is the similarity between the first-class nodes and the second-class nodes; the graph neural network model includes a multilayer perceptron, a spatiotemporal feature integration network, and a message-passing neural network connected in sequence; the multilayer perceptron is used to encode the features of the nodes and edges in the spatiotemporal topology graph; the spatiotemporal feature integration network is used to fuse the trajectory of each tracked target in the spatiotemporal topology graph at multiple time points; the message-passing neural network is used to update the features of the spatiotemporal topology graph; Based on the node similarity, the Hungarian algorithm is used to match the state data of all detected targets at time t with the trajectories of all tracked targets at time t, thereby completing the tracking of all detected targets. Specifically, the trajectory of each tracked target in the spatiotemporal topology map is fused across multiple time points, as follows: In the formula, Z represents the trajectory of the tracked target after multi-time fusion, C represents the number of binary nodes at time t, ε represents the attention score of the binary nodes at multiple time steps, and a i,j F is the result after softmax row normalization of ε. c For the trajectory data of all tracked targets at time t, F p For trajectory data of all tracked targets within a time period tL, where L is the set duration and d is the time interval. t W represents the data dimension after feature processing of the edges at time t by the multilayer perceptron. Q For F c The query mapping matrix, W K For F p The key-value mapping matrix, W V For F p The final value mapping matrix; The spatiotemporal topology graph is updated in the following ways: In the formula, (ν→e) represents the feature transfer from the node to the edge, and (e→v) represents the feature transfer from the edge to the node. These are the learning parameters for the features of the edges in the message-passing neural network. These are the learning parameters for the features of nodes in the message-passing neural network. The features of the edge between the i-th type 1 node and the j-th type 2 node in the l-th iteration are given. The features of the edge between the i-th type 1 node and the j-th type 2 node in the (l-1)-th iteration are given. For the features of the i-th class 1 node in the l-th iteration, For the features of the i-th class 1 node in the (l-1)th iteration, The features of the j-th binary node in the (l-1)th iteration are... for arrive The transmission of Φ(·) is a network that encodes features based on the node connection relationships in the spatiotemporal topology graph, and H is a set of two types of nodes; The loss function of the graph neural network model during the training phase is: In the formula, The total loss is the node similarity. This is the first type of node similarity loss. For the second type of node similarity loss, A ij This refers to the similarity between the i-th class 1 node and the j-th class 2 node, output by the graph neural network model based on the node features. For A ij The result of performing softmax row normalization, For A ij The result of performing softmax column normalization, Softmax(·) r To normalize the softmax rows, use Softmax(·). c For softmax column normalization, M represents the number of first-class nodes, and C represents the number of second-class nodes. Let p be the label value of the true similarity between the i-th type 1 node and the j-th type 2 node, and let p be the node similarity output by the graph neural network model based on the edge features. t Let y be the probability that the detected target and the tracked target belong to the same target, and let y be the true label of whether the detected target and the tracked target belong to the same target.

2. The multi-target tracking method based on spatiotemporal topology graphs according to claim 1, characterized in that, The process involves treating the state data of each detected target at time t as a first-class node and the trajectory of each tracked target at time t as a second-class node. A spatiotemporal topology map is constructed using the appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t. Specifically, this includes: The state data of each detected target at time t is taken as a type 1 node, and the trajectory of each tracked target at time t is taken as a type 2 node, thus constructing a node set; the type 1 nodes and the type 2 nodes are collectively referred to as nodes in the node set; The appearance features of each detected target at time t are used as the features of each of the first-class nodes, and the trajectory features of each tracked target at time t are used as the features of each of the second-class nodes. Based on the features of all type I nodes and all type II nodes, the features of the edges between type I nodes and type II nodes in the node set are modeled to obtain a spatiotemporal topology graph.

3. The multi-target tracking method based on spatiotemporal topology graphs according to claim 2, characterized in that, The features of the edges between type I nodes and type II nodes in the node set are modeled as follows: e i,j Let x be the feature of the edge between the i-th type 1 node and the j-th type 2 node. i y i Let x and y be the x and y coordinates of the center coordinates of the i-th type I node, respectively. j y j v represents the x-coordinate and y-coordinate of the center coordinates of the j-th binary node, respectively. x v is the lateral velocity in the pixel coordinate system. y Let t be the vertical velocity in the pixel coordinate system. i Let t be the node time of the i-th type 1 node. j Let h be the node time of the j-th binary node. i w i h represents the height and width of the bounding box of the i-th class 1 node, respectively. j w j f represents the height and width of the bounding box of the j-th binary node, respectively. i Let f be the feature of the i-th first-class node. j Let be the feature of the j-th binary node.

4. The multi-target tracking method based on spatiotemporal topology graphs according to claim 1, characterized in that, The spatiotemporal feature integration network includes a spatiotemporal transformer layer, a linear layer, a first normalization layer, a feedforward layer, and a second normalization layer connected in sequence.

5. The multi-target tracking method based on spatiotemporal topology graphs according to claim 1, characterized in that, The target detection algorithm is the YOLOX algorithm; the re-identification network is the OSNet network.

6. A computer system, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the multi-target tracking method based on a spatiotemporal topology graph as described in any one of claims 1-5.

Images