Multi-target tracking method and system based on spatio-temporal topological graph

Abstract

The invention discloses a multi-target tracking method and system based on a spatio-temporal topological graph, and relates to the field of image target tracking, and the method comprises the steps: employing a target detection algorithm to carry out the detection of an input image at a t moment, and obtaining the state data of all detection targets at the t moment; using the state data of each detection target at the t moment as a first-class node, using the trajectory of each tracked target at the t moment as a second-class node, and using the appearance features of all the detection targets at the t moment and the trajectory features of all the tracked targets at the t moment to construct a space-time topological graph; calculating node similarity in the spatio-temporal topological graph by using a graph neural network model; and based on the node similarity, matching the state data of all the detection targets at the t moment with the tracks of all the tracked targets at the t moment by using a Hungary algorithm to complete tracking of all the detection targets. According to the invention, long-term stable tracking of multiple targets is realized.

Description

Technical Field

[0001] The present application relates to the technical field of image target tracking, and in particular to a multi-target tracking method and system based on a spatiotemporal topological graph. Background Art

[0002] In recent years, with the rapid development of intelligent transportation and monitoring technology, perception plays an important role as an indispensable part. Existing intelligent transportation equipment and monitoring equipment often use RGB cameras as input to detect and locate targets in the surrounding environment.

[0003] Multi-target tracking is an auxiliary decision-making function based on the target detection and positioning functions. The multi-target tracking algorithm predicts the state of each target based on the state of the target detected by the detector, and further completes the decision of the intelligent task. By predicting the state of the target, it is possible to analyze the risks of the target, provide more accurate control for intelligent decision-making, and thus improve the safety and comfort of intelligent devices. As one of the auxiliary decision-making functions of the intelligent system, the multi-target tracking algorithm assigns states to the targets in the detection scene one by one and keeps tracking the target as time goes by. This not only improves the information of the detector, but also provides great help for subsequent autonomous driving decisions.

[0004] For RGB cameras, images are a dense 2D representation that expresses spatial information through pixel positions and pixel values, so effective texture information will greatly enhance the accuracy of association. To address this problem, existing multi-target tracking algorithms often use the Kalman filter as the input tracker, and calculate the intersection-over-union ratio by predicting the position and the bounding box provided by the detector to achieve multi-target tracking; and in order to avoid losing the texture features of the target by relying on the results of the Kalman filter, a re-identification network is constructed on the above basis, and the texture of the target is used as another key clue. By dividing the confidence of the detection, the accuracy of the algorithm is effectively improved. Although the above algorithm can achieve multi-target tracking, its tracking model is the association of data between two frames, ignoring the impact of historical data on the current state. The results are often difficult to apply to long-term tracking and may lead to tracking loss. Summary of the invention

[0005] The purpose of this application is to provide a multi-target tracking method and system based on a spatiotemporal topological graph, which can achieve long-term stable tracking of multiple targets.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a multi-target tracking method based on a spatiotemporal topological graph, the multi-target tracking method based on a spatiotemporal topological graph comprising:

[0008] Using a target detection algorithm to detect the input image at time t, and obtaining state data of all detected targets at time t; the state data includes: coordinates of a bounding box, a category of the bounding box, and a confidence level of the bounding box;

[0009] Based on the state data of all detected targets at time t, the re-identification network is used to determine the appearance features of all detected targets at time t;

[0010] Obtain the trajectories of all tracked targets at time t and the trajectory features of all tracked targets at time t;

[0011] The state data of each detected target at time t is regarded as a first-class node, and the trajectory of each tracked target at time t is regarded as a second-class node. The appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t are used to construct a spatiotemporal topological graph.

[0012] The graph neural network model is used to calculate the node similarity in the spatiotemporal topology graph; the node similarity is the similarity between the first type of nodes and the second type of nodes; the graph neural network model includes a multi-layer perceptron, a spatiotemporal feature integration network and a message passing neural network connected in sequence;

[0013] Based on the node similarity, the Hungarian algorithm is used to match the state data of all detected targets at time t with the trajectories of all tracked targets at time t to complete the tracking of all detected targets.

[0014] In a second aspect, the present application also provides a computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-target tracking method based on the spatiotemporal topological map described in the first aspect.

[0015] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0016] In addition to detecting the input image at time t and determining the state data of all detected targets at time t, this application also obtains the trajectories of all tracked targets at time t, fully considering the impact of historical data on the current state. In order to effectively match the detected target with the tracked target, this application also regards the state data of the detected target as a first-class node and the trajectory of the tracked target as a second-class node, constructs a spatiotemporal topological graph, and uses a graph neural network model to capture the similarity between the first-class nodes and the second-class nodes in the spatiotemporal topological graph, and effectively matches the state data of the detected target with the trajectory of the tracked target through the node similarity. In addition, this application also implements the feature update of the spatiotemporal topological graph through a graph neural network, fully ensuring the continuity of detection and trajectory in the time dimension, making this application more adaptable to the long-term stable tracking of multiple targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 A flowchart of a multi-target tracking method based on a spatiotemporal topological graph provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of the structure of the spatiotemporal feature integration network provided in the embodiment of the present application;

[0020] Figure 3 A diagram of the operation relationship within the space-time transformer layer provided in an embodiment of the present application;

[0021] Figure 4 This is a diagram of the internal structure of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The purpose of this application is to provide a multi-target tracking method and system based on a spatiotemporal topological graph, which can achieve long-term stable tracking of multiple targets.

[0024] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0025] Example 1

[0026] This embodiment provides a multi-target tracking method based on a spatiotemporal topological graph. Figure 1 As shown, the multi-target tracking method based on the spatiotemporal topological graph includes:

[0027] Step S1: Use a target detection algorithm to detect the input image at time t, and obtain the status data of all detected targets at time t; wherein t>0.

[0028] In this embodiment, an image sequence I={I1, I2, ..., It}, I t Represents the input image at time t. For the image sequence I, the YOLOX algorithm outputs its corresponding detection result D = {D1, D2, ..., D t}, D t represents the detection result of the input image at time t. Specifically, D t ={o1,o2,...,o M},o M represents the state data of the Mth detected target in the input image at time t. In fact, the detected target in the input image at time t may be one or more, that is, M≥1. For the state data of each detected target, it includes the coordinates, category and confidence of the bounding box corresponding to the detected target, that is, o M ={x1,y1,x2,y2,c,s}, (x1,y1), (x2,y2) are the coordinates of the diagonal vertices of the bounding box, c is the category of the bounding box, and s is the confidence of the bounding box.

[0029] Step S2: Based on the state data of all detected targets at time t, the re-identification network is used to determine the appearance features of all detected targets at time t.

[0030] In this embodiment, in order to determine the appearance features of the detection target, for the state data of each detection target, it is necessary to use the OSNet network to obtain its corresponding appearance features. Then, the appearance features of all detection targets in the input image at time t can be expressed as F t ={f1,f2,...,f M}, represents the appearance features of the Mth detected target in the input image at time t.

[0031] Step S3: Obtain the trajectories of all tracked targets at time t and the trajectory features of all tracked targets at time t.

[0032] In this embodiment, in order to facilitate the matching of the detection target with the tracked target, it is also necessary to obtain the trajectory of all tracked targets at time t ζ = {τ1, τ2, ..., τ C}, τ C is the trajectory of the Cth tracked target at time t. The trajectory feature of each tracked target at time t is actually composed of multiple appearance features of the tracked target.

[0033] Step S4: Take the state data of each detected target at time t as a first-class node, take the trajectory of each tracked target at time t as a second-class node, and use the appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t to construct a spatiotemporal topological graph.

[0034] In this embodiment, step S4 specifically includes:

[0035] Step S41: The state data of each detected target at time t is taken as a first-class node, and the trajectory of each tracked target at time t is taken as a second-class node to construct a node set.

[0036] The state data of each detected target and the trajectory of each tracked target will be stored as a node in the node set V, V = {o1,...,o M ,τ1,...,τ C}, and the nodes in the node set V are divided into first-class nodes and second-class nodes. All first-class nodes form the first-class node set F, and all second-class nodes form the second-class node set H.

[0037] Step S42: taking the appearance features of each detected target at time t as the features of each first-class node, and taking the trajectory features of each tracked target at time t as the features of each second-class node.

[0038] Among them, each node in the node set V has its own characteristics. The characteristics of the first type of nodes correspond to the appearance characteristics of the detected target, and the characteristics of the second type of nodes correspond to the trajectory characteristics of the tracked target. The characteristics of the second type of nodes are essentially the appearance characteristics of the tracked target, so the characteristics of the first type of nodes and the second type of nodes are consistent in attributes.

[0039] Step S43: Based on the features of all the first-class nodes and the features of all the second-class nodes, the features of the edges between the first-class nodes and the second-class nodes in the node set are modeled to obtain a spatiotemporal topological graph.

[0040] Among them, not any two nodes in the node set V need to build edges. Because this embodiment is looking for the similarity between the state data of the detected target and the trajectory of the tracked target, it is necessary to build edges between the first type of nodes and the second type of nodes. The edges between the first type of nodes and the second type of nodes do not need to be built. The final edge set is E = {e 1,1 ,e 1,2 ……e i,j}, the space-time topology graph is represented by G = (V, E).

[0041] The above modeling process is specifically as follows:

[0042]

[0043] e i,j is the feature of the edge between the i-th first-class node and the j-th second-class node, x i ,y i are the horizontal and vertical coordinates of the center coordinates of the i-th type one node, respectively, j ,y jare the horizontal and vertical coordinates of the center coordinates of the jth second-class node, respectively, v x is the lateral velocity in the pixel coordinate system, v y is the longitudinal velocity in the pixel coordinate system, t i is the node time of the i-th class one node, t j is the node time of the jth second-class node, h i 、w i are the height and width of the bounding box of the i-th class 1 node (which can be calculated by the coordinates of the bounding box), h j 、w j are the height and width of the bounding box of the jth second-class node (which can be calculated by the coordinates of the bounding box), f i is the feature of the i-th class 1 node, f j is the feature of the jth second-class node.

[0044] Step S5: Calculate the node similarity in the spatiotemporal topology graph using the graph neural network model; the node similarity is the similarity between the first type of nodes and the second type of nodes.

[0045] In this embodiment, the graph neural network model includes a multi-layer perceptron, a spatiotemporal feature integration network, and a message passing neural network connected in sequence. The multi-layer perceptron is used to encode the features of nodes and edges in the spatiotemporal topology graph; the spatiotemporal feature integration network is used to fuse the trajectories of each tracked target in the spatiotemporal topology graph at multiple moments; and the message passing neural network is used to update the features of the spatiotemporal topology graph. Figure 2 As shown in Figure 2, the spatiotemporal feature integration network includes sequentially connected spatiotemporal transformer layers (i.e., Temporal-SpatialTransformer layers, whose internal operation relationships are shown in Figure 2). Figure 3 ), a linear layer, a first normalization layer, a feed-forward layer, and a second normalization layer.

[0046] Specifically, in addition to obtaining the node representation corresponding to the detection and trajectory according to the spatiotemporal topological graph G = (V, E), it is also necessary to use the multi-layer perceptron in the graph neural network model to convert the data dimension of E into d t At the same time, since the trajectory of the tracked target has trajectory data at multiple historical moments, in order to express the overall state of the trajectory, it is necessary to use the spatiotemporal feature integration network to fuse the trajectory data of the tracked target at multiple moments. After the above fusion is completed, the multi-layer perceptron is required to process the features of the nodes and edges in the spatiotemporal topology graph to generate and Then, the message passing neural network is used to update the status of other nodes, thereby ensuring that the update of each graph node will be fed back to other nodes in the spatiotemporal topology graph.

[0047] like Figure 3As shown in the figure, the trajectory of each tracked target in the spatiotemporal topology graph is fused at multiple times, specifically:

[0048]

[0049] Where Z is the trajectory of the tracked target after multi-time fusion, ε is the multi-time attention score of the second type of node, and a i,j is the result of softmax row normalization of ε, F c is the trajectory data of all tracked targets at time t, F p is the trajectory data of all tracked targets within the time period tL, L is the set time length, d t is the data dimension after the multilayer perceptron processes the edge features at time t, W Q F c The query mapping matrix, W K F p The key-value mapping matrix, W V F p The final value mapping matrix.

[0050] Furthermore, the characteristics of the spatiotemporal topological graph are updated, specifically:

[0051]

[0052] In the formula, (ν→e) represents the feature transfer from the node to the edge, and (e→v) represents the feature transfer from the edge to the node. is the learning parameter of the message passing neural network for edge features, is the learning parameter of the message passing neural network for the node features, is the feature of the edge between the i-th first-class node and the j-th second-class node in the l-th iteration, is the feature of the edge between the i-th type 1 node and the j-th type 2 node in the l-1th iteration, is the feature of the i-th class one node in the l-th iteration, is the feature of the i-th class 1 node in the l-1th iteration, is the feature of the j-th second-class node in the l-1-th iteration, for arrive Φ(·) is a network that encodes features based on the node connection relationship of the spatiotemporal topological graph.

[0053] As a preferred implementation method, before applying the above-mentioned graph neural network model, it needs to be trained in order to better implement subsequent tracking tasks. In order to achieve effective training of the graph neural network model, the loss of node similarity is divided into two categories. The first category is the node similarity loss calculated based on the node features, and the second category is the node similarity loss calculated based on the edge features. The final total loss of node similarity is composed of the above two parts. The graph neural network model is trained with the goal of minimizing the total loss of node similarity. Among them, the loss function of the graph neural network model in the training stage is:

[0054]

[0055] In the formula, is the total loss of node similarity, is the first type of node similarity loss, is the second type of node similarity loss, A ij is the similarity between the i-th first-class node and the j-th second-class node output by the graph neural network model according to the characteristics of the nodes, For A ij The result of softmax row normalization is: For A ij The result of softmax column normalization, Softmax(·) r is the softmax row normalization, Softmax(·) c is the softmax column normalization, M is the number of class one nodes, C is the number of class two nodes, is the label value of the true similarity between the i-th first-class node and the j-th second-class node, p is the node similarity output by the graph neural network model based on the characteristics of the edge, and p t is the probability of whether the detected target and the tracked target belong to the same target, and y is the true label of whether the detected target and the tracked target belong to the same target.

[0056] Step S6: Based on the node similarity, the Hungarian algorithm is used to match the state data of all detected targets at time t with the trajectories of all tracked targets at time t to complete the tracking of all detected targets.

[0057] In this embodiment, the graph neural network model outputs the node similarity d(i, j), and then uses the Hungarian algorithm to calculate the matching relationship B={b i,j} M×C , b i,j is the correlation coefficient between the i-th first-class node and the j-th second-class node, if b i,j=1 indicates that the two are paired, otherwise the pairing is considered to have failed. In this way, continuous tracking of the image sequence can be achieved. If d(i,j) is greater than the set threshold of 0.5, the state data of the detected target at that moment is saved in the trajectory of the corresponding tracked target, otherwise it is regarded as a new tracked target. For the matched detected target, its state data will be used to update the state of the corresponding trajectory, and the old state will be retained. For an online multi-target tracking, due to limited storage space, it is necessary to delete and add the trajectory of the tracked target in a timely manner. For a trajectory, if its existence time is less than the first threshold A a , it will be deleted; for those exceeding the second threshold A max , it will be considered as not associated and the track will be deleted. For detections less than the third threshold τ, new tracks will not be constructed either.

[0058] Before executing the above-mentioned multi-target tracking method based on spatiotemporal topology graph, there is no track of the tracked target. When processing the second frame image, the state data of the detected target in the previous frame image will be added to the track pool as the initialized track. The purpose of tracking is to calculate the similarity between the state data of the detected target in the second frame image and the initialized track determined by the first frame image, and then repeat the above steps S1 to S5 continuously, so that new tracks will be generated and stored in the track pool, and the detection is always the output of the target detection algorithm at the latest moment, so as to achieve tracking.

[0059] Based on the above analysis, the multi-target tracking method based on the spatiotemporal topological graph integrates spatiotemporal features based on data from historical moments, updates the status of nodes based on a message passing neural network, designs a loss function corresponding to node similarity, constructs an effective trajectory management, realizes stable multi-target tracking, and solves the problem of multi-target tracking under long-term clues.

[0060] Example 2

[0061] This embodiment provides a computer system, which may be a server or a terminal, and its internal structure diagram may be as follows: Figure 4As shown. The computer system includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer system is used to provide computing and control capabilities. The memory of the computer system includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer system is used to store input image data. The input / output interface of the computer system is used to exchange information between the processor and an external device. The communication interface of the computer system is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the multi-target tracking method based on the spatiotemporal topology map is implemented.

[0062] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer system to which the scheme of the present application is applied. The specific computer system may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0063] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0064] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0065] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0066] All actions to obtain signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0067] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0068] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A multi-target tracking method based on spatiotemporal topological graph, characterized in that: The multi-target tracking method based on the spatiotemporal topological graph includes: Using a target detection algorithm to detect the input image at time t, and obtaining state data of all detected targets at time t; the state data includes: coordinates of a bounding box, a category of the bounding box, and a confidence level of the bounding box; Based on the state data of all detected targets at time t, the re-identification network is used to determine the appearance features of all detected targets at time t; Obtain the trajectories of all tracked targets at time t and the trajectory features of all tracked targets at time t; The state data of each detected target at time t is regarded as a first-class node, and the trajectory of each tracked target at time t is regarded as a second-class node. The appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t are used to construct a spatiotemporal topological graph. The graph neural network model is used to calculate the node similarity in the spatiotemporal topology graph; the node similarity is the similarity between the first type of nodes and the second type of nodes; the graph neural network model includes a multi-layer perceptron, a spatiotemporal feature integration network and a message passing neural network connected in sequence; Based on the node similarity, the Hungarian algorithm is used to match the state data of all detected targets at time t with the trajectories of all tracked targets at time t to complete the tracking of all detected targets.

2. The multi-target tracking method based on spatiotemporal topological graph according to claim 1, characterized in that: The state data of each detected target at time t is taken as a first-class node, the trajectory of each tracked target at time t is taken as a second-class node, and the appearance features of all detected targets at time t and the trajectory features of all tracked targets at time t are used to construct a spatiotemporal topological graph, which specifically includes: The state data of each detected target at time t is regarded as a first-class node, and the trajectory of each tracked target at time t is regarded as a second-class node, to construct a node set; the first-class nodes and the second-class nodes are collectively referred to as nodes in the node set; Taking the appearance feature of each detected target at time t as the feature of each of the first-category nodes, and taking the trajectory feature of each tracked target at time t as the feature of each of the second-category nodes; Based on the features of all the first-class nodes and the features of all the second-class nodes, the features of the edges between the first-class nodes and the second-class nodes in the node set are modeled to obtain a spatiotemporal topological graph.

3. The multi-target tracking method based on spatiotemporal topological graph according to claim 2 is characterized in that: The features of the edges between the first type of nodes and the second type of nodes in the node set are modeled, specifically: e i,j is the feature of the edge between the i-th first-class node and the j-th second-class node, x i ,y i are the horizontal and vertical coordinates of the center coordinates of the i-th type one node, respectively, j ,y j are the horizontal and vertical coordinates of the center coordinates of the jth second-class node, respectively, v x is the lateral velocity in the pixel coordinate system, v y is the longitudinal velocity in the pixel coordinate system, t i is the node time of the i-th class one node, t j is the node time of the jth type II node, h i 、w i are the height and width of the bounding box of the i-th class 1 node, h j 、w j are the height and width of the bounding box of the jth second-class node, respectively, and f i is the feature of the i-th class 1 node, f j is the feature of the jth second-class node.

4. The multi-target tracking method based on spatiotemporal topological graph according to claim 1, characterized in that: The multilayer perceptron is used to encode the features of nodes and edges in the space-time topology map; the space-time feature integration network is used to fuse the trajectories of each tracked target in the space-time topology map at multiple times; and the message passing neural network is used to update the features of the space-time topology map.

5. The multi-target tracking method based on spatiotemporal topological graph according to claim 4 is characterized in that: The trajectory of each tracked target in the spatiotemporal topological graph is fused at multiple times, specifically: Where Z is the trajectory of the tracked target after multi-time fusion, C is the number of class II nodes at time t, ε is the multi-time attention score of class II nodes, and a i,j is the result of softmax row normalization of ε, F c is the trajectory data of all tracked targets at time t, F p is the trajectory data of all tracked targets within the time period tL, L is the set time length, d t is the data dimension after the multilayer perceptron processes the edge features at time t, W Q F c The query mapping matrix, W K F p The key-value mapping matrix, W V F p The final value mapping matrix.

6. The multi-target tracking method based on spatiotemporal topological graph according to claim 4 is characterized in that: The feature of the spatiotemporal topological graph is updated, specifically: In the formula, (ν→e) represents the feature transfer from the node to the edge, and (e→v) represents the feature transfer from the edge to the node. is the learning parameter of the message passing neural network for edge features, is the learning parameter of the message passing neural network for the node features, is the feature of the edge between the i-th first-class node and the j-th second-class node in the l-th iteration, is the feature of the edge between the i-th type 1 node and the j-th type 2 node in the l-1th iteration, is the feature of the i-th class one node in the l-th iteration, is the feature of the i-th class 1 node in the l-1th iteration, is the feature of the j-th second-class node in the l-1-th iteration, for arrive , Φ(·) is a network that encodes features based on the node connection relationship of the spatiotemporal topological graph, and H is a set of two types of nodes.

7. The multi-target tracking method based on spatiotemporal topological graph according to claim 1, characterized in that: The loss function of the graph neural network model in the training phase is: In the formula, is the total loss of node similarity, is the first type of node similarity loss, is the second type of node similarity loss, A ij is the similarity between the i-th first-class node and the j-th second-class node output by the graph neural network model according to the characteristics of the nodes, For A ij The result of softmax row normalization is: For A ij The result of softmax column normalization, Softmax(·) r is the softmax row normalization, Softmax(·) c is the softmax column normalization, M is the number of class one nodes, C is the number of class two nodes, is the label value of the true similarity between the i-th first-class node and the j-th second-class node, p is the node similarity output by the graph neural network model based on the characteristics of the edge, and p t is the probability of whether the detected target and the tracked target belong to the same target, and y is the true label of whether the detected target and the tracked target belong to the same target.

8. The multi-target tracking method based on spatiotemporal topological graph according to claim 1, characterized in that: The spatiotemporal feature integration network includes a spatiotemporal transformer layer, a linear layer, a first normalization layer, a feedforward layer and a second normalization layer which are connected in sequence.

9. The multi-target tracking method based on spatiotemporal topological graph according to claim 1, characterized in that: The target detection algorithm is the YOLOX algorithm; the re-identification network is the OSNet network.

10. A computer system comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-target tracking method based on a spatiotemporal topological graph as described in any one of claims 1 to 9.

Images