Multi-uav multi-target tracking method and device, computer device and storage medium

Abstract

The application relates to a multi-unmanned aerial vehicle multi-target tracking method and device, computer equipment and a storage medium. After feature extraction is performed on multi-target images acquired by different unmanned aerial vehicles from different perspectives at the same observation moment, for the image features under each perspective, the image features at the current observation moment are subjected to feature enhancement according to the previous observation moment and prior multi-target trajectories, and then multi-target detection is performed, target detection frames are obtained, cross-space target identity recognition is realized based on the target detection result and target representation information under other perspectives, and finally the multi-target detection result is associated with the prior multi-target trajectory through the target identity recognition result to realize target tracking. The method can effectively improve multi-target tracking precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to a multi-drone multi-object tracking method and device, a computer device and a storage medium. BACKGROUND

[0002] Multi-drone multi-object tracking (MDMOT), which aims to predict the locations and identities of all targets from multiple simultaneously captured drone videos, is a potentially challenging task. It can be applied to many real-world scenarios, such as smart cities, earth observation, and military security.

[0003] Unlike traditional multi-object tracking (MOT), which only predicts from a single video, MDMOT needs to associate targets at multiple time instants in the temporal dimension and match common targets in different videos. Therefore, existing MDMOT methods mainly contain two steps, namely, localization and identification. Specifically, it first obtains the coordinate frame of each frame through a detection network, and then associates candidate targets at different time instants and different perspectives. Although effective, existing methods often lose targets when faced with challenges such as small targets and cluttered backgrounds in drone videos. At the same time, the current method is difficult to effectively apply cross-machine information, leading to difficulties in target association. SUMMARY

[0004] Therefore, it is necessary to provide a multi-drone multi-object tracking method and device capable of robust and effective cooperative long-term tracking, a computer device and a storage medium in view of the above technical problems.

[0005] A multi-drone multi-object tracking method, the method comprising:

[0006] obtaining a multi-target image data set, the multi-target image data set comprising a plurality of multi-target image subsets obtained by a plurality of drones from different perspectives for observing the same space, each of the multi-target image subsets comprising a plurality of multi-target images arranged in time sequence in the same perspective;

[0007] grouping the multi-target images corresponding to the same observation time in each of the multi-target image subsets into a group of input data sets, to obtain a plurality of groups of input data sets sorted according to observation time;

[0008] performing feature extraction on all multi-target images in a group of input data sets to obtain image features corresponding to each frame of multi-target image;

[0009] According to the prior multi-target track corresponding to the view angle and the multi-target detection result at the previous observation time, the image features at the current observation time are enhanced, and then multi-target detection is performed based on the enhanced image features to obtain the multi-target detection result at the view angle.

[0010] According to the multi-target detection result corresponding to the view angle, multi-target representation extraction is performed, and the extracted multi-target preliminary representation is cross-space interacted with the multi-target preliminary representation of other view angles to obtain the multi-target representation corresponding to the view angle.

[0011] The multi-target detection results at the current observation time under different view angles are identified according to the corresponding multi-target representation, and are associated with the prior multi-target track to realize multi-target tracking.

[0012] In one embodiment, the multi-target detection network is used to enhance the image features at the current observation time according to the prior multi-target track corresponding to the view angle and the multi-target detection result at the previous observation time, and then perform multi-target detection based on the enhanced image features to obtain the multi-target detection result at the view angle.

[0013] The target detection network includes a plurality of parallel sub-target detection networks corresponding to detecting targets in multi-target images under each different view angle.

[0014] In one embodiment, in each of the sub-target detection networks:

[0015] According to the multi-target detection result at the previous observation time and the prior multi-target track, the prior feature blocks corresponding to each target are cropped from the image features at the previous observation time.

[0016] According to the target category, the average value of all prior feature blocks under each target category is calculated, and the average value is used as the time sequence prior feature of the target category.

[0017] According to the time sequence prior feature of each target category, the image features at the current observation time under the same view angle are enhanced.

[0018] The detection head is used to perform multi-target detection based on the enhanced image features to obtain the multi-target detection result at the current observation time under the view angle.

[0019] In one embodiment, the enhancement of the image features at the current observation time under the same view angle according to the time sequence prior feature of each target category includes:

[0020] The image features of the current observation time are correlated with the time sequence prior features of each target category after convolution operation to obtain a positioning response graph corresponding to each target category;

[0021] The positioning response graphs of all target categories are summed to obtain a similarity graph, which is multiplied by the image features after convolution operation, and the image features are enhanced through residual connection.

[0022] In one embodiment, for different perspectives, multi-target feature extraction is performed according to the multi-target detection results corresponding to the perspective, and the extracted multi-target preliminary feature is interacted with the multi-target preliminary features of other perspectives to obtain the multi-target feature corresponding to the perspective, using a target feature extraction network;

[0023] The target feature extraction network includes a plurality of sub-target feature extraction networks corresponding to extracting target features in multi-target images under each different perspective;

[0024] Each of the sub-target feature extraction networks is connected to each other sub-target feature extraction network.

[0025] In one embodiment, in each of the sub-target feature extraction networks:

[0026] After encoding the multi-target detection results into a form that can be processed by the feature extraction network, the original target detection frame is expanded according to a preset expansion size to obtain an expanded detection frame;

[0027] According to the original target detection frame and the expanded detection frame, respectively, the corresponding local image features are obtained by cutting on the image features, and each of the local image features is fused to obtain a multi-target preliminary feature under a certain perspective;

[0028] The multi-target preliminary feature under a certain perspective is interacted with the multi-target preliminary features of other perspectives to obtain the multi-target feature corresponding to the perspective, wherein, when the cross-space interaction is performed, the embedding features of the same target under different perspectives are enhanced and interacted through the combination of linear layer, self-attention and mutual attention operation, and feedforward network to obtain the multi-target feature corresponding to the perspective.

[0029] In one embodiment, each group of input data sets is sequentially input into a multi-angle multi-target tracking network according to the arrangement order to realize multi-target tracking, wherein the multi-angle multi-target tracking network includes a feature extraction network, a target detection network, a target feature extraction network, and a correlation network;

[0030] The feature extraction network is used to extract features of all multi-target images in a group of input data sets to obtain image features corresponding to each frame of multi-target image;

[0031] The target detection network is used to perform feature enhancement on image features of a current observation time according to a prior multi-target trajectory corresponding to a view angle and a multi-target detection result of a previous observation time, and then perform multi-target detection based on the enhanced image features to obtain a multi-target detection result under the view angle;

[0032] The target representation extraction network is used to perform multi-target representation extraction according to a multi-target detection result corresponding to a view angle, and perform cross-space interaction between the extracted multi-target preliminary representation and multi-target preliminary representations of other view angles to obtain a multi-target representation corresponding to the view angle;

[0033] The association network is used to perform multi-target identification on multi-target detection results under different view angles at a current observation time according to corresponding multi-target representations, and perform multi-target association with the prior multi-target trajectory to realize multi-target tracking.

[0034] The application also provides a multi-unmanned aerial vehicle multi-target tracking device, which comprises:

[0035] An image data acquisition module is configured to acquire a multi-target image dataset, wherein the multi-target image dataset comprises a plurality of multi-target image subsets obtained by observing a same space from different view angles by a plurality of unmanned aerial vehicles, and each multi-target image subset comprises a plurality of multi-target images arranged in time sequence under a same view angle.

[0036] An image data grouping module is configured to group multi-target images corresponding to a same observation time in each multi-target image subset into a group of input data sets to obtain a plurality of groups of input data sets sorted according to observation times.

[0037] An image feature extraction module is configured to extract features of all multi-target images in a group of input data sets to obtain image features corresponding to each multi-target image.

[0038] A multi-target detection module is configured to perform feature enhancement on image features of a current observation time according to a prior multi-target trajectory corresponding to a view angle and a multi-target detection result of a previous observation time, and then perform multi-target detection based on the enhanced image features to obtain a multi-target detection result under the view angle.

[0039] A multi-target representation extraction module is configured to perform multi-target representation extraction according to a multi-target detection result corresponding to a view angle, and perform cross-space interaction between the extracted multi-target preliminary representation and multi-target preliminary representations of other view angles to obtain a multi-target representation corresponding to the view angle.

[0040] The target association module is configured to perform multi-target identification on the multi-target detection results at the current observation time and different perspectives according to corresponding multi-target representations, and perform multi-target association with the prior multi-target trajectories, so as to realize multi-target tracking.

[0041] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0042] A multi-target image data set is obtained, which includes multiple multi-target image subsets obtained by multiple unmanned aerial vehicles observing the same space from different perspectives, and each multi-target image subset includes multiple multi-target images arranged in time sequence at the same perspective;

[0043] The multi-target images corresponding to the same observation time in each multi-target image subset are grouped into an input data set, and multiple input data sets sorted by observation time are obtained;

[0044] Feature extraction is performed on all multi-target images in an input data set, and image features corresponding to each multi-target image are obtained;

[0045] For different perspectives, the image features at the current observation time are enhanced according to the prior multi-target trajectories corresponding to the perspective and the multi-target detection results at the previous observation time, and then multi-target detection is performed based on the enhanced image features, to obtain the multi-target detection results at the perspective;

[0046] For different perspectives, multi-target representation extraction is performed according to the multi-target detection results corresponding to the perspective, and the extracted multi-target preliminary representation is interacted with the multi-target preliminary representation of other perspectives across space, to obtain the multi-target representation corresponding to the perspective;

[0047] The multi-target detection results at the current observation time and different perspectives are identified according to corresponding multi-target representations, and are associated with the prior multi-target trajectories, to realize multi-target tracking.

[0048] A computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0049] A multi-target image data set is obtained, which includes multiple multi-target image subsets obtained by multiple unmanned aerial vehicles observing the same space from different perspectives, and each multi-target image subset includes multiple multi-target images arranged in time sequence at the same perspective;

[0050] The multi-target images corresponding to the same observation time in each of the multi-target image subsets are combined into a set of input data sets, and a plurality of sets of input data sets sorted according to observation time are obtained;

[0051] Feature extraction is performed on all multi-target images in a set of input data sets, and image features corresponding to each frame of multi-target image are obtained;

[0052] For different perspectives, the image features of the current observation time are enhanced according to the prior multi-target trajectory corresponding to the perspective and the multi-target detection result of the previous observation time, and then multi-target detection is performed based on the enhanced image features to obtain the multi-target detection result under the perspective;

[0053] For different perspectives, multi-target representation extraction is performed according to the multi-target detection result corresponding to the perspective, and the extracted multi-target preliminary representation is interacted with the multi-target preliminary representation of other perspectives to obtain the multi-target representation corresponding to the perspective;

[0054] The multi-target detection results under different perspectives at the current observation time are identified according to the corresponding multi-target representation, and are associated with the prior multi-target trajectory to realize multi-target tracking.

[0055] The above multi-UAV multi-target tracking method, device, computer equipment and storage medium, by extracting features from multi-target images obtained by different UAVs from different perspectives at the same observation time, for the image features under each perspective, first enhancing the image features of the current observation time according to the previous observation time and the prior multi-target trajectory, then performing multi-target detection to obtain each target detection frame, then realizing cross-space target identity recognition based on the target detection result and the target representation information under other perspectives, and finally associating the multi-target detection result to the prior multi-target trajectory through the target identity recognition result to realize target tracking. The method can effectively improve the multi-target tracking accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A flowchart of a multi-UAV multi-target tracking method in one embodiment;

[0057] Figure 2 A schematic diagram of a sub-target detection network structure in one embodiment;

[0058] Figure 3 A flowchart of multi-target preliminary representation extraction in a sub-target representation extraction network in one embodiment;

[0059] Figure 4 A flowchart of multi-perspective interaction in a sub-target representation extraction network in one embodiment;

[0060] Figure 5 A structural schematic diagram of a multi-vehicle multi-target tracking network in an embodiment;

[0061] Figure 6 A structural block diagram of a multi-vehicle multi-target tracking device in an embodiment;

[0062] Figure 7 An internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0063] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0064] In the prior art, in order to complete the task of locating and identifying targets from the videos captured by multiple unmanned vehicles simultaneously, the multi-vehicle multi-target tracking (MDMOT) usually adopts the strategy of associating local targets to obtain identity, however, their positioning and identification stages are heavily dependent on single-frame information, leading to positioning being very sensitive to visual information decay, and it is difficult to capture discriminative representations for target identification, therefore, they usually show unreliable performance in challenging scenarios, such as occlusion between targets and high similarity problems, as shown in Figure 1 A multi-vehicle multi-target tracking method is provided in the present application, comprising the following steps:

[0065] Step S100, acquiring a multi-target image data set, wherein the multi-target image data set comprises multiple multi-target image subsets obtained by observing the same space from different perspectives by multiple unmanned vehicles respectively, and each multi-target image subset comprises multiple frames of multi-target images arranged in time sequence in the same perspective.

[0066] Step S110, grouping the multi-target images corresponding to the same observation time in each multi-target image subset into a group of input data sets, to obtain multiple groups of input data sets sorted according to observation time.

[0067] Step S120, performing feature extraction on all multi-target images in a group of input data sets, to obtain image features corresponding to each frame of multi-target image.

[0068] Step S130, performing feature enhancement on the image features of the current observation time according to the prior multi-target trajectory corresponding to the perspective and the multi-target detection result of the previous observation time, and then performing multi-target detection based on the enhanced image features, to obtain the multi-target detection result under the perspective.

[0069] Step S140, for different views, according to the multi-target detection result corresponding to the view, the multi-target representation is extracted, and the multi-target preliminary representation extracted is interacted with the multi-target preliminary representation of other views to obtain the multi-target representation corresponding to the view.

[0070] Step S150, at the current observation time, the multi-target detection result under different views is identified according to the corresponding multi-target representation, and is associated with the prior multi-target trajectory to realize multi-target tracking.

[0071] In the embodiment, the image features at the current time are enhanced according to the prior timing experience under each view, so as to better perform multi-target detection with the help of prior features in time sequence, and further multi-target identification is performed according to the spatial features of each target under different views, and finally each target at the current observation time is associated with the target trajectory obtained in advance according to the multi-target identification result, so as to realize accurate tracking of the multi-target. The method realizes tracking of the target by associating the features of each target in time and space, so as to overcome the problem that the target positioning in the prior art depends heavily on single frame information, and the target is lack of distinguishing features, and at the same time, the problem that the small target and the cluttered background lead to easy loss of the target is solved.

[0072] In step S100, multiple unmanned aerial vehicles simultaneously observe multiple targets in a specified observation area, so as to obtain target images of the multiple targets in the specified observation area under different views (i.e. multi-target image subsets).

[0073] In the embodiment, the targets in the specified observation area can include multiple types of moving targets, for example, when the observation area is a road or an intersection, the target categories can include vehicles, bicycles, motorcycles or other engineering vehicles.

[0074] In step S110, each multi-target image subset is obtained under different views, but when processing it, the method simultaneously processes multi-target images of multiple views at a certain observation time, aiming to combine different features of each target under each view when identifying the target, that is, more image features of one target can be obtained to realize accurate identity recognition.

[0075] In the embodiment, by sequentially processing each input data set in time sequence, the identity of each target and the position of each target at each observation time are obtained, so that each target in multiple observation times is associated to obtain the trajectory of each target.

[0076] In step S120, the feature extraction network is used to extract features of each multi-target image in the group of input data sets.

[0077] In this embodiment, the feature extraction network includes multiple feature extraction sub-networks that respectively process each frame of multi-target images.

[0078] In this embodiment, each feature extraction subnetwork is a backbone network with shared parameters.

[0079] In step S130, for different viewpoints, the image features at the current observation time are enhanced based on the prior multi-target trajectories corresponding to that viewpoint and the multi-target detection results at the previous observation time. Then, multi-target detection is performed based on the enhanced image features to obtain the multi-target detection results under that viewpoint using a target detection network. The target detection network includes multiple parallel sub-target detection networks corresponding to the detection of targets in the multi-target images under different viewpoints.

[0080] In this embodiment, the temporal-oriented target localization (TTL) network uses information from previous observation times to enhance the image features at the current observation time to produce more accurate target detection results.

[0081] Furthermore, the information from previous observation times includes the detection results of each target at the corresponding viewpoint at the previous observation time, as well as the already generated target trajectories.

[0082] Furthermore, each sub-target detection network includes two stages: extracting temporal priors by category based on information from previous observation times, and then performing feature enhancement through temporal interaction.

[0083] Specifically, such as Figure 2 As shown, in each sub-target detection network: based on the multi-target detection results F from the previous observation time... t-1 and prior multi-target trajectories In the image features from the previous observation time, the prior feature blocks ROIAlign corresponding to each target are cropped. According to the target category, the average value of all prior feature blocks under each target category is calculated, and the average value is used as the temporal prior feature CTP for the target category. cls Based on the temporal prior features of each target category, feature enhancement is performed on the image features at the current observation time from the same perspective. Finally, the detection head is used to perform multi-target detection based on the feature-enhanced image features to obtain the multi-target detection results at the current observation time from that perspective.

[0084] In this embodiment, the average value of all prior feature blocks under each target category is calculated using the following formula:

[0085]

[0086] In formula (1), ROIAlign represents the image features of the previous observation time The prior feature block corresponding to each target is cropped in the middle, and the subscript cls is an integer between 1 and num c , cls represents the target category, and num c represents the number of targets. represents the time sequence prior feature of the cls-th target category, N cls and Loc cls respectively represent the number and position of all targets in this category.

[0087] In this embodiment, the feature interaction module is used to enhance the features in the image features of the current observation time in the same view according to the time sequence prior features of each target category. The module aims to relay the previous time sequence prior feature CTP to the current frame.

[0088] In this embodiment, the similarity between the original image features of the multi-target image of the current observation time and CTP cls is enhanced by using the feature extraction network.

[0089] Based on the method in the twin network, which has achieved remarkable success in single target tracking (SOT), cross-correlation operation is introduced into sub-target detection. However, unlike the use of instance-level template features for cross-correlation operation in SOT, the use of time sequence prior features in this method helps to locate more missed targets in the past, rather than just enhancing the features of the targets successfully observed in the past.

[0090] Referring to the time interaction module in Figure 2 , in this embodiment, the feature enhancement in the image features of the current observation time in the same view according to the time sequence prior features of each target category includes: after the image features of the current observation time are subjected to convolution operation, cross-correlation is performed with the time sequence prior features of each target category respectively to obtain the positioning response map corresponding to each target category The positioning response maps of all target categories are summed to obtain a similarity map M S , which is multiplied by the image features subjected to convolution operation , and the image features are enhanced by residual connection to obtain enhanced image features F t .

[0091] Specifically, in the time interaction module, the image features of the current observation time are input into a 3x3 convolution layer φ(·) to obtain , and cross-correlation is performed with the time prior features of each category to obtain the positioning response map of each category This process can be represented as:

[0092]

[0093] In formula (2), * represents a cross-correlation operation. A series of positioning response maps can be obtained by formula (2) Then, the sum is obtained, and the overall similarity map M is obtained S , which represents a region where a target is likely to exist, inferred by means of a clue of tracking a target in the past.

[0094] Then, the similarity map M S is multiplied by , and the original features are enhanced through a residual connection. Finally, a 3x3 convolution layer is used to improve the enhanced features, so as to obtain the enhanced image features F t , and the process can be represented as:

[0095]

[0096] Finally, in each target detection sub-network, the enhanced image features F t are subjected to target detection by using a detection head, and the target detection result D t is obtained, which is marked by using a target detection frame on each detected target in the image features.

[0097] In step S140, for different perspectives, multi-target representation is extracted according to the multi-target detection result corresponding to the perspective, and the extracted multi-target preliminary representation is cross-spacedly interacted with the multi-target preliminary representation of other perspectives, so as to obtain the multi-target representation corresponding to the perspective. A target representation extraction network (spatial-oriented target identification (STI)) is used, and the target representation extraction network includes a plurality of sub-target representation extraction networks corresponding to the extraction of target representations in multi-target images under each different perspective, and each sub-target representation extraction network is connected with each other sub-target representation extraction network.

[0098] In this embodiment, each sub-target representation extraction network aims to capture the identity embedding of each target from the enhanced image features. In this process, information in the surrounding background of the target itself is captured at the same time. Through spatial interaction, the information of the same target under different perspectives can be interacted. Therefore, each sub-target representation network includes two stages, extraction of target preliminary representation, i.e., target identity preliminary representation, and interaction of target cross-perspective information.

[0099] In the embodiment, after the multi-target detection result is encoded into a form processable by the characteristic extraction network, the original target detection frame is expanded according to a preset expansion size to obtain an expanded detection frame, the original target detection frame and the expanded detection frame are respectively cropped on the image feature to obtain corresponding local image features, and the local image features are fused to obtain multi-target preliminary characteristics under a certain perspective. This part is the target preliminary characteristic extraction part. Then, the multi-target preliminary characteristics under a certain perspective and the multi-target preliminary characteristics under other perspectives are cross-space interacted to obtain multi-target characteristics corresponding to the perspective. When cross-space interaction is performed, the embedding features of the same target under different perspectives are enhanced and interacted through a combination of linear layers, self-attention and mutual attention operations, and a feedforward network to obtain multi-target characteristics corresponding to the perspective.

[0100] Specifically, the process of target preliminary characteristic extraction is as shown in Figure 3 In this part, the enhanced image feature F t and the target detection result D t obtained in step S130 are simultaneously used.

[0101] Further, due to the task difference between the target detection network and the target characteristic extraction network, they cannot directly share features, that is, the enhanced image feature F t cannot be directly input into the target characteristic extraction network. Therefore, the first 3 layers of features of the enhanced image feature F t are selected for subsequent prediction, and then a 3x3 and a 1x1 convolution are used to encode into the required feature for STI.

[0102] Further, the original target detection frame is expanded according to a preset detection frame size expansion multiple. In the embodiment, two expansion multiples a1 and a2 are preset. After the original target detection frame is expanded according to the two multiples, three target detection frames are obtained on the image feature, wherein more information in the original target detection frame is target information, and the other two expanded target detection frames contain information of the target and the surrounding background. A set of multi-level local image features containing the target itself and the surrounding background are obtained by cropping on the image feature according to the three target detection frames, wherein each feature dimension is 64x3x3.

[0103] Further, when the local image features are fused, all features in the set are spliced along the channel dimension to obtain

[0104] ​To adjust the impact of background information at different levels and degrees on each objective, compression is achieved through max-pooling and average-pooling. The spatial dimension is determined, and the resulting vector is then passed through a 1d convolution l(·) and a fully connected layer δ(·), and summed. Channel-level weights are then obtained using the sigmoid function. Finally, the obtained weights are multiplied by a Hadamard product and connected via residuals to obtain... The process can be represented as follows:

[0105]

[0106] In formula (4), σ(·) represents the sigmoid function.

[0107] Next, a conversion module will be used to... Convert to the target's identity identifier The conversion module consists of a 3×3 convolution that compresses the spatial dimension and a 1×1 convolution that adjusts the number of channels.

[0108] Specifically, the process of interacting with cross-perspective information of the target is as follows: Figure 4 As shown in the diagram, two perspectives are used as examples for illustration.

[0109] In this embodiment, the embeddings e1, e2, and their sum from different viewpoints of a target are fed into a linear layer to obtain a tensor. To enhance the advantages of features in each viewpoint, M1 and M2 are multiplied by their corresponding transpose tensors, and then passed through a softmax layer to obtain a weight map for feature enhancement in each viewpoint.

[0110] Specifically, enhanced weighted graph The following formula is used to calculate:

[0111]

[0112] In formula (5), · represents the dot product operation. and This represents the i-th and j-th elements of M1 or M2.

[0113] Next, connect M1 and M2 with M respectively. s The transpose of the matrix is ​​used for matrix multiplication to find and enhance the commonalities of the shared objective across different perspectives. Similarly, a softmax layer is used to obtain a weight map across perspectives.

[0114] Specifically, a weighted graph from multiple perspectives. The following formula is used to calculate:

[0115]

[0116] In formula (6), is the value at position (i, j), denotes the M s th element of the jth column of the matrix.

[0117] The weight map W k of different views can be fused by the hyperparameter λ, using the following formula:

[0118]

[0119] Finally, the target identity representation after aggregating the advantages of different views, i.e., the final multi-target representation, can be obtained by the weight map W k of different views and the feedforward network (FFN) represented by the linear layer, which is to identify the identity of the target after detecting the position of the multi-target. The process can be represented as:

[0120]

[0121] In the target detection part in step S130, in order to provide the robustness of the template feature, the temporal prior feature CTP cls is modeled in a long-term manner during the tracking process, and the modeling is represented as:

[0122]

[0123] In formula (9), and denote the current and previous template features, and α is a hyperparameter for adjusting the weight.

[0124] In step S150, according to the multi-target representation, the multi-target position at the current observation time is associated with each target at the previous observation time, so as to update the multi-target trajectory and realize multi-target tracking.

[0125] Specifically, after obtaining the detection and identification target, firstly, the detection target is associated with the existing trajectory under the corresponding view angle. Specifically, the sum of the affinity of the trajectory and the detection target in motion and appearance is taken as the cost matrix to be input into the Hungarian algorithm for matching. Among them, the motion metric is obtained by calculating the IoU of the predicted trajectory position of the Kalman filter and the detection box. The appearance metric is the cosine distance between the identity embedding of the trajectory and the detection target. Due to the difference in the value range of the cosine distance and the IoU, we set the negative number of the cost matrix to 0. After completing the single association of a frame, we associate the targets across the level at the current time. In the cross-view matching, the appearance information measures the affinity between the targets, which is obtained by the cosine distance of the identity embedding of the targets. If the appearance similarity of the targets under different view angles is greater than the threshold 0.3, TSMMT assigns them the same ID. After completing the target association in the time domain and the spatial domain, the representation E of the tracking target is long-term modeled:

[0126] E = β × E t-1 + (1 - β) × E t (10)

[0127] In formula (10), E t-1 and E t represent the embedding of the target at the current and previous time, and β is a hyperparameter.

[0128] In another embodiment, the above-mentioned networks can also be constructed into a multi-angle multi-target tracking network, which includes the above-mentioned feature extraction network, target detection network, target representation extraction network and association network. The feature extraction network is used to extract features of all multi-target images in a group of input data sets to obtain image features corresponding to each frame of multi-target image. The target detection network is used to perform feature enhancement on the image features of the current observation time according to the prior multi-target trajectory corresponding to the view angle and the multi-target detection result at the previous observation time, and then perform multi-target detection based on the enhanced image features to obtain the multi-target detection result under the view angle. The target representation extraction network is used to extract multi-target preliminary representation for different view angles according to the multi-target detection result corresponding to the view angle, and perform cross-space interaction between the extracted multi-target preliminary representation and the multi-target preliminary representation of other view angles to obtain the multi-target representation corresponding to the view angle. The association network is used to perform multi-target identification on the multi-target detection result under different view angles at the current observation time according to the corresponding multi-target representation, and perform multi-target association with the prior multi-target trajectory to realize multi-target tracking.

[0129] In this embodiment, the input data sets are input into the multi-angle multi-target tracking network in turn according to the arrangement order, so that multi-target tracking can be realized.

[0130] As Figure 5As shown, the application also provides a multi-angle multi-target tracking network structure diagram taking two view angles as an example.

[0131] In this embodiment, when training the multi-angle multi-target tracking network, since it contains several sub-tasks, multiple loss functions are used to optimize the network.

[0132] Specifically, L1 loss is used as the regression loss of the region detection network and the final output in the training process, that is, and In addition, a new loss function is introduced for the target detection network. The ground truth of the similarity map M s is defined as a combination of multiple Gaussian distributions. For each target, its supervision signal is a Gaussian-like mask, which is represented as:

[0133]

[0134] In formula (11), represents the center position of the i-th target, σ i represents the standard deviation of the target scale adaptation. After generating a set of ground truth masks t = {t1,..., t n}, the sum along the channel dimension is obtained M s , and the supervision signal T is obtained.

[0135] Then, the Logistic-MSE loss is used to train the target detection network, and the loss function is represented as:

[0136]

[0137] In formula (12), and T xy represent the values of M s and T at the corresponding positions.

[0138] Then the loss function of the target detection is represented as:

[0139] L loc = L reg + L cls + γL ttl (13)

[0140] In formula (13), γ is a hyperparameter L ttl adjusting the weight of L ttl .

[0141] Specifically, for the loss function of the multi-target representation extraction network, it is assumed that there are K and V samples in a pair of training images. For each training sample, the improved non-parametric softmax and cross-entropy are used to optimize the multi-target representation extraction network:

[0142]

[0143] In formula (14), v, k + and k - are identity embeddings of training samples and corresponding positive and negative samples, respectively. At the same time, in order to constrain the cosine similarity between targets, an auxiliary loss L aux based on cosine distance is introduced for STI, denoted as:

[0144]

[0145] In formula (15), n is 1 when two samples are positive, and 0 otherwise. Combining L emb and L aux , the loss function L id of the target identity recognition task is denoted as:

[0146] L id = L emb + ηL aux (16)

[0147] In formula (16), η is a weight parameter.

[0148] In this embodiment, the total loss function of the multi-angle multi-target tracking network can be denoted as:

[0149]

[0150] In formula (17), ω1, ω2 are learnable parameters.

[0151] In this paper, the effectiveness of the method is also proved by comparing the experimental simulation of the method with that of a plurality of existing methods. The results of the experimental simulation are shown in Table 1.

[0152] Table 1 Experimental simulation results of the method and a plurality of existing methods

[0153]

[0154] As can be seen from Table 1, the performance of the methods without using appearance features is generally better than that of the methods using appearance features (marked in gray). Among the methods without using appearance features, CBM and ABM achieve the best performance in MOTA and IDF1, respectively. Compared with CBM, the proposed methods outperform 1.62% (56.54%-54.92%), 3.89% (52.12%-48.23%), and 2.76% (54.34%-51.58%) in MOTA, respectively. Meanwhile, TSMMT outperforms ABM by 2.08% (71.75%-69.67%), 1.77% (68.58%-66.81%), and 2.06% (70.30%-68.24%) in IDF1, respectively. Among the methods without using MIA-Net, CB achieves the best performance. Compared with CB, the proposed methods outperform 2.41%-3.70% and 3.53%-3.65% in MOTA and IDF1, respectively. The above comparisons prove that the proposed methods outperform the existing methods without using appearance features with a significant advantage. Due to the high similarity between targets in the UAV view and the limited available information, it is difficult to extract the appearance features of the targets. Among the previous methods using appearance features, the model using QDTrack achieves the best performance. Compared with FQ and CQ using QDTrack, TSMMT outperforms 5.52%-10.90% and 5.47%-11.12% in IDF1, respectively. DeepSORT is successful in MOT applications in various scenarios. Compared with FD, TSMMT outperforms 11.27% (71.75%-60.48%), 16.14% (68.58%-52.44%), and 13.84% (70.30%-56.46%) in IDF1, respectively. Compared with ABS and CBS equipped with SBS-50 feature extraction network, the proposed tracker outperforms 13.89%-15.86% and 9.72-14.71% in IDF1. Meanwhile, TSMMT leads ABS and CBS by 23.52% (41.71%-18.19%) and 23.41% (41.71%-18.30%) in MDA. The significant advantage of TSMMT in IDF1 and MDA proves that TSMMT effectively extracts more discriminative features compared with other algorithms.

[0155] To further verify the robustness of the model, TSMMT was evaluated under different weather conditions and compared with the representative appearance-free methods ABM and CBM, as shown in Table 2.

[0156] Table 2 Performance comparison with representative appearance-free methods under different weather conditions

[0157]

[0158] As can be seen from Table 2, under different environments, the performance of the tracker proposed in the present method is better than that of CBM on MOTA by 0.77%~5.61%, on IDF1 by 2.07%~6.05%, and on MDA by 1.76%~7.14%. In terms of MOTA and IDF1, TSMMT is higher than ABM by 3.05%~7.52% and 1.11%~2.70% respectively. The significant performance advantage of TSMMT under different weather conditions proves its superior robustness to external environment.

[0159] Due to the high similarity between targets in the view of the unmanned aerial vehicle and less available information, it is difficult to extract the ReID features of the target. In the existing appearance-based method, DeepSORT and SBS-50 only identify the target through the slice of the target. QDTrack extracts more discriminative features through contrastive learning, thereby achieving the best performance in the contrast algorithm on the MDMT dataset. Compared with FQ and CQ using QDTrack, TSMMT exceeds 5.52%~10.90% and 5.47%~11.12% respectively on the IDF1 index. DeepSORT is successful in the application of MOT in multiple scenes. Compared with FD, TSMMT exceeds 11.27% (71.75%-60.48%), 16.14% (68.58%-52.44%) and 13.84% (70.30%-56.46%) respectively on IDF1. Compared with ABS and CBS equipped with SBS-50 ReID network, the proposed tracker exceeds 13.89%~15.86% and 9.72~14.71% respectively on IDF1. At the same time, TSMMT leads ABS and CBS by 23.52% (41.71%-18.19%) and 23.41% (41.71%-18.30%) respectively on the MDA index. The significant advantage of TSMMT on IDF1 and MDA proves that TSMMT has obtained more discriminative features compared with other algorithms.

[0160] Further, the performance of the tracker in the present method is compared with representative appearance-based models ABS and CBS in Table 3. It is clear that TSMMT outperforms CBS by 2.11-8.96% in MOTA, 12.01%-16.73% in IDF1, and 20.27%-40.81% in MDA when facing variable weather conditions. Compared with ABS by ABS, TSMMT shows superior performance by 8.98%-15.14%, 12.06%-18.07%, and 19.84%-41.21% in these key indicators. The significant advantages in these key indicators emphasize the effectiveness of TSMMT, especially the accuracy of the tracker proposed in the present method in identifying targets.

[0161] Table 3 Performance comparison with representative appearance-based methods under different weather conditions

[0162]

[0163]

[0164] In the above multi-UAV multi-target tracking method, a new tracker TSMMT (i.e., UAV multi-target tracking network) designed for the multi-UAV multi-target tracking task. In the present method, the impact of ignoring spatio-temporal cues on the MDMOT network is first analyzed. In order to overcome this limitation, temporal-oriented target localization (TTL) and spatial-oriented target identification (STI) are proposed to interact spatio-temporal features in the tracking process. A large number of experimental results prove the accuracy and robustness of the network.

[0165] It should be understood that, although Figure 1 The steps in the flowchart of the method can be executed in the order indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the method can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or sub-steps or stages of other steps.

[0166] In one embodiment, as Figure 6As shown, a multi-UAV multi-target tracking apparatus is provided, comprising: an image data acquisition module 200, an image data grouping module 210, an image feature extraction module 220, a multi-target detection module 230, a multi-target representation extraction module 240, and a target association module 250, wherein:

[0167] The image data acquisition module 200 is configured to acquire a multi-target image data set, wherein the multi-target image data set comprises a plurality of multi-target image subsets obtained by observing the same space from different perspectives by a plurality of UAVs, and each of the multi-target image subsets comprises a plurality of multi-target images arranged in time sequence at the same perspective;

[0168] The image data grouping module 210 is configured to group the multi-target images corresponding to the same observation time in each of the multi-target image subsets into a group of input data sets, and obtain a plurality of groups of input data sets sorted according to observation time;

[0169] The image feature extraction module 220 is configured to perform feature extraction on all multi-target images in a group of input data sets, and obtain image features corresponding to each frame of multi-target image;

[0170] The multi-target detection module 230 is configured to, for different perspectives, perform feature enhancement on the image features of the current observation time according to the prior multi-target trajectory corresponding to the perspective and the multi-target detection result at the previous observation time, and perform multi-target detection based on the image features after feature enhancement, to obtain the multi-target detection result at the perspective;

[0171] The multi-target representation extraction module 240 is configured to, for different perspectives, perform multi-target representation extraction according to the multi-target detection result corresponding to the perspective, and perform cross-space interaction between the extracted multi-target preliminary representation and the multi-target preliminary representation of other perspectives, to obtain the multi-target representation corresponding to the perspective;

[0172] The target association module 250 is configured to perform multi-target identification on the multi-target detection results at different perspectives at the current observation time according to the corresponding multi-target representation, and perform multi-target association with the prior multi-target trajectory, to realize multi-target tracking.

[0173] The specific limitations of the multi-UAV multi-target tracking apparatus can be referred to the limitations of the multi-UAV multi-target tracking method in the foregoing, which will not be repeated here. Each module in the above multi-UAV multi-target tracking apparatus can be realized by software, hardware, and combinations thereof, in whole or in part. Each of the above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each of the above modules.

[0174] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a multi-UAV multi-target tracking method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0175] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0176] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0177] A multi-target image dataset is obtained, which includes multiple subsets of multi-target images obtained by multiple UAVs observing the same space from different perspectives. Each subset of multi-target images includes multiple frames of multi-target images arranged in chronological order from the same perspective.

[0178] The multi-target images corresponding to the same observation time in each of the multi-target image subsets are combined into a set of input datasets, resulting in multiple sets of input datasets sorted according to the observation time;

[0179] Feature extraction is performed on all multi-target images in a set of input datasets to obtain the image features of each frame of multi-target images;

[0180] For different perspectives, feature enhancement is performed on the image features at the current observation time based on the prior multi-target trajectory corresponding to that perspective and the multi-target detection results at the previous observation time. Then, multi-target detection is performed based on the feature-enhanced image features to obtain the multi-target detection results under that perspective.

[0181] According to the multi-target detection result corresponding to the view angle, multi-target representation extraction is performed, and the extracted multi-target preliminary representation is cross-space interacted with the multi-target preliminary representation of other view angles to obtain the multi-target representation corresponding to the view angle.

[0182] According to the multi-target detection result corresponding to the view angle, multi-target representation extraction is performed, and the extracted multi-target preliminary representation is cross-space interacted with the multi-target preliminary representation of other view angles to obtain the multi-target representation corresponding to the view angle.

[0183] In one embodiment, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium. The computer program is executed by a processor to implement the following steps:

[0184] A multi-target image data set is obtained, and the multi-target image data set includes a plurality of multi-target image subsets obtained by observing the same space from different view angles by a plurality of unmanned aerial vehicles. Each multi-target image subset includes a plurality of multi-target images arranged in time sequence under the same view angle.

[0185] The multi-target images corresponding to the same observation time in each multi-target image subset are combined to form a group of input data sets, and a plurality of groups of input data sets sorted by observation time are obtained.

[0186] Feature extraction is performed on all multi-target images in a group of input data sets to obtain image features corresponding to each frame of multi-target image.

[0187] According to the multi-target detection result corresponding to the view angle, multi-target representation extraction is performed, and the extracted multi-target preliminary representation is cross-space interacted with the multi-target preliminary representation of other view angles to obtain the multi-target representation corresponding to the view angle.

[0188] According to the multi-target detection result corresponding to the view angle, multi-target representation extraction is performed, and the extracted multi-target preliminary representation is cross-space interacted with the multi-target preliminary representation of other view angles to obtain the multi-target representation corresponding to the view angle.

[0189] According to the multi-target detection result corresponding to the view angle, multi-target representation extraction is performed, and the extracted multi-target preliminary representation is cross-space interacted with the multi-target preliminary representation of other view angles to obtain the multi-target representation corresponding to the view angle.

[0190] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0191] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0192] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for tracking multiple unmanned aerial vehicles (UAVs) and multiple targets, characterized in that, The method includes: A multi-target image dataset is obtained, which includes multiple subsets of multi-target images obtained by multiple UAVs observing the same space from different perspectives. Each subset of multi-target images includes multiple frames of multi-target images arranged in chronological order from the same perspective. The multi-target images corresponding to the same observation time in each of the multi-target image subsets are combined into a set of input datasets, resulting in multiple sets of input datasets sorted according to the observation time; Feature extraction is performed on all multi-target images in a set of input datasets to obtain the image features of each frame of multi-target images; For different perspectives, feature enhancement is performed on the image features at the current observation time based on the prior multi-target trajectory corresponding to that perspective and the multi-target detection results at the previous observation time. Then, multi-target detection is performed based on the feature-enhanced image features to obtain the multi-target detection results at that perspective. In this process, a target detection network including multiple parallel sub-target detection networks corresponding to the detection of targets in multi-target images at different perspectives is used for multi-target detection. In each of the sub-target detection networks: based on the multi-target detection results at the previous observation time and the prior multi-target trajectory, prior feature blocks corresponding to each target are cropped from the image features at the previous observation time. According to the target category, the average value of all prior feature blocks under each target category is calculated, and the average value is used as the temporal prior feature for the target category. Feature enhancement is performed on the image features at the current observation time at the same perspective based on the temporal prior features of each target category. The detection head is then used to perform multi-target detection based on the feature-enhanced image features to obtain the multi-target detection results at the current observation time at that perspective. For different perspectives, multi-target representations are extracted based on the multi-target detection results corresponding to that perspective. The extracted preliminary multi-target representations are then interacted across space with the preliminary multi-target representations of other perspectives to obtain the multi-target representation corresponding to that perspective. The multi-target detection results from different perspectives at the current observation time are used to identify multiple targets based on their corresponding multi-target representations, and then associated with the prior multi-target trajectories to achieve multi-target tracking.

2. The multi-UAV multi-target tracking method according to claim 1, characterized in that, The feature enhancement of image features at the current observation time from the same viewpoint based on the temporal prior features of each target category includes: The image features at the current observation time are convolved and then cross-correlated with the temporal prior features of each target category to obtain the localization response map for each target category. The similarity map is obtained by summing the localization response maps of all target categories, multiplying it with the image features after convolution, and then performing feature enhancement on the image features through residual connections.

3. The multi-UAV multi-target tracking method according to claim 2, characterized in that, For different perspectives, multi-target representations are extracted based on the multi-target detection results corresponding to that perspective. The extracted preliminary multi-target representations are then interacted across space with the preliminary multi-target representations of other perspectives to obtain the multi-target representation corresponding to that perspective. Among them, the extraction of multi-target representations for different perspectives is carried out using a target representation extraction network. The target representation extraction network includes multiple sub-target representation extraction networks that extract target representations from multi-target images from different perspectives. Each of the sub-target representation extraction networks is interconnected with each of the other sub-representation extraction networks.

4. The multi-UAV multi-target tracking method according to claim 3, characterized in that, In each of the sub-target representation extraction networks: After encoding the multi-target detection results into a form that the representation extraction network can process, the original target detection box is enlarged according to a preset magnification size to obtain an enlarged detection box; Local image features are obtained by cropping the image features based on the original target detection box and the enlarged detection box respectively, and the local image features are fused to obtain a preliminary representation of multiple targets from a certain perspective. The preliminary multi-target representation from one perspective is interacted with the preliminary multi-target representation from other perspectives across space to obtain the multi-target representation corresponding to that perspective. In the process of cross-space interaction, the embedded features of the same target under different perspectives are enhanced and interacted through a combination of linear layers, self-attention and mutual attention operations, and feedforward networks to obtain the multi-target representation corresponding to the perspective.

5. The method according to any one of claims 1-4, characterized in that, The method includes: Each set of input datasets is sequentially input into a multi-angle multi-target tracking network in the order of arrangement to achieve multi-target tracking. The multi-angle multi-target tracking network includes a feature extraction network, a target detection network, a target representation extraction network, and an association network. The feature extraction network is used to extract features from all multi-target images in a set of input datasets to obtain the image features of each frame of multi-target images; The target detection network is used to perform feature enhancement on the image features at the current observation time based on the prior multi-target trajectory corresponding to the viewpoint and the multi-target detection results at the previous observation time for different viewpoints. Then, multi-target detection is performed based on the feature-enhanced image features to obtain the multi-target detection results under that viewpoint. The target representation extraction network is used to extract multi-target representations from different perspectives based on the multi-target detection results corresponding to each perspective. The extracted preliminary multi-target representations are then interacted across space with the preliminary multi-target representations from other perspectives to obtain the multi-target representations corresponding to that perspective. Using the aforementioned association network, the multi-target detection results from different perspectives at the current observation time are used to identify multiple targets based on their corresponding multi-target representations, and then associated with the prior multi-target trajectories to achieve multi-target tracking.

6. A multi-UAV multi-target tracking device, characterized in that, The device includes: The image data acquisition module is used to acquire a multi-target image dataset, which includes multiple subsets of multi-target images obtained by multiple UAVs from different perspectives of the same space. Each subset of multi-target images includes multiple frames of multi-target images arranged in chronological order from the same perspective. The image data grouping module is used to group the multi-target images corresponding to the same observation time in each of the multi-target image subsets into a set of input datasets, thereby obtaining multiple sets of input datasets sorted according to the observation time; The image feature extraction module is used to extract features from all multi-target images in a set of input datasets to obtain the image features of each frame of multi-target images. A multi-target detection module is used to perform feature enhancement on the image features at the current observation time based on the prior multi-target trajectory corresponding to the viewpoint and the multi-target detection result at the previous observation time for different viewpoints. Then, multi-target detection is performed based on the feature-enhanced image features to obtain the multi-target detection result at the current observation time. The module employs a target detection network that includes multiple parallel sub-target detection networks corresponding to the detection of targets in multi-target images at different viewpoints. In each sub-target detection network: based on the multi-target detection result at the previous observation time and the prior multi-target trajectory, prior feature blocks corresponding to each target are cropped from the image features at the previous observation time. According to the target category, the average value of all prior feature blocks under each target category is calculated and used as the temporal prior feature for the target category. Feature enhancement is performed on the image features at the current observation time under the same viewpoint based on the temporal prior features of each target category. The detection head is then used to perform multi-target detection based on the feature-enhanced image features to obtain the multi-target detection result at the current observation time under the current viewpoint. The multi-target representation extraction module is used to extract multi-target representations based on the multi-target detection results corresponding to different perspectives, and to perform cross-space interaction between the extracted preliminary multi-target representations and the preliminary multi-target representations of other perspectives to obtain the multi-target representation corresponding to the perspective. The target association module is used to identify multiple targets based on the corresponding multi-target representations of the multi-target detection results from different perspectives at the current observation time, and to associate them with the prior multi-target trajectories to achieve multi-target tracking.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images