Target detection and tracking method for autonomous driving scenarios based on graph neural network

Abstract

A method for target detection and tracking in autonomous driving scenarios based on graph neural networks, comprising obtaining a feature map of the previous frame image through a first backbone network and a feature map of the current frame image through a second backbone network; establishing a pedestrian target detection network based on the target center point using a cross-attention network, obtaining feature maps for Re-ID tasks and feature maps for target detection tasks through the cross-attention network, and outputting target detection results; establishing a target embedding feature extraction algorithm based on graph neural networks, performing multiple neural network and attention aggregation on the target map through the graph neural network, obtaining the target's Re-ID features for target tracking, so that the target simultaneously outputs the detection results and embedding features of the pedestrian target in one forward propagation of the network, reducing computational redundancy and making the multi-target tracking process more concise.

Description

Technical Field

[0001] The present invention relates to the field of target detection and tracking technology, and in particular to a method for target detection and tracking in autonomous driving scenarios based on graph neural networks. Background Art

[0002] Multi-target tracking aims to analyze videos to identify and track multiple targets belonging to one or more categories. Compared with target detection tasks, multi-target tracking tasks require not only detecting the required targets in each frame of the video, but also associating the same targets in the previous and next frames, and processing newly appearing and disappearing targets. As one of the basic and key technologies in the field of computer vision, multi-target tracking technology has been widely used in all aspects of social life, and its application scope mainly includes autonomous driving, intelligent video analysis, intelligent transportation, human-computer interaction, etc. In the scenarios of the above fields, pedestrian targets are often the largest and most valuable tracking targets in the image. A single pedestrian target occupies a smaller proportion in the image, and is more difficult to detect and track.

[0003] From a practical application perspective, my country, as the world's most populous country, has always had the largest number of pedestrians and the most valuable targets in most scenarios. However, compared to other objects, pedestrians have a higher degree of autonomy and flexibility, and their movement trajectories and intentions are often difficult to determine. Therefore, in autonomous driving scenarios, detecting and tracking pedestrians, understanding and analyzing their movement trajectories, and thus issuing warnings of abnormal pedestrian behavior or accurately predicting the vehicle's driving status can not only protect pedestrian safety, but also ensure traffic safety and keep roads clear. This is of great significance to pedestrians themselves and road traffic safety.

[0004] Although domestic and foreign researchers have made many improvements to the joint technology of pedestrian target detection and tracking under autonomous driving and have achieved many research results, there are still some defects due to many factors. At present, most pedestrian tracking technologies under autonomous driving are based on the pedestrian multi-target tracking algorithm of the detection-tracking paradigm. This algorithm usually uses three models to solve the target detection task, target feature extraction task and target association task respectively. However, on the one hand, this algorithm only uses the apparent features or spatial information of the target, and the target information used is relatively single; on the other hand, it does not consider the impact of the spatial relationship between targets on target association. Therefore, the existing tracking algorithm does not comprehensively consider the target's apparent information and spatial information.

[0005] Moreover, due to the diversity of the driving environment, pedestrians may be obstructed during movement, and factors such as lighting and shadows in the night environment will blur the outline of pedestrian targets, making it difficult to distinguish between the target and the background, and unable to accurately detect and track them. Secondly, pedestrians themselves have characteristics such as diverse postures and different clothing, which further deepens the difficulty of detecting and tracking pedestrian targets. These all pose great challenges to the continuity of pedestrian target detection and tracking.

[0006] In summary, in autonomous driving scenarios, developing an accurate and efficient multi-target pedestrian tracking framework, taking into account the characteristics of pedestrian targets in different environments and comprehensively considering both detection and tracking tasks, is currently a challenge. Furthermore, in autonomous driving scenarios, in addition to the requirements for tracking accuracy, multi-target pedestrian tracking also places higher demands on real-time performance. Therefore, a method that comprehensively considers both target detection and multi-target tracking is urgently needed. Summary of the Invention

[0007] The purpose of the present invention is to address the deficiencies of the existing technology and provide a method for target detection and tracking in autonomous driving scenarios based on graph neural networks, which models the relationship between targets of variable sizes in spatial and temporal domains and can greatly enrich the identification features used to detect associations with targets.

[0008] The technical solution of the present invention is: a method for detecting and tracking targets in autonomous driving scenarios based on a graph neural network, characterized by comprising the following steps:

[0009] S1. The previous frame image I in the video t-1 As the input of the first backbone network, the feature map F of the previous frame image is obtained through the first backbone network t-1 , and from the feature map F t-1 Extract the central feature vector C of the pedestrian target t-1 ;

[0010] At the same time, the current frame image I in the video t As the input of the second backbone network, the feature map F of the current frame image is obtained through the second backbone network t ;

[0011] S2. Using the cross attention network, a pedestrian target detection network based on the target center point is established, and the feature map F of the current frame image is converted into t As the input of the cross attention network, the feature map F for the Re-ID task is obtained through the cross attention network. T1 And the feature map F for the target detection task T2 , and by the feature map F T2Perform convolution and output the target detection result, which is used as the previous frame image of the next calculation cycle;

[0012] S3. Establish a target embedding feature extraction algorithm based on graph neural network, and transform the central feature vector C of the previous frame image t-1 And the feature map F of the current frame image T1 As input to graph neural networks;

[0013] The graph neural network transforms the central feature vector C t-1 Build the target graph G as the target node t-1 , and use the feature map F T1 The feature information in the target node is generated to construct the target graph G t ;

[0014] Through the graph neural network to the target graph G t-1 and G t Perform multiple neural networks and attention aggregation to obtain the target's Re-ID features for target tracking.

[0015] Furthermore, in S1, the backbone network adopts an enhanced version of the deep layer aggregation network DLA-34.

[0016] Furthermore, in S2, the feature map F is obtained through the cross attention network T1 and feature map F T2 The specific steps are as follows:

[0017] 1) The feature map of the current frame image output by the backbone network is expressed as Use the cross attention network to pass it through the maximum pooling layer and the average pooling layer to obtain statistical information

[0018] 2) Statistical information Through different convolutional layers, the three-dimensional feature map T1 for the Re-ID task and the three-dimensional feature map T2 for the detection task are generated respectively, and the intermediate feature maps T1 and T2 are reshaped into Where N'=H'×W', H' represents the height of the two-dimensional feature map, W' represents the width of the two-dimensional feature map, M1 represents the two-dimensional feature map used for the Re-ID task, and M2 represents the two-dimensional feature map used for the object detection task;

[0019] 3) Perform matrix multiplication on the transpose of the two-dimensional feature maps M1 and M2, and generate the autocorrelation weight map through the softmax layer The calculation formula is as follows:

[0020]

[0021] Where, represents the relationship between the i-th and j-th channels, represents the transpose of M1 or M2 in the i-th channel, and represents the transpose of M1 or M2 in the j-th channel;

[0022] 4) Perform rectangular multiplication between the transposes of the two-dimensional feature maps M1 and M2, and generate a cross-relationship weight map through the softmax layer The calculation formula is as follows:

[0023]

[0024] In the formula, 1 represents the Re-ID task, 2 represents the detection task, represents the impact of the Re-ID task on the detection task in the i-th channel, or the impact of the detection task on the Re-ID task in the j-th channel, represents the impact of the Re-ID task on the detection task in the i-th channel, and represents the impact of the detection task on the Re-ID task in the j-th channel;

[0025] 5) By fusing the autocorrelation weight and the cross-correlation weight through the trainable parameter λ, we get The calculation formula is as follows:

[0026] W 1 / 2 =λ×W S1 / S2 +(1-λ)×W M1 / M2

[0027] Where W S1 / S2 Represents the autocorrelation weight of the Re-ID task or detection task, W M1 / M2 Represents the cross-relationship weight of the Re-ID task or detection task, W 1 / 2 Represents the weight mapping after the fusion of Re-ID task or detection task;

[0028] 6) The cross attention network converts the original feature map F t Rearrange to The shape of N = H × W, where N = H × W, and then perform matrix multiplication between the rearranged features and the weight map fused in 5) to obtain enhanced features for the Re-ID task and the detection task, and the enhanced features are combined with the original feature map F through residual attention. t Fusion, and finally form the feature map F for the Re-ID task T1 And the feature map F of the detection task T2 .

[0029] Furthermore, in S2, the loss function of the pedestrian target detection network is designed as follows:

[0030] 1) Calculation of target center point category loss

[0031]

[0032] Where, L cls represents the target center point category loss, represents the target center heat map predicted by the network, Y represents the probability that the point on the real heat map belongs to the target center point, and α represents the balance factor, usually α = 0.5;

[0033] 2) Calculation of target center point position offset loss

[0034]

[0035] Where, L off represents the target center position offset loss, G O represents the true target center offset, Indicates the predicted coordinate offset;

[0036] 3) Target area size loss calculation

[0037]

[0038] Where, L size represents the target area size loss, Represents the predicted length and width of the target, G S Indicates the actual length and width of the target;

[0039] 4) Loss calculation of pedestrian target detection network

[0040] L det =L cls +λ off L off +λ size L size

[0041] Where λ off and λ size Usually, they are set to 1 and 0.1 respectively.

[0042] Furthermore, in S3, the feature map F of the current frame image is T1 Expressed as Through the feature map The feature information of each position in generates W×H target nodes and constructs the target graph G t , where W represents the width of the feature map and H represents the height of the feature map;

[0043] Through the graph neural network to the target graph G t-1 and G t Perform multiple neural networks and attention aggregation, update the node information between the two target graphs, and output the updated pedestrian target feature vector of the previous frame image And the updated feature map of the current frame image Then, the Re-ID branch is used to extract the feature map Extract the target Re-ID features Among them, x represents the horizontal coordinate of the center point of the target, y represents the vertical coordinate of the center point of the target, and C represents the number of categories.

[0044] Furthermore, the Re-ID branch is a convolutional layer containing 128 kernels, which is used to extract feature maps. Re-ID features at each position in , generating feature maps Then take (x, y) as the center from the feature map Extract the Re-ID features of the target.

[0045] Furthermore, the loss function of the target embedding feature extraction algorithm is designed as follows:

[0046] Through each true target box in the image Get the center of the target on the heat map Extract the Re-ID feature vector of pedestrian targets And use the fully connected layer and softmax function to map it to the class distribution vector P = {p(k), k∈[1,K]}, and represent the one-hot encoding of the label as L i (k),

[0047] Re-ID loss calculation:

[0048]

[0049] Where, L identify represents the Re-ID loss, k represents the number of all target IDs in the training set;

[0050] Overall loss calculation:

[0051]

[0052] Where w1 is the learnable parameter for the balanced object detection task, and w2 is the learnable parameter for the balanced Re-ID task.

[0053] The beneficial effects of adopting the above technical solution: This method addresses the problem that the target features extracted from pedestrian targets are not highly discriminative, resulting in frequent switching of pedestrian IDs, and performance degradation due to excessive competition between detection tasks and Re-ID tasks. Graph neural networks can be used to model the relationship between targets of variable sizes in spatial and temporal domains, extract more discriminative pedestrian identification features, and design a cross-attention network module to separate the detection task and the Re-ID task, thereby greatly improving the speed and accuracy of pedestrian multi-target tracking, so that the target can simultaneously output the detection results and embedded features of the pedestrian target in one forward propagation of the network, reducing computational redundancy and making the multi-target tracking process more concise.

[0054] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of the overall network structure of this method;

[0056] Figure 2 Schematic diagram of the cross-attention network structure;

[0057] Figure 3 Schematic diagram of the graph neural network structure;

[0058] Figure 4 The effect of the cross-attention network on the target detection task;

[0059] Figure 5 Provides a visual display of the tracking effect of the test machine. DETAILED DESCRIPTION

[0060] See also Figure 1 , a method for target detection and tracking in autonomous driving scenarios based on graph neural networks. The specific steps are as follows:

[0061] S1. The previous frame image I in the video t-1 As the input of the first backbone network, the feature map F of the previous frame image is obtained through the first backbone network t-1 , and from the feature map F t-1 Extract the central feature vector C of the pedestrian target t-1 ;

[0062] At the same time, the current frame image I in the video t As the input of the second backbone network, the feature map F of the current frame image is obtained through the second backbone network t ;

[0063] When this method is in the first calculation cycle, the two frames of images both use the first frame of image in the video, and the first backbone network and the second backbone network both use the enhanced version of the deep aggregation network DLA-34, and parameters can be shared between the two backbone networks.

[0064] S2. Using the cross attention network, a pedestrian target detection network based on the target center point is established. The cross attention network is as follows Figure 2 As shown in the figure, the cross-attention network can learn the commonalities and characteristics between the features of the target detection task and the Re-ID task. For commonality learning, the shared information between the two tasks can be learned through the designed cross-relation mechanism; for feature learning, the cross-attention network can learn the self-relationship that reflects the correlation between different feature channels to enhance the feature representation of each task;

[0065] The feature map F of the current frame image t As the input of the cross attention network, the feature map F for the Re-ID task is obtained through the cross attention network. T1 And the feature map F for the target detection task T2 , and by the feature map F T2 Perform convolution and output the target detection result, which is used as the previous frame image of the next calculation cycle.

[0066] Obtain feature map F through cross-attention network T1 and feature map F T2 The specific steps are as follows:

[0067] 1) The feature map of the current frame image output by the backbone network is expressed as Use the cross attention network to pass it through the maximum pooling layer and the average pooling layer to obtain statistical information

[0068] 2) Statistical information Through different convolutional layers, the three-dimensional feature map T1 for the Re-ID task and the three-dimensional feature map T2 for the detection task are generated respectively, and the intermediate feature maps T1 and T2 are reshaped into Where N'=H'×W', H' represents the height of the two-dimensional feature map, W' represents the width of the two-dimensional feature map, M1 represents the two-dimensional feature map used for the Re-ID task, and M2 represents the two-dimensional feature map used for the object detection task;

[0069] 3) Perform matrix multiplication on the transpose of the two-dimensional feature maps M1 and M2, and generate the autocorrelation weight map through the softmax layer The calculation formula is as follows:

[0070]

[0071] Where, represents the relationship between the i-th and j-th channels, represents the transpose of M1 or M2 in the i-th channel, and represents the transpose of M1 or M2 in the j-th channel;

[0072] 4) Perform rectangular multiplication between the transposes of the two-dimensional feature maps M1 and M2, and generate a cross-relationship weight map through the softmax layer The calculation formula is as follows:

[0073]

[0074] In the formula, 1 represents the Re-ID task, 2 represents the detection task, represents the impact of the Re-ID task on the detection task in the i-th channel, or the impact of the detection task on the Re-ID task in the j-th channel, represents the impact of the Re-ID task on the detection task in the i-th channel, and represents the impact of the detection task on the Re-ID task in the j-th channel;

[0075] 5) By fusing the autocorrelation weight and the cross-correlation weight through the trainable parameter λ, we get The calculation formula is as follows:

[0076] W 1 / 2 =λ×W S1 / S2 +(1-λ)×W M1 / M2

[0077] Where W S1 / S2 Represents the autocorrelation weight of the Re-ID task or detection task, W M1 / M2 Represents the cross-relationship weight of the Re-ID task or detection task, W 1 / 2 Represents the weight mapping after the fusion of Re-ID task or detection task;

[0078] 6) The cross attention network converts the original feature map F t Rearrange to The shape of N = H × W, where N = H × W, and then perform matrix multiplication between the rearranged features and the weight map fused in 5) to obtain enhanced feature representations for Re-ID and detection tasks. To prevent the loss of initial information, the network draws on the residual idea of ​​ResNet and combines the enhanced features with the original feature map F through residual attention. t Fusion, and finally form the feature map F for the Re-ID task T1 And the feature map F of the detection task T2 .

[0079] S3. Establish a target embedding feature extraction algorithm based on graph neural network. Although the existing end-to-end multi-target tracking methods have achieved certain results, the previous methods are usually aimed at pedestrian targets in a single frame image. The extracted features are independent of each other and ignore the relationship between objects. However, this method believes that this object relationship is very useful for associating targets in the previous and next frames. For example, if the similarity scores of two pedestrian targets across frames increase with high confidence (that is, the two objects are likely to have the same identity), the similarity scores between these two objects and other objects should be suppressed to avoid confusion in target association. Therefore, in order to achieve better performance of the joint multi-target tracking framework, this method utilizes object relationships by designing a graph neural network, such as Figure 3 As shown in the figure, the graph neural network of this method uses the features that take object relationships into consideration for the task of target association, which makes the features extracted for each object no longer isolated, but can be adjusted by the features of its related objects in the spatial and temporal domains.

[0080] The specific method is as follows:

[0081] The central feature vector C of the previous frame image t-1 And the feature map F of the current frame image T1 As input to graph neural networks;

[0082] The graph neural network transforms the central feature vector C t-1 Build the target graph G as the target node t-1 ;

[0083] For the feature map F of the current frame image T1 Since the location information of the target in the current frame image cannot be directly obtained, this method considers that the feature map F of the current frame image T1 The feature information of each position in the image may be a potential target area. Therefore, this method converts the feature map F of the current frame image into T1 Expressed as Through the feature map The feature information of each position in generates W×H target nodes and constructs the target graph G t , where W represents the width of the feature map and H represents the height of the feature map;

[0084] Then the target graph G is trained through the graph neural network t-1 and G t Perform multiple neural networks and attention aggregation, update the node information between the two target graphs, and output the updated pedestrian target feature vector of the previous frame image And the updated feature map of the current frame image Then, the Re-ID branch is used to extract the feature map Extract the target Re-ID features Used for target tracking, where x represents the horizontal coordinate of the center point of the target, y represents the vertical coordinate of the center point of the target, and C represents the number of categories.

[0085] The Re-ID branch is a convolutional layer containing 128 kernels, which is used to extract feature maps. Re-ID features at each position in , generating feature maps Then take (x, y) as the center from the feature map Extract the target Re-ID features

[0086] Loss function design for target detection tasks

[0087] The network output of this method includes the results of the target detection task and the pedestrian target features of the Re-ID task. For the learning of the target detection task, this method follows the loss function design of the pedestrian target detection network based on the target center. The specific loss is calculated as follows:

[0088] 1) Calculation of target center point category loss

[0089]

[0090] Where, L cls represents the target center point category loss, represents the target center heat map predicted by the network, Y represents the probability that the point on the real heat map belongs to the target center point, and α represents the balance factor, usually α = 0.5;

[0091] 2) Calculation of target center point position offset loss

[0092]

[0093] Where, L off represents the target center position offset loss, G O represents the true target center offset, Indicates the predicted coordinate offset;

[0094] 3) Target area size loss calculation

[0095]

[0096] Where, L size represents the target area size loss, Represents the predicted length and width of the target, G S Indicates the actual length and width of the target;

[0097] 4) Loss calculation of pedestrian target detection network

[0098] L det =L cls +λ off L off +λ size L size

[0099] Where λ off and λ size Usually, they are set to 1 and 0.1 respectively.

[0100] Loss function design for Re-ID task

[0101] For the learning of pedestrian target embedding features in the Re-ID task, this method regards it as a classification task to learn the features of identifying different identity IDs. During the network training process, all targets with the same identity ID in the dataset are regarded as the same category, and their IDs are used as the classification labels for Re-ID.

[0102] Through each true target box in the image Get the center of the target on the heat map Extract the Re-ID feature vector of pedestrian targets And use the fully connected layer and softmax function to map it to the class distribution vector P = {p(k), k∈[1,K]}, and represent the one-hot encoding of the label as L i (k), Re-ID loss calculation:

[0103]

[0104] Where, L identify represents the Re-ID loss, k represents the number of all target IDs in the training set;

[0105] Overall loss calculation:

[0106]

[0107] Where w1 is the learnable parameter for the balanced object detection task, and w2 is the learnable parameter for the balanced Re-ID task.

[0108] Experimental results and analysis

[0109] 1. Dataset and Experimental Setup

[0110] This method is tested on the MOT17 dataset. Since labels are not provided in the MOT17 test set, the results of this method need to be uploaded to the MOT Challenge official website for testing in order to obtain evaluation. However, due to the limitation of the number of tests, this method uses the MOT20 training set as the validation set and conducts ablation experiments on the validation set.

[0111] The experimental environment for this method is Ubuntu 20.04LTS, and the model is trained using a GeForce RTX 3090. To ensure that this method performs well on the MOT17 dataset and accelerate model training, this paper first pre-trains it on the CrowdHuman dataset. This is because the CrowdHuman dataset can improve human detection performance and has strong domain generalization capabilities.

[0112] After pre-training, this paper combines City Person, Caltech, and MOT17 datasets into a single dataset. By adding more training data, this model enhances the generalization of pedestrian target features. The input image resolution is 1088×608, the model is trained for 30 epochs, the initial learning rate is 0.00001, and the batch size is 8. The learning rate is reduced by a factor of 10 every 20 epochs.

[0113] 2. Comparison and analysis of experimental results

[0114] To compare our method with existing multi-target tracking methods, we conducted validation experiments on the MOT17 dataset. Table 1 shows the comparison results between our method and existing multi-target tracking algorithms. Compared to existing tracking methods, our method performs equally well in almost all metrics and achieves optimal results in both IDs and IDF1, thus enabling excellent long-term and stable tracking.

[0115] Table 1. Comparison of experimental results

[0116]

[0117]

[0118] 3. Comparison and analysis of visualization results

[0119] In order to better demonstrate the decoupling effect of the cross-attention network on the target detection task and the Re-ID task, this paper fuses the attention map output by the network with the original image. Figure 4 As shown in the figure, three different tracking scene graphs are selected in the test set as the effect display graphs, among which, Figure 4The first row shows the detection results without the decoupling module, while the second row shows the detection results after decoupling using the cross-attention network. As can be seen, the response area of ​​the pedestrian target within the red circle in each column of the first row is significantly smaller than that in the second row. This not only affects the detection task, but also the tracking task. Therefore, it can be seen that there is a certain conflict between the object detection task and the Re-ID task during the learning process, and decoupling them through the cross-attention network can alleviate this learning conflict.

[0120] like Figure 5 As shown in the figure, the tracking status of this method on the MOT17 test set is demonstrated. Each row in the figure represents a video sequence in the MOT17 test set, and each column from left to right represents the tracking status of this method every 30 frames. It can be seen from the figure that this method can achieve good long-term stable tracking effect in target tracking, proving the effectiveness and feasibility of this method.

[0121] In summary, this paper proposes a method for object detection and tracking in autonomous driving scenarios based on graph neural networks. By leveraging the powerful information aggregation capabilities of graph neural networks to strengthen the connection between objects in the current frame and those in the previous frame, the network outputs highly discriminative object embedding features. This alleviates challenges such as poor compatibility between object detection and multi-object tracking methods, difficulty detecting pedestrians, and low tracking efficiency. Extensive experiments on datasets demonstrate the effectiveness of the proposed method.

Claims

1. A method for target detection and tracking in autonomous driving scenarios based on graph neural networks, characterized in that: The following steps are involved: S1. The previous frame image I in the video t-1 As the input of the first backbone network, the feature map F of the previous frame image is obtained through the first backbone network t-1 , and from the feature map F t-1 Extract the central feature vector C of the pedestrian target t-1 ; At the same time, the current frame image I in the video t As the input of the second backbone network, the feature map F of the current frame image is obtained through the second backbone network t ; S2. Using the cross attention network, a pedestrian target detection network based on the target center point is established, and the feature map F of the current frame image is converted into t As the input of the cross attention network, the feature map F for the Re-ID task is obtained through the cross attention network. T1 And the feature map F for the target detection task T2 , and by the feature map F T2 Perform convolution and output the target detection result, which is used as the previous frame image of the next calculation cycle; S3. Establish a target embedding feature extraction algorithm based on graph neural network, and transform the central feature vector C of the previous frame image t-1 And the feature map F of the current frame image T1 As input to graph neural networks; The graph neural network transforms the central feature vector C t-1 Build the target graph G as the target node t-1 , and use the feature map F T1 The feature information in the target node is generated to construct the target graph G t ; Through the graph neural network to the target graph G t-1 and G t Perform multiple neural networks and attention aggregation to obtain the target's Re-ID features for target tracking.

2. The method for target detection and tracking in autonomous driving scenarios based on graph neural networks according to claim 1, characterized in that: In S1, the backbone network adopts an enhanced version of the deep layer aggregation network DLA-34.

3. The method for target detection and tracking in autonomous driving scenarios based on graph neural networks according to claim 1, characterized in that: In S2, the feature map F is obtained through the cross attention network T1 and feature map F T2 The specific steps are as follows: 1) The feature map of the current frame image output by the backbone network is expressed as Use the cross attention network to pass it through the maximum pooling layer and the average pooling layer to obtain statistical information 2) Statistical information Through different convolutional layers, the three-dimensional feature map T1 for the Re-ID task and the three-dimensional feature map T2 for the detection task are generated respectively, and the intermediate feature maps T1 and T2 are reshaped into Where N'=H'×W', H' represents the height of the two-dimensional feature map, W' represents the width of the two-dimensional feature map, M1 represents the two-dimensional feature map used for the Re-ID task, and M2 represents the two-dimensional feature map used for the object detection task; 3) Perform matrix multiplication on the transpose of the two-dimensional feature maps M1 and M2, and generate the autocorrelation weight map through the softmax layer The calculation formula is as follows: Where, represents the relationship between the i-th and j-th channels, represents the transpose of M1 or M2 in the i-th channel, and represents the transpose of M1 or M2 in the j-th channel; 4) Perform rectangular multiplication between the transposes of the two-dimensional feature maps M1 and M2, and generate a cross-relationship weight map through the softmax layer The calculation formula is as follows: In the formula, 1 represents the Re-ID task, 2 represents the detection task, represents the impact of the Re-ID task on the detection task in the i-th channel, or the impact of the detection task on the Re-ID task in the j-th channel, represents the impact of the Re-ID task on the detection task in the i-th channel, and represents the impact of the detection task on the Re-ID task in the j-th channel; 5) By fusing the autocorrelation weight and the cross-correlation weight through the trainable parameter λ, we get The calculation formula is as follows: W 1 / 2 =λ×W S1 / S2 +(1-λ)×W M1 / M2 Where W S1 / S2 Represents the autocorrelation weight of the Re-ID task or detection task, W M1 / M2 Represents the cross-relationship weight of the Re-ID task or detection task, W 1 / 2 Represents the weight mapping after the fusion of Re-ID task or detection task; 6) The cross attention network converts the original feature map F t Rearrange to The shape of N = H × W, where N = H × W, and then perform matrix multiplication between the rearranged features and the weight map fused in 5) to obtain enhanced features for the Re-ID task and the detection task, and the enhanced features are combined with the original feature map F through residual attention. t Fusion, and finally form the feature map F for the Re-ID task T1 And the feature map F of the detection task T2 .

4. The method for target detection and tracking in autonomous driving scenarios based on graph neural networks according to claim 1, characterized in that: In S2, the loss function of the pedestrian target detection network is designed as follows: 1) Calculation of target center point category loss Where, L cls represents the target center point category loss, represents the target center heat map predicted by the network, Y represents the probability that the point on the real heat map belongs to the target center point, and α represents the balance factor, usually α = 0.5; 2) Calculation of target center point position offset loss Where, L off represents the target center position offset loss, G O represents the true target center offset, Indicates the predicted coordinate offset; 3) Target area size loss calculation Where, L size represents the target area size loss, Represents the predicted length and width of the target, G S Indicates the actual length and width of the target; 4) Loss calculation of pedestrian target detection network L det =L cls +λ off L off +λ size L size Where λ off and λ size Usually, they are set to 1 and 0.1 respectively.

5. The method for target detection and tracking in autonomous driving scenarios based on graph neural networks according to claim 1, characterized in that: In S3, the feature map F of the current frame image is T1 Expressed as Through the feature map The feature information of each position in generates W×H target nodes and constructs the target graph G t , where W represents the width of the feature map and H represents the height of the feature map; Through the graph neural network to the target graph G t-1 and G t Perform multiple neural networks and attention aggregation, update the node information between the two target graphs, and output the updated pedestrian target feature vector of the previous frame image And the updated feature map of the current frame image Then, the Re-ID branch is used to extract the feature map Extract the target Re-ID features Among them, x represents the horizontal coordinate of the center point of the target, y represents the vertical coordinate of the center point of the target, and C represents the number of categories.

6. The method for target detection and tracking in autonomous driving scenarios based on graph neural networks according to claim 5, characterized in that: The Re-ID branch is a convolutional layer containing 128 kernels, which is used to extract feature maps. Re-ID features at each position in , generating feature maps Then take (x, y) as the center from the feature map Extract the target Re-ID features.

7. The method for target detection and tracking in autonomous driving scenarios based on graph neural networks according to claim 1, characterized in that: The loss function of the target embedding feature extraction algorithm is designed as follows: Through each true target box in the image Get the center of the target on the heat map Extract the Re-ID feature vector of pedestrian targets And use the fully connected layer and softmax function to map it to the class distribution vector P = {p(k), k∈[1,K]}, and represent the one-hot encoding of the label as L i (k), Re-ID loss calculation: Where, L identify represents the Re-ID loss, k represents the number of all target IDs in the training set; Overall loss calculation: Where w1 is the learnable parameter for the balanced object detection task, and w2 is the learnable parameter for the balanced Re-ID task.

Images