Air-to-ground target re-identification method based on neighborhood feature set attention

Abstract

The application discloses a kind of based on neighborhood feature set attention air-to-ground target re-identification method, comprising: S1, the multi-source data set comprising re-identification ReID data set and general classification data set is constructed;S2, the deep metric learning network model comprising feature extraction module, neighborhood feature attention module and similarity evaluation module is constructed;S3, based on multi-source data set, the deep metric learning network model is iteratively trained;S4, the image to be identified and the all images of query set in re-identification ReID data set are respectively input into the deep metric learning network model trained, and the candidate image of first place in order is selected as the final matching result of the target re-identification to be identified.The NFCA attention module of the application enhances the semantic consistency of cross-feature by neighborhood feature centralization, retains accurate position coding, effectively suppresses background noise, and improves the robustness of features to view angle change and occlusion.

Description

Technical Field

[0001] This invention relates to the field of computer vision. More specifically, this invention relates to an air-to-ground target re-identification method based on neighborhood feature-focused attention, used in the fields of UAV perception and intelligent surveillance to solve the problem of accurate target matching and re-identification in challenging scenarios such as cross-viewpoint, lighting changes, and complex backgrounds. Background Technology

[0002] Air-to-ground image target re-identification aims to match targets under different camera perspectives, lighting conditions, and occlusion scenarios using computer vision technology. It is a core supporting technology for fields such as UAV reconnaissance and intelligent security. However, existing technologies suffer from the following key bottlenecks in air-to-ground scenarios, which severely restrict the re-identification accuracy and generalization ability of the models.

[0003] Interference from invalid information: In air-to-ground scenarios, the viewpoint changes rapidly and pixel-level targets are densely packed. Traditional deep learning networks tend to focus on background noise, redundant features, or non-critical local areas, while ignoring the core features of the target. For example, in images taken by drones, background information such as ground vegetation and buildings is easily misclassified by the network as important features, leading to a decrease in the discriminative power of the extracted features and directly affecting the accuracy of target re-identification.

[0004] The problem of non-adaptive similarity evaluation: Existing methods mostly rely on classification loss to indirectly optimize the feature space and use non-deep learning linear metrics such as Euclidean distance and cosine distance for similarity evaluation. However, in air-to-ground scenarios, airborne cameras move quickly and change significantly from different perspectives, resulting in a highly nonlinear distribution of target features. Therefore, linear metrics cannot characterize the complex nonlinear feature relationships between the same target from different perspectives in air-to-ground scenarios, making it difficult to meet the needs of real-world scenarios.

[0005] Insufficient data coverage is a problem: Traditional re-identification methods rely on finely labeled ReID datasets (assigning a unique ID to each object), but obtaining samples in air-to-ground scenarios is difficult, and limited training data is insufficient to cover the feature diversity of real-world scenarios (such as changes in target characteristics under different lighting, poses, and occlusions). The large difference between the distribution of training data and the distribution of real-world scenarios leads to high model generalization errors and a sharp decline in performance in unknown scenarios.

[0006] Current research attempts to address these issues by employing attention mechanisms to enhance feature discrimination or by using adaptive methods, but a unified solution encompassing "attention mechanism-metric adaptation-data supplementation" has yet to be established, making it difficult to simultaneously resolve the three major bottlenecks. Therefore, this patent proposes an air-to-ground target re-identification method that can simultaneously suppress invalid information, adapt to nonlinear feature distributions, and expand data coverage. Summary of the Invention

[0007] One object of the present invention is to solve at least the above-mentioned problems and / or defects, and to provide at least the advantages described below.

[0008] To achieve these objectives and other advantages of the present invention, an air-to-ground target re-identification method based on neighborhood feature-focused attention is provided, comprising: S1. Construct a multi-source dataset that includes a ReID dataset and a general classification dataset; S2. Construct a deep metric learning network model that includes a feature extraction module, a neighborhood feature attention module, and a similarity evaluation module; S3. Iteratively train the deep metric learning network model based on multi-source datasets; S4. Input the image to be identified and all the images in the query set of the ReID dataset into the trained deep metric learning network model respectively, and select the candidate image ranked first as the final matching result of the re-identification of the target to be identified. The feature extraction module uses ResNet34 as the backbone network, and the neighborhood feature concentration attention (NFCA) module introduces the neighborhood feature concentration mechanism (NFC) on the basis of coordinate attention (CA) to enhance cross-feature interaction and preserve position encoding. The similarity evaluation module constructs a data-driven nonlinear similarity evaluation function to directly evaluate the similarity of sample pairs as the target.

[0009] Preferably, in S1, the ReID dataset includes: the JC-2 dataset specifically for air-to-ground scenarios and a public ReID dataset; The JC-2 dataset contains 681 vehicle targets and 5470 corresponding detection boxes. All samples were captured by drones at different times and distances. The public ReID datasets include the Market-1501 pedestrian dataset and the VehicleID vehicle dataset, and each target in the public ReID dataset is assigned a unique ID and contains multi-view images; The general classification dataset is a general image dataset without ReID annotation, including: OxfordFlower class data, food-101 class data, and Omniglot class data. For each class, 80-100 images with the same shape are randomly selected to increase the diversity of features.

[0010] Preferably, in a multi-source dataset, for each target ID, images with the same ID are randomly selected to form a positive sample pair with a label of 1, and images with different IDs are selected to form a negative sample pair with a label of 0. The positive and negative sample pairs are both augmented to increase data diversity.

[0011] Preferably, in S3, positive and negative sample pairs are randomly selected as batch sample pairs, and the batch sample pairs are input into the Siamese network deep metric learning network model for iterative training. The optimizer for the deep metric learning network model was Adam, with an initial learning rate of 0.0001 and a weight decay coefficient of 1e-5. During the iterative training process, the batch size was set to 64, and a total of 120 training rounds were performed. After the loss converged, the network parameters with the lowest loss during training were used as the model for subsequent inference stages, thus completing the training of the deep metric learning network model.

[0012] Preferably, the workflow of the neighborhood feature focused attention (NFCA) module is as follows: S210. The high-dimensional feature map output by the backbone network. y Pooling is performed horizontally and vertically to obtain two sub-feature maps. y 1 and y 2; S211, will y 1. y 2. After encoding via vertical and horizontal channels, y 1. y 2. Transformed into a feature map after 2D global pooling A h , A w ; S211. The features after neighborhood feature centralization are obtained by the following formula. y ′: In the above formula, ⊕ represents feature splicing. F NFC () represents the neighborhood feature centralization operation; S211, will y Re-decomposed into sub-feature maps along the channel dimension y 1′ and y 2′; S211. The weighted attention features are calculated using the following formula. : In the above formula, C 1 is a convolutional layer used to build attention.

[0013] Preferably, the nonlinear similarity evaluation function It is characterized by the following formula: In the above formula, f i and f j These are two features output by the Siamese network. f fla It is a fully connected layer. σ This is the Sigmoid function.

[0014] The present invention has at least the following beneficial effects: First, the feature discrimination power of the present invention is significantly improved. The NFCA attention module enhances cross-feature semantic consistency by centering neighborhood features, retains accurate position encoding, effectively suppresses background noise, and improves the robustness of features to changes in viewpoint and occlusion.

[0015] Secondly, the nonlinear similarity evaluation employed in this invention is more accurate: the end-to-end learned similarity function overcomes the limitations of linear metrics such as Euclidean distance, accurately characterizing the nonlinear feature distribution of air-to-ground scenarios. Compared with traditional linear metric methods, it solves the problem of "disconnect between training error and actual performance".

[0016] Third, the generalization ability of this invention is significantly enhanced: the multi-source data fusion strategy expands the coverage of the feature space, reduces the distance between the training data distribution and the real distribution, and lowers the generalization error. On the self-developed JC-2 dataset and the public datasets Market-1501 and VehicleID, the mAP reaches 82.1%, 92.8%, and 83.6%, respectively, surpassing advanced methods such as TransReID by 1.2%-2.4%.

[0017] Third, this invention has the advantages of being lightweight and efficient: the NFCA module has no additional training parameters, avoiding the risk of overfitting, making it suitable for deployment on edge devices such as drones, and balancing accuracy and efficiency.

[0018] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0019] Figure 1 This is a network architecture diagram of the deep metric learning network of the present invention; Figure 2 This is a schematic diagram of the neighborhood feature concentration attention module of the present invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0021] This invention proposes an air-to-ground target re-identification method based on neighborhood feature-based attention. It achieves accurate matching through a three-dimensional optimization process of "feature enhancement, metric adaptation, and data fusion." The specific technical solution is as follows: S1. Construction and Preprocessing of Multi-Source Datasets S10. Data collection is performed. This step combines a ReID dataset with a general classification dataset to expand the coverage of the feature space. The ReID dataset mainly includes: the JC-2 dataset for air-to-ground scenarios and the pedestrian and vehicle ReID dataset. Specifically, the JC-2 dataset for air-to-ground scenarios contains 681 vehicle targets and 5470 corresponding detection boxes. All samples were obtained by drones at different times and distances. Pedestrian and vehicle ReID datasets, such as the Market-1501 pedestrian dataset and the VehicleID vehicle dataset, are datasets in which each target is assigned a unique ID and contains multi-view images.

[0022] The general classification dataset introduces general image datasets without ReID annotations, such as OxfordFlower, food-101, and Omniglot datasets. 80-100 images with the same shape are randomly selected for each category to supplement the diversity of features.

[0023] S11. Generate sample pairs. For each target ID, randomly select images with the same ID to form a positive sample pair with a label of 1, and select images with different IDs to form a negative sample pair with a label of 0, so as to provide positive and negative sample support for model training.

[0024] S12. Perform data augmentation to further enrich the diversity of data and improve the model's adaptability to changes in different scenarios.

[0025] S2, Network Model Construction like Figure 1 As shown, an end-to-end deep metric learning network was designed, whose core includes a feature extraction module, a neighborhood feature concentration attention (NFCA) module, and a similarity evaluation module. The NFCA module is implemented in a plug-and-play manner. The feature extraction module uses ResNet34 as the backbone network. After removing the last fully connected layer of the network, it outputs a high-dimensional feature map with shape [B, C, W, H] to capture the global and local features of the target.

[0026] High-dimensional features output by the feature extraction module y The input is used to obtain attention features from the NFCA module. The architecture and data flow of the NFCA module are as follows: Figure 2 As shown, the processing flow includes the following: S210, Input high-dimensional features y Pooling is performed along two channel dimensions, horizontal and vertical, splitting the feature map into two sub-feature maps. After channel encoding, their shapes are [B, C, 1, H] and [B, C, W, 1], respectively. S211. The two sub-feature maps are concatenated along the channel. Then, the concatenated sub-feature map is enhanced using the neighborhood feature concentration mechanism. That is, the features after neighborhood feature concentration are calculated by equation (1). y ′: (1) Wherein, ⊕ represents feature concatenation; A h and A w For high-dimensional features y Perform horizontal and vertical 2D global pooling. After pooling, the feature sizes are [B, C, W, 1] and [B, C, 1, H], respectively. F NFC () represents the neighborhood feature centralization operation; S212, Features y Sub-feature maps are obtained by segmenting along the channel. y 1′ and y 2′, then through a 1×1 convolutional layer C 1. Adjustments are made to generate horizontal and vertical attention weights. Then, through attention weighting, the weights are fused with the original feature map to output attention-enhanced features. The weighted attention features are obtained through equation (2). : (2) in, y 1′ and y 2′ is y The features of the channel dimension split are [B, C, W, 1] and [B, C, 1, H] after splitting; C 1 is a convolutional layer used to build attention; y It is the high-dimensional feature output by the feature extraction module. y , and after building attention y 1′ and y 2' Perform matrix multiplication to obtain the weighted attention features y ′′.

[0027] The similarity evaluation module contains fully connected layers. Its input is the absolute value of the difference between the output feature pairs of the Siamese network, and the output features are of shape [B, S]. Finally, the final similarity result is obtained through a sigmoid function mapping. Specifically, the similarity evaluation module constructs a data-driven non-linear similarity evaluation function. It directly uses sample pair similarity as the target, and learns nonlinear metric relationships through fully connected layers, that is, it measures the similarity between features through equation (3): (3) in, f i and f j These are two features output by the Siamese network. f fla It is a fully connected layer. σ This is the Sigmoid function. (Through...) f i and f j The difference between feature pairs is represented by taking the absolute value after subtraction. The result is then input into a fully connected layer, and the network output is mapped to the [0,1] interval using the Sigmoid function to directly obtain the similarity.

[0028] S3, Model Training S30. In the training process, positive and negative sample pairs are first randomly selected from the constructed multi-source dataset. These sample pairs contain positive samples of the same ID target and negative samples of different ID targets, providing rich learning samples for model training.

[0029] S31. Perform parameter initialization. The backbone network uses weights pre-trained on the ImageNet dataset, while other parameters in the network are randomly initialized to ensure the reasonable starting point of model training.

[0030] S32. The processing flow for the iterative training phase is as follows: S320. The selected batch of sample pairs is input into the Siamese network. After being processed layer by layer by the feature extraction module, the samples enter the NFCA module for feature enhancement. This module enables the network to focus on the core features of the target and suppress invalid information. S321. The similarity of sample pairs is calculated using an end-to-end sample pair similarity evaluation mechanism, and the BCE loss is calculated. The similarity evaluation mechanism learns the relationship between features through a fully connected layer and obtains the similarity result through a Sigmoid function mapping. S322. All network parameters are updated using the backpropagation algorithm to optimize the model. The optimizer used in iterative training is Adam, with an initial learning rate of 0.0001 and a weight decay coefficient of 1e-5. During training, the batch size is set to 64, and a total of 120 training rounds are performed. The learning rate is automatically adjusted based on changes in the loss value until the loss converges, ensuring the model can fully learn the features and patterns in the data, thus improving the model's generalization ability and recognition accuracy. S323. Save the network parameters at the point of lowest loss during training and use them as the model for subsequent inference stages.

[0031] S4, Target Re-identification Reasoning Target re-identification inference is the process of matching targets based on training weights after model training is completed. Its process includes: S40. Prepare the input by determining the target image to be recognized and the query set in the ReID dataset for re-identification (it should be noted that the query set is a set that has been partitioned in the existing ReID dataset and will not be described here). S40. Input the target image to be identified and all images in the candidate image library into the trained network respectively. The network will perform feature extraction and processing on these two types of images respectively. S40. After processing by the feature extraction module and the NFCA module, the network outputs the enhanced features of the target to be identified and all images in the candidate image library, and then calculates the similarity S between the features of the target to be identified and the features of each image in the candidate image library.

[0032] S40. Sort all similarities in descending order, and after processing with a preset threshold, select the candidate image that ranks first (Top-1) and use it as the final matching result for the re-identification of the target to be identified.

[0033] The above solution is merely an illustration of a preferred example and is not limited thereto. When implementing this invention, appropriate substitutions and / or modifications can be made according to the user's needs.

[0034] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for the present invention. Other modifications can be readily made by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and examples shown and described herein.

Claims

1. A method for re-identifying air-to-ground targets based on neighborhood feature-based attention, characterized in that, include: S1. Construct a multi-source dataset that includes a ReID dataset and a general classification dataset; S2. Construct a deep metric learning network model that includes a feature extraction module, a neighborhood feature attention module, and a similarity evaluation module; S3. Iteratively train the deep metric learning network model based on multi-source datasets; S4. Input the image to be identified and all the images in the query set of the ReID dataset into the trained deep metric learning network model respectively, and select the candidate image ranked first as the final matching result of the re-identification of the target to be identified. The feature extraction module uses ResNet34 as the backbone network, and the neighborhood feature concentration attention (NFCA) module introduces the neighborhood feature concentration mechanism (NFC) on the basis of coordinate attention (CA) to enhance cross-feature interaction and preserve position encoding. The similarity evaluation module constructs a data-driven nonlinear similarity evaluation function to directly evaluate the similarity of sample pairs as the target.

2. The air-to-ground target re-identification method based on neighborhood feature-based attention as described in claim 1, characterized in that, In S1, the ReID dataset includes: the JC-2 dataset specifically for air-to-ground scenarios and the public ReID dataset; The JC-2 dataset contains 681 vehicle targets and 5470 corresponding detection boxes. All samples were captured by drones at different times and distances. The public ReID datasets include the Market-1501 pedestrian dataset and the VehicleID vehicle dataset, and each target in the public ReID dataset is assigned a unique ID and contains multi-view images; The general classification dataset is a general image dataset without ReID annotation, including: OxfordFlower class data, food-101 class data, and Omniglot class data. For each class, 80-100 images with the same shape are randomly selected to increase the diversity of features.

3. The air-to-ground target re-identification method based on neighborhood feature-based focused attention as described in claim 1, characterized in that, In a multi-source dataset, for each target ID, images with the same ID are randomly selected to form a positive sample pair with a label of 1, and images with different IDs are selected to form a negative sample pair with a label of 0. The positive and negative sample pairs are both augmented to increase data diversity.

4. The air-to-ground target re-identification method based on neighborhood feature-based focused attention as described in claim 3, characterized in that, In S3, positive and negative sample pairs are randomly selected as batch sample pairs, and the batch sample pairs are input into the Siamese network deep metric learning network model for iterative training. The optimizer for the deep metric learning network model was Adam, with an initial learning rate of 0.0001 and a weight decay coefficient of 1e-5. During the iterative training process, the batch size was set to 64, and a total of 120 training rounds were performed. After the loss converged, the network parameters with the lowest loss during training were used as the model for subsequent inference stages, thus completing the training of the deep metric learning network model.

5. The air-to-ground target re-identification method based on neighborhood feature-based focused attention as described in claim 1, characterized in that, The workflow of the neighborhood feature-focused attention (NFCA) module is as follows: S210. Pool the high-dimensional feature map y output by the backbone network along the horizontal and vertical directions to obtain two sub-feature maps y1 and y2. S211. After encoding y through the vertical and horizontal channels, y is transformed into a feature map A after 2D global pooling. h A w ; S211. The feature y′ after neighborhood feature centralization is obtained by the following formula: In the above formula, ⊕ represents feature splicing, F NFC () represents the neighborhood feature centralization operation; S211. Re-divide y′ into sub-feature maps y1′ and y2′ along the channel dimension; S211. The weighted attention features are calculated using the following formula. : In the above formula, C1 is the convolutional layer used to build attention.

6. The air-to-ground target re-identification method based on neighborhood feature-based focused attention as described in claim 1, characterized in that, The nonlinear similarity evaluation function It is characterized by the following formula: In the above formula, f i and f j These are two features output by the Siamese network, f fla For a fully connected layer, σ is the Sigmoid function.

Images