Radar insect and bird classification method based on multi-scale feature fusion

Abstract

The invention discloses a radar insect and bird classification method based on multi-scale feature fusion. The method comprises the following steps: firstly, carrying out short-time Fourier transform (STFT) on a radar echo signal, and carrying out multi-scale time-frequency analysis by adopting multiple window lengths to generate a multi-scale time-frequency graph containing different time resolutions and frequency resolutions; the multi-scale time-frequency map is then input into a hierarchical feature extraction network, which includes a generic visual feature branch and a radar-specific physical feature branch. And finally, completing classification of insects, birds and clutters through a multi-task classification head. According to the method, general vision priori and radar field knowledge are effectively fused, multi-scale information utilization is optimized through an attention mechanism, automatic, high-precision and high-robustness classification of insect and bird targets in radar monitoring can be achieved, and the method has practical application value.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of bio-radar and artificial intelligence, and specifically relates to a method for high-precision and automated classification of migratory insects and birds using radar time-frequency maps. Background Technology

[0002] Bio-radar is an effective tool for monitoring large-scale migratory insects and birds. Species identification through radar echo analysis is of great significance for ecological protection and early warning of migratory pests. The core task in radar bio-monitoring is the accurate classification of detected individual targets, distinguishing between insects and birds, which forms the basis for all subsequent analyses of migratory behavior.

[0003] In existing bio-radar monitoring practices, common methods often utilize the echo intensity, morphology, or time-frequency images of individual targets for feature extraction and classification. However, due to the complex time-frequency coupling effect of radar target signals, different categories of targets exhibit some overlap in their feature representations. Coupled with the influence of background noise and clutter, the robustness and accuracy of classification remain challenging.

[0004] Especially in deep learning-based time-frequency map classification methods, convolutional networks pre-trained on natural images are often used directly to process radar time-frequency maps. These methods tend to overlook the unique physical patterns of radar maps, such as Doppler shift and temporal dynamics, thus limiting the model's classification potential in complex environments. Therefore, there is an urgent need to develop a novel method that integrates general visual representation capabilities with radar-specific knowledge to achieve more stable and accurate automated identification of insect and bird targets. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned challenges by proposing a radar insect and bird classification method based on multi-scale feature fusion. This method combines the general visual feature extraction capabilities of the large-scale pre-trained model CLIP with a specially designed radar physical feature enhancement module. It achieves efficient feature fusion through multi-scale time-frequency analysis and cross-scale attention mechanisms, ultimately completing the automated classification of insects, birds, and clutter through multi-task learning. The technical solution is as follows: Step 1: Data Preprocessing and Multi-Scale Time-Frequency Analysis Short-Time Fourier Transform (STFT) was used to perform time-frequency analysis on the radar target echo signals to be classified. In order to take into account time details, frequency details, and time-frequency equalization, three window functions of different lengths—short window, medium window, and long window—were selected to process the same signal, resulting in three time-frequency maps with different time-frequency resolutions. An interpolation factor was introduced during the transformation process to improve the spectral precision. Subsequently, the center frequency of the spectrum was calculated based on the amplitude-weighted average, and the time-frequency map was cropped based on this center frequency, retaining only the effective information area. Finally, the cropped time-frequency map was normalized and scaled to a uniform size.

[0006] Step 2: Hierarchical Feature Extraction The three time-frequency images at different scales obtained in step one are then input into a parallel two-branch feature extraction network: The first branch is the general visual feature extraction branch, which uses a pre-trained CLIP visual encoder to extract deep general semantic features from the time-frequency map by freezing the parameters of the front network layers of the model and only fine-tuning the parameters of the rear network layers. The second branch is the radar-specific physical feature extraction branch, which contains two parallel substructures. It extracts the Doppler features of the target using convolution kernels along the frequency axis and extracts the temporal dynamic features of the target using convolution kernels along the time axis. After concatenating and fusing the Doppler features and the temporal dynamic features, the radar-specific features are obtained by global average pooling. Finally, at each scale, the general semantic features are concatenated with radar-specific features to form a set of multi-scale enhanced feature vector sequences.

[0007] Step 3: Cross-scale attention feature fusion The enhanced feature vector sequence obtained in step two is input into the cross-scale attention module. First, the features are mapped to the attention space through linear projection. The correlation score between features at each scale in the sequence is calculated using a multi-head self-attention mechanism. Based on this, context-aware information enhancement is performed on the feature representation at each scale. Then, learnable dynamic scale weights are introduced to perform weighted summation on the enhanced features at different scales to generate a global fusion feature vector.

[0008] Step 4: Multi-task classification and model training The global fusion feature vectors obtained in step three are input in parallel into the main classification head and the auxiliary classification head. The main classification head performs the main classification task of distinguishing between insects, birds, and clutter, while the auxiliary classification head performs the auxiliary classification task of distinguishing between living and non-living things. A combined loss function containing the main classification loss and the auxiliary classification loss is constructed, where Focal Loss is used for the main classification task to solve the class imbalance problem, and cross-entropy loss is used for the auxiliary classification task. The entire model is then optimized and trained end-to-end using this combined loss function.

[0009] Beneficial effects: 1. Superior feature representation: By combining the powerful and general visual prior of the CLIP model with the specific physical patterns of radar time-frequency maps, the features extracted through the dual-branch structure are more expressive in distinguishing insects, birds and clutter. 2. Full utilization of multi-scale information: Through the cross-scale attention mechanism, the model can automatically learn the importance of information at different scales and perform intelligent fusion, which effectively improves the robustness of classification of targets with different shapes and signal-to-noise ratios.

[0010] 3. Enhanced training and generalization capabilities: The introduction of auxiliary classification tasks provides additional optimization signals, while Focal Loss is used to address class imbalance, resulting in stronger stability and better generalization capabilities during model training.

[0011] 4. More reliable automated classification results: Through an end-to-end deep transfer learning framework, high-precision and automated processing of insect and bird classification in radar monitoring scenarios is achieved, which has practical value. Attached Figure Description

[0012] Figure 1 The image shows a three-scale time-frequency diagram of an insect echo signal after short-time Fourier transform analysis and preprocessing. Figure 2 The image shows a three-scale time-frequency diagram of a bird echo signal after short-time Fourier transform analysis and preprocessing. Figure 3 The image shows a three-scale time-frequency diagram of a certain clutter signal after short-time Fourier transform analysis and preprocessing. Figure 4 1. A schematic diagram of the structure of the general visual extraction branch; Figure 5 A schematic diagram of the structure for extracting branches for specific physical features of radar; Figure 6 A schematic diagram of the cross-scale attention module; Figure 7 1. A structural diagram of the multi-task classification module; Figure 8 A confusion matrix diagram for classifying insects and birds; Figure 9 The above is a flowchart of the present invention. Detailed Implementation

[0013] The method provided by this invention aims to achieve high-precision insect and bird classification from radar time-frequency maps. Its core process includes four main steps: data preprocessing, hierarchical feature extraction, cross-scale attention fusion, and multi-task classification.

[0014] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0015] Step 1: Data Preprocessing and Multi-Scale Time-Frequency Analysis The goal of this step is to convert the one-dimensional radar echo time-domain signal into a two-dimensional, multi-scale time-frequency image. The radar target echo signals to be classified... Time-frequency analysis is performed using Short-Time Fourier Transform (STFT). Its mathematical expression is: (1) in, For window functions, This is the result of the short-time Fourier transform. It is a local time variable used to describe time relative to the center time. The offset. To achieve multi-scale analysis, this embodiment uses three window functions of different lengths: Short window (16 points): Provides high temporal resolution for capturing rapidly changing temporal details; Medium window (32 points): Balances time-frequency resolution and provides comprehensive information; Long window (64 points): Provides high frequency resolution for accurate analysis of Doppler features.

[0016] For the same signal segment Processing with three different window lengths yields three time-frequency analysis plots with varying time-frequency resolutions, termed the three-scale time-frequency plots. These three plots respectively focus on capturing the temporal details, frequency details, and time-frequency equalization representation of the signal. Simultaneously, a 10x interpolation factor is introduced during the Fourier transform of each time-domain segment to improve spectral detail. Subsequently, to eliminate the impact of spectral center frequency drift on the analysis results, the center frequency (spectral centroid) is estimated based on an amplitude-weighted average. : (2) in This indicates summation along the frequency dimension. This represents the modulus of the short-time Fourier transform. At this center frequency... Based on the baseline, in the interval [ f c -400Hz, f c The time-frequency image is cropped within a range of +400Hz, retaining only the effective information area. Finally, the cropped time-frequency image is normalized to the [0, 1] interval and uniformly scaled to 224×224 pixels to adapt to the input size of the subsequent neural network. Figure 1 This paper presents a three-scale time-frequency diagram of an insect echo signal after short-time Fourier transform analysis and preprocessing. Figure 2This paper presents a three-scale time-frequency plot of a bird echo signal after short-time Fourier transform analysis and preprocessing. Figure 3 This shows the three-scale time-frequency plot of a certain clutter signal after short-time Fourier transform analysis and preprocessing. Step 2: Hierarchical Feature Extraction This stage aims to extract rich and discriminative features from multi-scale time-frequency maps. The three preprocessed time-frequency maps... ( k (e.g., 1, 2, 3) are each fed into a parallel two-branch network structure: (A) General Visual Feature Extraction Branch: The processed time-frequency map is input into the CLIP visual encoder. The backbone of CLIP is the ViT-L / 14 model, which contains 24 Transformer layers, and its structure is as follows: Figure 4 As shown. To balance versatility and adaptability, this embodiment freezes the parameters of the first 17 layers and only fine-tunes the parameters of the last 7 layers. Through this progressive transfer learning, the model can retain general visual priors while learning radar time-frequency characteristics. Input time-frequency graph Entering this branch yields the deep feature vector: (3) in This refers to the CLIP vision encoder. d c The feature vectors output by CLIP Dimensions.

[0017] (B) Radar-specific physical feature extraction branch: This branch is a specially designed lightweight convolutional network, the structure of which is as follows: Figure 5 The diagram shows the physical pattern used to explicitly capture radar spectra. It comprises two parallel sub-branches:  Doppler feature extraction sub-branch: employing multiple sizes N A convolution kernel of ×1 (e.g., 7×1, 5×1) is used to perform convolution operations along the frequency axis of the time-frequency plot to capture the Doppler features of the target. For the input time-frequency plot... The Doppler feature map output by this branch is: (4) in It is the output feature map of the Doppler feature extraction branch. and This indicates the weights and biases of the convolution kernel in that branch. This represents the activation function.

[0018]  Temporal feature extraction sub-branch: employs multiple 1× MConvolutional kernels (such as 1×7, 1×5) are used to perform convolution operations along the time axis of the time-frequency map to capture the temporal dynamic features caused by target attitude changes, wing flapping, and other behaviors. Similarly, the temporal dynamic feature map extracted along the time direction can be represented as: (5) in It is the output feature map of the temporal feature extraction branch. and This indicates the weights and biases of the convolution kernel in that branch. This represents the activation function.

[0019] The feature maps extracted from the two sub-branches are concatenated along the channel dimension and fused using a standard 3×3 convolution. The fused feature map is then subjected to global average pooling to obtain radar-specific feature vectors. (6) in, dr Indicates radar characteristics The dimension, [;] indicates the concatenation operation. It is a global average pooling function used to compress feature maps into vectors.

[0020] Finally, the universal feature vector at each scale With radar-specific feature vectors By combining the features, a composite characteristic is formed: (7) in DC + dr Indicates comprehensive features Dimensions.

[0021] Step 3: Cross-scale attention feature fusion This stage aims to intelligently fuse feature information from different scales, and its structure is as follows: Figure 6 As shown. First, the set of enhanced feature vectors obtained in the previous stage... Through a linear projection layer, it is mapped to an attention space of uniform dimension: (8) in , , Represents the projection weight matrix. , , This represents the projected query, key, and value vector, used for subsequent attention calculation. The attention calculation formula is as follows: (9) in It is a normalization function. This is the key vector dimension, used for scaling. The attention mechanism dynamically assigns attention weights to different scales by calculating the correlation scores between features at each scale in the sequence, and accordingly performs context-aware augmentation on the feature representation at each scale, outputting an updated feature sequence. , , .

[0022] Next, to highlight the importance of different scales, learnable dynamic scale weights are introduced on top of attention fusion. And transform them into normalized weights using Softmax. Assign it to the new feature Explicit weighted summation is performed to obtain the final global fusion feature vector. : (10) This vector encapsulates the most critical information across all scales.

[0023] Step 4: Multi-task classification and model training This stage aims to utilize the fused features for the final classification decision and optimize the entire model. The global feature vector... Two parallel classification heads are fed in, with the following structure: Figure 7 As shown.

[0024] Main classifier head: A multilayer perceptron (MLP) consisting of one or more fully connected layers, used to perform the core three-class classification task, outputting a class vector indicating whether the target belongs to insects, birds or clutter, which is used for subsequent probability normalization and determination.

[0025] Auxiliary classification head: Consists of a fully connected layer, used to perform an auxiliary binary classification task, outputting a class vector indicating whether the target belongs to a living organism (insect, bird) or a non-living organism (clutter), which is used for subsequent probability normalization and determination.

[0026] During model training, this invention employs a multi-task joint optimization strategy, calculating the loss for each of the two tasks separately. The loss for the main classification task... Focal Loss is employed to effectively address the class imbalance problem commonly encountered in real-world observations. This loss method is used for auxiliary classification tasks. Standard cross-entropy loss is used. Total loss function. We obtain the following by weighted summation of these two losses: (11) in It is a hyperparameter that balances the importance of the two tasks. The introduction of Focal Loss can effectively alleviate the class imbalance problem, while the supervision signal of the auxiliary task provides additional regularization constraints for the model.

[0027] In the optimizer settings, an improved AdamW optimizer is used to avoid instability caused by the coupling of weight decay and adaptive learning rate. Based on the total loss... Backpropagation and gradient updates are performed on all trainable parameters in the model (including the CLIP fine-tuning layer, radar module, attention module, and classification head). The entire training process is conducted end-to-end. A learning rate scheduling scheme combining warm-up and cosine annealing is introduced, where the learning rate is gradually increased in the initial training stage to stabilize convergence, and gradually decreased in the mid-to-late stages to improve the model's generalization ability. To further enhance the model's robustness and generalization, various data augmentation techniques are incorporated, including a Mixup strategy based on Beta distribution and random rotation and horizontal flipping of the time-frequency graph, thereby generating additional data distributions during the training phase to alleviate overfitting. In addition, to ensure training stability, this invention normalizes and prunes gradients during backpropagation and sets weight decay coefficients to suppress overfitting; mixed precision is used to accelerate computation, and a distributed parallel strategy is used to meet the needs of efficient training on large-scale samples. For model convergence control, this invention introduces an early stopping mechanism based on validation set performance. When no performance improvement is observed for several consecutive rounds (e.g., 10 rounds), training is terminated early to reduce unnecessary computation.

[0028] To verify the effectiveness of the method of this invention, a radar target dataset containing insects, birds, and clutter was constructed. The dataset has a total of 7325 samples, including 2931 insect samples, 2088 bird samples, and 2306 clutter samples. The dataset was randomly divided into a training set (5127 samples), a validation set (1099 samples), and a test set (1099 samples) in a ratio of 7:1.5:1.5 according to the stratified sampling principle.

[0029] The experiment was implemented using the deep learning framework PyTorch, with a high-performance computing workstation equipped with seven NVIDIA V100 GPUs. The training parameters were set as follows: batch size of 32, initial learning rate of 0.0001, AdamW optimizer with a weight decay factor of 0.05, γ in the Focal Loss set to 2.0, and the balancing parameter λ set to 0.2.

[0030] The experiment used accuracy, precision, recall, and F1 score as performance evaluation metrics. Among them, accuracy reflects the overall correctness of the model's classification, while the F1 score comprehensively reflects the model's balanced performance across different classes.

[0031] The test set was input into the trained model for evaluation. Experimental results show that the recognition results of the method of this invention on the test set are as follows:

[0032] The overall accuracy reached 95.91%, demonstrating the effectiveness of the CLIP pre-trained model fusion with the physical feature branch. The recall rate for birds was as high as 99.68%, indicating that the model almost never missed bird targets, which is crucial for applications such as flight safety early warning. The F1-Scores for all three classes were above 94%, indicating that the model's performance across different classes is very balanced, with no obvious classification bias. The classification confusion matrix is ​​as follows. Figure 8 As shown.

[0033] In summary, this invention, through innovative design of network structure and learning strategy, effectively combines the powerful capabilities of general artificial intelligence models with expertise in the radar field, achieving high-precision and robust classification of radar targets such as insects and birds, and providing an advanced technical means for automated radar ecological monitoring. The scope of protection of this invention is not limited to the specific embodiments described above; any equivalent substitutions or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A radar insect and bird classification method based on multi-scale feature fusion, characterized in that, include: Step 1: Data Preprocessing and Multi-Scale Time-Frequency Analysis Short-time Fourier transform (STFT) was used to perform time-frequency analysis on the radar target echo signals to be classified. In order to take into account time details, frequency details and time-frequency equalization, three window functions of different lengths, namely short window, medium window and long window, were selected to process the same signal to obtain three time-frequency diagrams with different time-frequency resolutions. Interpolation factors were introduced in the transformation process to improve the spectral fineness. Subsequently, the center frequency of the spectrum is calculated based on the amplitude-weighted average, and the time-frequency graph is cropped using this center frequency as a reference, retaining only the effective information area; finally, the cropped time-frequency graph is normalized and scaled to a uniform size. Step 2: Hierarchical Feature Extraction The three time-frequency images at different scales obtained in step one are then input into a parallel two-branch feature extraction network: The first branch is the general visual feature extraction branch, which uses a pre-trained CLIP visual encoder to extract deep general semantic features from the time-frequency map by freezing the parameters of the front network layers of the model and only fine-tuning the parameters of the rear network layers. The second branch is the radar-specific physical feature extraction branch, which contains two parallel substructures. It extracts the Doppler features of the target using convolution kernels along the frequency axis and extracts the temporal dynamic features of the target using convolution kernels along the time axis. After concatenating and fusing the Doppler features and the temporal dynamic features, the radar-specific features are obtained by global average pooling. Finally, at each scale, the general semantic features are concatenated with radar-specific features to form a multi-scale enhanced feature vector sequence; Step 3: Cross-scale attention feature fusion The enhanced feature vector sequence obtained in step two is input into the cross-scale attention module. First, the features are mapped to the attention space through linear projection. Then, the correlation scores between features at different scales in the sequence are calculated using a multi-head self-attention mechanism, and context-aware information enhancement is performed on the feature representation at each scale accordingly. Next, learnable dynamic scale weights are introduced to perform weighted summation on the enhanced features at different scales, generating a global fusion feature vector. Step 4: Multi-task classification and model training The global fusion feature vectors obtained in step three are input in parallel into the main classification head and the auxiliary classification head. The main classification head performs the main classification task of distinguishing between insects, birds, and clutter, while the auxiliary classification head performs the auxiliary classification task of distinguishing between living and non-living things. A combined loss function containing the main classification loss and the auxiliary classification loss is constructed, where FocalLoss is used for the main classification task to solve the class imbalance problem, and cross-entropy loss is used for the auxiliary classification task. The entire model is then optimized and trained end-to-end using this combined loss function.

2. The radar insect and bird classification method based on multi-scale feature fusion as described in claim 1, characterized in that, In step one, three different window functions of varying lengths are selected: Short window (16 points): Provides high temporal resolution for capturing rapidly changing temporal details; Medium window (32 points): Balances time-frequency resolution and provides comprehensive information; Long window (64 points): Provides high frequency resolution for accurate analysis of Doppler features.

3. The radar insect and bird classification method based on multi-scale feature fusion as described in claim 1, characterized in that, In step one, the center frequency is estimated based on the amplitude-weighted average. : ； in This indicates summation along the frequency dimension. This represents the modulus of the short-time Fourier transform.

4. The radar insect and bird classification method based on multi-scale feature fusion as described in claim 1, characterized in that, In step two, the three time-frequency maps obtained from the preprocessing are... (k=1, 2, 3) are each fed into a parallel two-branch network structure: (A) General visual feature extraction branch: The processed time-frequency map is input into the CLIP visual encoder; the backbone of CLIP is the ViT-L / 14 model, which contains 24 Transformer layers; input time-frequency map Entering this branch yields the deep feature vector: ； in dc represents the CLIP visual encoder, and dc represents the feature vector output by CLIP. The dimension; (B) Radar-Specific Physical Feature Extraction Branch: This branch is a specially designed lightweight convolutional network for explicitly capturing the physical patterns of radar maps. It contains two parallel sub-branches:  Doppler Feature Extraction Sub-branch: Multiple N×1 convolutional kernels are used to perform convolution operations along the frequency axis of the time-frequency map to capture the Doppler features of the target; for the input time-frequency map... The Doppler feature map output by this branch is: ； in It is the output feature map of the Doppler feature extraction branch. and This indicates the weights and biases of the convolution kernel in that branch. Indicates the activation function;  Temporal feature extraction sub-branch: Multiple 1×M convolution kernels are used to perform convolution operations along the time axis of the time-frequency map to capture the temporal dynamic features caused by target attitude changes, wing flapping, and other behaviors; Similarly, the temporal dynamic feature map extracted along the time direction can be represented as: ； in It is the output feature map of the temporal feature extraction branch. and This indicates the weights and biases of the convolution kernel in that branch. Indicates the activation function; The feature maps extracted from the two sub-branches are concatenated along the channel dimension and fused using a standard 3×3 convolution. The fused feature map is then subjected to global average pooling to obtain radar-specific feature vectors. ； Where dr represents radar characteristics The dimension, [;] indicates the concatenation operation. It is a global average pooling function used to compress feature maps into vectors; Finally, the universal feature vector at each scale With radar-specific feature vectors By combining the features, a composite characteristic is formed: ； Where dc+dr represents the comprehensive feature Dimensions.

5. The radar insect and bird classification method based on multi-scale feature fusion as described in claim 1, characterized in that, In step three, based on attention fusion, learnable dynamic scale weights are introduced. And transform them into normalized weights using Softmax. Assign it to the new feature Explicit weighted summation is performed to obtain the final global fusion feature vector. : 。 6. The radar insect and bird classification method based on multi-scale feature fusion as described in claim 1, characterized in that, In step four, the global feature vector Two parallel classification heads are fed in: Main classification head: A multilayer perceptron (MLP) consisting of one or more fully connected layers, used to perform the core three-class classification task, outputting a class vector indicating whether the target belongs to insects, birds or clutter, which is used for subsequent probability normalization and determination; Auxiliary classification head: Consists of a fully connected layer, used to perform an auxiliary binary classification task, outputting a class vector indicating whether the target belongs to a living organism (insect, bird) or a non-living organism (clutter), which is used for subsequent probability normalization and determination.

Images