Radio frequency interference multi-label classification method based on time-frequency cross attention fusion network

Abstract

The present application relates to the technical field of radio astronomy signal processing and deep learning, and particularly relates to a radio frequency interference multi-label classification method based on time-frequency cross attention fusion network, comprising: converting dynamic spectrum data output by a radio telescope into an image tensor; processing the image tensor into a re-calibration feature map; performing bidirectional cross attention fusion, residual connection, layer normalization and global pooling processing on the time domain feature and the frequency domain feature based on the re-calibration feature map, so as to obtain time-frequency fusion features; obtaining multi-scale features based on intermediate feature maps of multiple different scales of intermediate layers in the backbone feature extraction network, and performing global pooling on the re-calibration feature map to obtain a backbone bypass feature; obtaining prediction probabilities of each radio frequency interference category and multi-label categories based on the time-frequency fusion features, the multi-scale features and the backbone bypass feature; and the present application can improve the performance of radio frequency interference multi-label classification.

Description

Technical Field

[0001] This invention relates to the fields of radio astronomy signal processing and deep learning technology, specifically to a multi-label classification method for radio frequency interference based on a time-frequency cross-attention fusion network. Background Technology

[0002] Radio astronomy observations rely on the precise capture of faint radio signals from the universe; however, increasingly severe radio frequency interference (RFI) has become a core bottleneck restricting scientific output. With the construction and operation of new-generation large-aperture radio telescopes (such as FAST, SKA, and QTT), observational sensitivity has significantly improved, posing unprecedented challenges to RFI detection and classification. Unlike traditional single-label classification, in actual observations, a single time-spectrum image often contains multiple types of interference signals simultaneously, such as narrowband interference, broadband pulses, and radar signals. This presents a dual challenge to the semantic understanding and multi-label recognition capabilities of classification models.

[0003] Traditional RFI detection methods (such as SumThreshold, SVD morphological separation, etc.) rely on manual parameter tuning and have poor generalization ability in complex mixed RFI scenarios. In recent years, RFI classification methods based on deep learning mainly adopt the convolutional neural network (CNN) architecture. Although some progress has been made, the following shortcomings exist: (1) Most of them model RFI recognition as a single-label classification task, ignoring the universality of multiple interference sources coexisting in real scenarios; (2) They lack effective modeling of joint features in the time and frequency domains, making it difficult to capture long-range dependencies across time and frequency dimensions; (3) The feature extraction path is single and the multi-scale feature fusion is insufficient, resulting in limited recognition accuracy for complex superimposed interference.

[0004] Therefore, there is an urgent need in this field for a method that can fully exploit time-frequency joint features and dynamically fuse multi-scale information to achieve high-precision and robust multi-label classification and recognition of various RFI signals. Summary of the Invention

[0005] In view of the above problems, the present invention provides a radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network, which solves the technical problems of low accuracy and poor generalization ability of RFI multi-label classification caused by single feature extraction path and insufficient time-frequency joint feature modeling in the prior art.

[0006] This invention provides a multi-label classification method for radio frequency interference based on a time-frequency cross-attention fusion network, comprising the following steps: Step S1: Obtain the dynamic spectrum data output by the radio telescope, and obtain the image tensor through size adjustment, pixel value normalization, channel copying and batch organization; Step S2: Input the image tensor into the backbone feature extraction network to obtain a high-dimensional feature map; The high-dimensional feature map is subjected to global adaptive average pooling, bottleneck structure processing, and channel-by-channel multiplication in sequence to obtain a recalibrated feature map. Step S3: The recalibrated feature map is processed by feature projection and position encoding to obtain the position-encoded feature map; The feature maps after position encoding are aggregated along the frequency dimension and along the time dimension to obtain time domain features and frequency domain features. Using the time-domain features and the frequency-domain features as mutual queries and keys, bidirectional cross-attention fusion, residual connection, layer normalization, and global pooling are performed to obtain time-frequency fusion features. Step S4: Based on the intermediate feature maps of multiple intermediate layers of different scales in the backbone feature extraction network, multi-scale features are obtained, and the recalibrated feature maps are globally pooled to obtain backbone bypass features. Step S5: Perform gated weighted fusion of the time-frequency fusion features, multi-scale features, and backbone bypass features to obtain gated fusion features; The gated fusion features are used to obtain the predicted probability of each radio frequency interference category through fully connected mapping and activation function, and the radio frequency interference multi-label category is obtained based on the predicted probability.

[0007] Preferably, in step S1, the dynamic spectrum data is a time-frequency two-dimensional image, with the horizontal axis representing the time dimension and the vertical axis representing the frequency dimension, and the brightness value of each pixel representing the signal strength at the corresponding time and frequency.

[0008] Preferably, in step S2, the backbone feature extraction network is an EfficientNet-B3 network; In step S2, the expression for the global adaptive average pooling process is:

[0009] in, Indicates the channel descriptor. and These represent the height and width of the input feature map, respectively. This represents summing over all positions in the spatial dimension. Indicates the spatial location of the input feature map The characteristic values ​​of all channels, Represents the real number field. Indicates batch size, This indicates the number of channels in the output feature map; The expression for bottleneck structure processing is:

[0010] in, Represents the channel weight vector. This represents the Sigmoid activation function. This represents a 1×1 convolution operation. This represents the modified linear unit activation function; The expression for the channel-by-channel multiplication process is:

[0011] in, This indicates the recalibrated feature map. This indicates element-wise multiplication.

[0012] Preferably, in step S3, the expression for processing the recalibrated feature map through feature projection and position encoding is:

[0013] in, This represents the projected feature map. This indicates a batch normalization operation. The feature dimension for time-frequency cross-attention processing;

[0014] in, This represents the feature map after position encoding. This indicates a two-dimensional position embedding.

[0015] Preferably, in step S3, the steps of frequency-dimension aggregation and time-dimension aggregation of the position-encoded feature maps specifically include time-domain feature construction and frequency-domain feature construction, with the following expressions:

[0016] in, Representing time-domain features, Presentation layer normalization operation, Indicates that the input dimension is Output dimension is Linear projection layer, This represents the feature map after reshaping along the frequency dimension;

[0017] in, Represents frequency domain characteristics, This represents the feature map after being reshaped along the time dimension.

[0018] Preferably, in step S3, the steps of performing bidirectional cross-attention fusion, residual connection, layer normalization, and global pooling processing using the time-domain features and the frequency-domain features as query and key values ​​respectively specifically include: In the time-to-frequency direction, using time-domain features as queries and frequency-domain features as keys and values, a query-key-value matrix is ​​generated through linear projection. After reshaping into a multi-head form, the scaled dot product attention is calculated, expressed as: , ,

[0019]

[0020] in, This represents the cross-attention output in the time-to-frequency direction. For time-domain query matrix, For frequency domain key matrix, It is a frequency-domain matrix. Let be the projection matrix. express transpose, Scaling factor This represents the normalized exponential function; In the frequency-to-time direction, using frequency domain features as the query and time domain features as the key and value, cross-attention calculation is performed, expressed as: , ,

[0021]

[0022] in, This represents the cross-attention output from frequency to time. For frequency domain lookup matrix, The time-domain key matrix, For time-domain matrices, The projection matrix; and After the output is processed through multi-head splicing and linear output projection, residual connection and layer normalization are performed. The expression is as follows:

[0023] in, This indicates the features after fusion in the time direction. This indicates the features after frequency direction fusion. and These are the attention outputs from two directions after multi-head splicing and linear projection, respectively; Will and A joint time-frequency grid is formed by dimensional expansion, and then the channel dimensions are merged by linear projection until a time-frequency fusion feature is obtained after global average pooling. The expression is as follows:

[0024] in, This represents the time-frequency fusion feature vector. This indicates a global average pooling operation. Indicates that the input dimension is Output dimension is Linear projection layer, and These are the temporal fusion features and frequency fusion features after dimensional expansion, respectively. This indicates a splicing operation along the channel dimension.

[0025] Preferably, in step S4, the expression for obtaining multi-scale features based on intermediate feature maps of multiple intermediate layers at different scales in the backbone feature extraction network is:

[0026] in, Indicates the first The result of projecting the feature map in the middle of the layer. Indicates the first Intermediate layer feature map The spatial dimensions of the registered location at the corresponding scale;

[0027] in, Indicates the first sampled number after upsampling. Layer feature map, This indicates the bilinear interpolation upsampling operation. Indicates target resolution ;

[0028] in, Multi-scale features are represented. This represents the concatenation of three upsampled feature maps along the channel dimension; In step S4, the expression for the backbone bypass characteristics is:

[0029] in, Represents the backbone bypass feature vector. Indicates the flattening operation. This indicates a global adaptive average pooling operation. Indicates that the input dimension is Output dimension is The linear projection layer.

[0030] Preferably, in step S5, the expression for the gated fusion feature is:

[0031]

[0032] in, Represents the gated fusion feature vector. For the gated vector, As a time-frequency fusion feature, Characterized by a backbone bypass. Indicates that the input dimension is Output dimension is The first linear transformation, Indicates that the input dimension is Output dimension is The second linear transformation; In step S5, the expression for the predicted probability is:

[0033]

[0034]

[0035]

[0036] in, This represents the expanded feature vector. Indicates that the input dimension is Output dimension is The fully connected layer This represents the feature vector after activation and regularization. This indicates a Dropout operation with a drop rate of 0.3. Represents the original predicted score vector for each category; Indicates the first The predicted probabilities of each category, Indicates the first The original predicted scores for each category, It is a natural constant. This represents the total number of radio frequency interference categories. The expression for obtaining the radio frequency interference multi-tag category based on the predicted probability is:

[0037] in, Indicates the first The final judgment results for each category.

[0038] Preferably, it further includes: a step of training the time-frequency cross-attention fusion network, including: determining the loss function and training parameters, using the AdamW optimizer and cosine annealing strategy to perform error backpropagation, and obtaining a trained deep neural network; The expression for the loss function is as follows:

[0039] in, This indicates that when the loss function, This represents the true label of the k-th class, where K represents the number of classes. Represents the natural logarithm. Indicates the predicted label; The learning rate is adjusted according to a cosine annealing strategy during each training cycle, expressed as:

[0040] in, This represents the learning rate corresponding to the t-th training period. This represents the preset minimum learning rate. This represents the initial learning rate, and t represents the current training cycle number. denoted by , where represents the total number of training cycles, and cos represents the cosine function.

[0041] Compared with the prior art, the present invention has at least the following beneficial effects: (1) Time-frequency joint feature modeling: This invention proposes the TFCA time-frequency cross attention module, which explicitly models the joint dependency between the time dimension and the frequency dimension in the RFI time spectrum through the bidirectional cross attention mechanism in the two directions of time → frequency and frequency → time, effectively capturing complex RFI features across different time scales and frequency ranges.

[0042] (2) Parallel multi-branch feature fusion: The present invention constructs a parallel multi-branch feature fusion architecture. The main path extracts time-frequency joint features through the serial processing of SE→projection→TFCA. The parallel MSFF branch and backbone bypass extract multi-scale hierarchical features and global features respectively. The three features are dynamically fused through a learnable gating mechanism, complementing the feature representations at different levels.

[0043] (3) Multi-label classification modeling: This invention models RFI identification as a multi-label classification task, which supports the simultaneous identification of multiple interference types in a single time spectrum image, and is more in line with the needs of multiple interference sources coexisting in actual radio astronomy observations. Attached Figure Description

[0044] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0045] Figure 1 The flowchart shows the radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network provided by the present invention.

[0046] Figure 2 This is a detailed flowchart of the radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network of the present invention. Detailed Implementation

[0047] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0048] This invention provides a multi-label classification method for radio frequency interference based on a time-frequency cross-attention fusion network. It takes dynamic spectrum data output by a radio telescope as input, and performs data preprocessing, backbone feature extraction and channel attention recalibration, time-frequency cross-attention fusion, multi-scale feature extraction and gated weighted fusion. Finally, the predicted probability of each radio frequency interference category is output through the classification head to obtain the multi-label classification result.

[0049] To illustrate the effectiveness of the method proposed in this invention, the following detailed description of the above technical solution is provided through a specific embodiment. This embodiment uses pulsar dynamic spectrum data observed by a radio telescope at an astronomical observatory as an example. Figure 1 As shown, a multi-label classification method for radio frequency interference based on a time-frequency cross-attention fusion network is disclosed. The specific implementation steps are as follows: Step S1: Obtain the dynamic spectrum data output by the radio telescope, and obtain the image tensor through size adjustment, pixel value normalization, channel copying and batch organization. In this step, the present invention processes the dynamic spectrum data to obtain an image tensor for backbone feature extraction.

[0050] Specifically, the dynamic spectrum data output by the radio telescope receiver is a two-dimensional time-frequency image, with the horizontal axis representing the time dimension and the vertical axis representing the frequency dimension. The brightness value of each pixel represents the signal strength at the corresponding time and frequency.

[0051] (1) Image size standardization After obtaining the original dynamic spectrum data, the time-frequency two-dimensional image is first adjusted to a uniform 224×224 pixel resolution using a bilinear interpolation algorithm. Bilinear interpolation performs linear interpolation in both the horizontal and vertical directions, enabling scaling transformations of arbitrary sizes while maintaining image smoothness and avoiding the jagged effect caused by nearest-neighbor interpolation.

[0052] (2) Pixel value normalization The resized image is then normalized to map pixel values ​​from the original range of [0, 255] to the interval [0, 1]. The normalization formula is as follows:

[0053] in, This represents the normalized pixel value. This represents the original pixel value before normalization, and 255.0 represents the maximum pixel value of the 8-bit grayscale image.

[0054] Normalization eliminates the absolute dimensional differences in the original signal intensity, giving dynamic spectrum data acquired under different observation conditions a uniform numerical range.

[0055] (3) Channel copying and format conversion A single-channel grayscale image is converted into a three-channel RGB image through channel duplication. Specifically, a single-channel image of size [1, 224, 224] is duplicated three times along the channel dimension to obtain a three-channel image of size [3, 224, 224]. The purpose of this operation is to make single-channel astronomical observation data compatible with a backbone feature extraction network pre-trained on the ImageNet dataset, which expects a three-channel image as input.

[0056] (4) Batch organization Finally, the processed three-channel images are organized in batches to obtain images with dimensions of [dimension value missing]. The image tensor, where In this embodiment, the batch size is set to 32. Represents the real number field. Batch organization enables the subsequent backbone network to utilize the parallel computing power of GPUs to process multiple time-spectrum graphs simultaneously, improving training and inference efficiency.

[0057] Through this step, the raw radio astronomy dynamic spectrum data is converted into standardized image tensors with unified format and normalized values ​​after being uniform in size, normalized in value, adapted in channels, and organized in batches.

[0058] Step S2: Input the image tensor into the backbone feature extraction network to obtain a high-dimensional feature map; The high-dimensional feature map is subjected to global adaptive average pooling, bottleneck structure processing, and channel-by-channel multiplication in sequence to obtain a recalibrated feature map. This step involves multi-step feature extraction of the image tensor to obtain a recalibrated feature map.

[0059] The backbone feature extraction network of this invention adopts the EfficientNet-B3 network. The EfficientNet-B3 network employs a composite scaling strategy, uniformly scaling the network width, depth, and input resolution to achieve a balance between accuracy and computational efficiency. The basic building block of EfficientNet-B3 is the MBConv convolutional block, which internally includes depthwise separable convolutions, a Squeeze-and-Excitation optimization mechanism, and a Swish activation function. The backbone network is pre-trained on the ImageNet-1K dataset, initialized using pre-trained weights from the IMAGENET1K_V1 version, giving the model good general visual feature extraction capabilities.

[0060] Image tensor The features submodule of the input backbone network is used for forward propagation computation. The features submodule consists of a series of MBConv convolutional blocks stacked sequentially. The input image is processed by each MBConv block in turn. At each downsampling stage, the spatial resolution is halved while the number of channels increases. For a 224×224 input, after 32x downsampling, the backbone network outputs a high-dimensional feature map. ,in, This represents a high-dimensional feature map output by the backbone network; Indicates batch size, Indicates the height of the output feature map. This indicates the width of the output feature map. This indicates the number of channels in the output feature map.

[0061] In this embodiment, , , That is, the backbone network will input Image compression to A high-dimensional feature map, where each spatial location corresponds to a position in the input image. The sensory field area.

[0062] Obtaining high-dimensional feature maps Subsequently, this invention performs a global adaptive average pooling operation, independently compressing the feature map into a single scalar value for each channel, and aggregating global spatial information, expressed as:

[0063] Among them, among them, This represents the channel descriptor, where each element represents the global response strength of the corresponding channel. and These represent the height and width of the feature map, respectively. This represents summing over all positions in the spatial dimension. Representing the spatial location of the feature map The characteristic values ​​of all channels, Indicates batch size, This indicates the number of channels in the output feature map.

[0064] Through adaptive average pooling, the complete spatial dimensions ( Compress into a single value to generate a channel descriptor. It encodes global statistics for each channel across the entire feature map.

[0065] Compressed channel descriptor By addressing the bottleneck structure, the nonlinear dependencies between channels are learned, expressed as follows:

[0066] in, Represents the channel weight vector. This represents the Sigmoid activation function. This represents a 1×1 convolution operation. This represents the modified linear unit activation function.

[0067] The bottleneck structure described above first reduces the number of channels from Compress to Compression ratio Then, the original number of channels is restored through a second 1×1 convolution. This structure reduces the number of parameters while enhancing the model's nonlinear expressiveness, enabling channel weights to reflect the complex dependencies between channels.

[0068] The learned channel weights are applied to the high-dimensional feature map through channel-wise multiplication to obtain the channel-enhanced recalibrated feature map, expressed as:

[0069] in, This indicates the recalibrated feature map. This indicates element-wise multiplication.

[0070] By using channel-wise multiplication, the feature values ​​of each channel are scaled according to their importance. Channels with high information content are enhanced, while channels with low relevance are suppressed, thus achieving adaptive recalibration of backbone features.

[0071] Step S3: The recalibrated feature map is processed by feature projection and position encoding to obtain the position-encoded feature map; The feature maps after position encoding are aggregated along the frequency dimension and along the time dimension to obtain time domain features and frequency domain features. Using the time-domain features and the frequency-domain features as mutual queries and keys, bidirectional cross-attention fusion, residual connection, layer normalization, and global pooling are performed to obtain time-frequency fusion features. In this step, the recalibrated high-dimensional features are mapped to a low-dimensional space, spatial location information is injected, and time-frequency cross-attention calculation is performed to obtain time-frequency fusion features.

[0072] (1) Feature projection and position encoding processing First, channel dimensionality reduction will be performed using 1×1 convolution, reducing the number of channels from... Down to 1×1 convolution, while maintaining spatial resolution, achieves cross-channel information exchange and dimensionality compression through linear transformation. The expression for batch normalization and ReLU activation function following the projection operation is:

[0073] in, This represents the projected feature map. This indicates a batch normalization operation. This represents the feature dimension for time-frequency cross-attention processing.

[0074] In some embodiments, the present invention employs batch normalization to normalize the mini-batch of each channel, accelerating training convergence and improving model stability. The convolutional kernel weights of the feature projection are initialized using a Kaiming normal distribution, and the bias is initialized to 0.

[0075] Next, a learnable 2D location embedding is added to the projected feature map. The shape of the learnable 2D location embedding has the same spatial dimension as the projected feature map. The initial values ​​are randomly sampled from a normal distribution with a mean of 0 and a standard deviation of 0.02, and the expression is:

[0076] in, This represents the feature map after position encoding. This represents a learnable two-dimensional position embedding.

[0077] The introduction of positional encoding enables the network to perceive feature maps in the original... Location information within the spatial grid is crucial for the time-frequency cross-attention module to correctly understand the correspondence between the time dimension (rows) and the frequency dimension (columns).

[0078] In the subsequent steps, this invention implements bidirectional cross-attention calculation in the time and frequency dimensions, and explicitly models the time-frequency joint feature dependency relationship, as described in detail below.

[0079] (2) Time-frequency feature separation and aggregation The location-encoded feature map is reshaped into time-domain and frequency-domain features along complementary dimensions.

[0080] The steps for constructing time-domain features include: expanding the feature map along the frequency dimension, with 7 frequency points in each row, and reshaping it into... Then, it is compressed to a linear projection layer and layer normalization. The expression is:

[0081] in, Representing time-domain features, Presentation layer normalization operation, Indicates that the input dimension is Output dimension is Linear projection layer, This represents the feature map after reshaping along the frequency dimension.

[0082] From this, we obtain Each row corresponds to the aggregated representation of a time step across the entire frequency range.

[0083] The steps for constructing frequency domain features include: The feature map is expanded along the time dimension, with 7 time points in each column, and reshaped into... Then, it is compressed to a linear projection layer and layer normalization. :

[0084] in, Represents frequency domain characteristics, This represents the feature map after being reshaped along the time dimension.

[0085] From this, we obtain Each row corresponds to the aggregated representation of a frequency channel across the entire time range.

[0086] (3) Two-way cross-attention fusion Next, time-domain features and frequency-domain features are used as queries and keys respectively for bidirectional cross-attention fusion. In the time-to-frequency direction, time-domain features are used as queries, and frequency-domain features as keys and values. A query-key-value matrix is ​​generated through linear projection, reshaped into a multi-head form, and then scaled dot-product attention is calculated. The expression is: , ,

[0087]

[0088] in, This represents the cross-attention output in the time-to-frequency direction. For time-domain query matrix, For frequency domain key matrix, It is a frequency-domain matrix. The projection matrix is ​​learnable. express transpose, Scaling factor This represents the normalized exponential function.

[0089] Symmetrically representing the frequency domain features along the frequency-to-time direction. As a query, time-domain features As both keys and values, the same cross-attention computation is performed: , ,

[0090]

[0091] in, This represents the cross-attention output from frequency to time. For frequency domain lookup matrix, The time-domain key matrix, For time-domain matrices, This is a learnable projection matrix. This calculation enables each frequency channel to aggregate information from all time steps, capturing frequency patterns that vary along the time axis.

[0092] In this embodiment, the number of attention heads , .

[0093] The cross-attention outputs from both directions are processed through multi-head splicing and linear output projection, followed by residual connection and layer normalization, as shown in the expression:

[0094] in, This indicates the features after fusion in the time direction. This indicates the features after frequency direction fusion. and These represent the attention outputs from two directions after multi-head splicing and linear projection, respectively. Presentation layer normalization operation.

[0095] Through the above processing, residual connections enable gradients to be directly propagated back to the input layer, alleviating the gradient vanishing problem in deep networks; layer normalization independently normalizes each sample in the feature dimension, stabilizing the feature distribution during training.

[0096] (4) Global pooling processing The two fusion representations and A joint time-frequency grid is formed by dimensional expansion. Specifically, [the grid will be constructed]. Replication along the frequency dimension extends to ,Will Replication along the time dimension extends to The two are spliced ​​together to obtain The joint representation. Then, the channel dimensions are merged through linear projection to... The time-frequency fusion features are obtained by global average pooling, and the expression is:

[0097] in, This represents the time-frequency fusion feature vector. This indicates a global average pooling operation. Indicates that the input dimension is Output dimension is Linear projection layer, and These are the temporal fusion features and frequency fusion features after dimensional expansion, respectively. This indicates a splicing operation along the channel dimension.

[0098] Step S4: Based on the intermediate feature maps of multiple intermediate layers of different scales in the backbone feature extraction network, multi-scale features are obtained, and the recalibrated feature maps are globally pooled to obtain backbone bypass features. In this step, multi-scale intermediate layer feature extraction and fusion are performed first.

[0099] By registering forward propagation hooks in the feature sequence of the backbone network, intermediate layer feature maps are captured at specified indices without modifying the forward propagation logic of the backbone network. For EfficientNet-B3 with a 224×224 input, the registration locations include: Stage 2 has a spatial size of 56×56, 48 channels, and a downsampling factor of 4. This layer retains fine-grained details of radio frequency interference signals, such as the fine frequency structure of narrowband interference. Stage4 corresponds to a spatial size of 28×28, 136 channels, and a downsampling factor of 8. This layer extracts medium-scale radio frequency interference morphological features, such as the timing pattern of pulse interference. Stage 6 has a spatial size of 14×14, 232 channels, and a downsampling rate of 16. This layer captures large-scale RF interference burst structures, such as the spectral envelope of broadband interference.

[0100] For each intermediate layer feature map ( First, it is projected onto a uniform channel dimension of 64 through a 1×1 convolution, and then batch normalized and ReLU activated. The expression is as follows:

[0101] in, Indicates the first The result of projecting the feature map in the middle of the layer. The spatial dimensions of the corresponding scale registration locations are 56, 28, and 14, respectively. 1×1 convolutions achieve cross-channel information exchange and dimensionality compression while maintaining spatial resolution.

[0102] Then, the projected feature maps of each layer are upsampled to a uniform target resolution using bilinear interpolation. The spatial dimensions are consistent with those of the backbone output feature map, and the expression is:

[0103] in, Indicates the first sampled number after upsampling. Layer feature map, This indicates a bilinear interpolation upsampling operation.

[0104] The three upsampled feature maps are concatenated along the channel dimension to obtain... The multi-scale representation is then fused into a unified dimension through 1×1 convolution, batch normalization, and ReLU activation:

[0105] in, This represents the fused multi-scale feature map. This represents the concatenation of three upsampled feature maps along the channel dimension.

[0106] Finally, through global adaptive average pooling and linear projection, the multi-scale feature vector is obtained:

[0107] in, Represents multi-scale feature vectors. This indicates a global adaptive average pooling operation. This indicates that the input dimension is 256 and the output dimension is... Linear projection layer, Consistent with the feature dimensions of time-frequency fusion.

[0108] Next, backbone bypass feature extraction is performed, and global adaptive average pooling is applied to the recalibrated feature map to compress the spatial dimension. Then expand and project linearly to Dimension, expressed as:

[0109] in, Represents the backbone bypass feature vector. Indicates the flattening operation. This indicates a global adaptive average pooling operation. Indicates that the input dimension is Output dimension is The linear projection layer.

[0110] Through the above steps, the global semantic features extracted by the backbone network are preserved, serving as a supplementary information source for the features after time-frequency cross-attention processing, thus preventing feature degradation that may be caused by attention calculations.

[0111] Step S5: Perform gated weighted fusion of the time-frequency fusion features, multi-scale features, and backbone bypass features to obtain gated fusion features; The gated fusion features are used to obtain the predicted probability of each radio frequency interference category through fully connected mapping and activation function, and the radio frequency interference multi-label category is obtained based on the predicted probability.

[0112] In this step, gated weighted fusion processing was performed first.

[0113] The gate vector is obtained by concatenating time-frequency fusion features and multi-scale features, followed by two layers of MLP and sigmoid activation. First, the time-frequency fusion features and multi-scale features are concatenated along the feature dimension:

[0114] in, This represents the concatenated joint feature vector. This indicates a concatenation operation along the feature dimension. .

[0115] Then, the gating vector is calculated using a gating network with a bottleneck structure. The first-layer linear transformation will... Dimensional compression to After ReLU activation, the second-level linear transformation restores the dimension to . The dimension is finally limited to a certain value using the Sigmoid function. Interval:

[0116] in, Represents the gate vector, This represents the Sigmoid activation function. Indicates that the input dimension is Output dimension is The first linear transformation, Indicates that the input dimension is Output dimension is The second linear transformation. Gated vector. Each element represents the contribution weight of the time-frequency fusion feature on the corresponding feature dimension.

[0117] Finally, the fused features are obtained through gated interpolation, expressed as:

[0118] in, Represents the gated fusion feature vector. For the gated vector, As a time-frequency fusion feature, Characterized by a backbone bypass. These are the complementary weights of the gate vector. When the gate vector... When a certain dimension of the feature is close to 1, that dimension's feature is mainly contributed by the time-frequency fusion feature; when When the value approaches 0, this dimension of the feature is mainly contributed by an equally weighted combination of backbone bypass features and multi-scale features. This gated fusion mechanism allows the model to adaptively adjust the contribution ratio of each feature based on the features of the input spectrogram, achieving dynamic feature fusion at the sample level.

[0119] After obtaining the gated fusion features, these features are input into the classification head network. The classification head consists of two fully connected layers. The first fully connected layer divides the feature dimension from... Extended to :

[0120] in, This represents the expanded feature vector. Indicates that the input dimension is Output dimension is The fully connected layer has weights that are normally distributed. Initialize the biases to 0. Apply the ReLU activation function to the expanded features to introduce a non-linear transformation, then use a Dropout regularization layer to randomly drop 30% of the neurons to prevent overfitting.

[0121] in, This represents the feature vector after activation and regularization. This indicates a Dropout operation with a drop rate of 0.3.

[0122] The second fully connected layer reduces the feature dimension from Mapping to the number of categories Generate the original predicted score for each category:

[0123] in, This represents the original predicted score vector for each category. This represents the number of radio frequency interference categories, corresponding to narrowband interference, pulse interference, frequency modulation interference, broadband interference, frequency sweep interference, burst interference, and mixed interference, respectively.

[0124] The original predicted scores for each category Independently mapped to via the Sigmoid activation function Intervals are used to obtain independent predicted probabilities for each category:

[0125] in, Indicates the first The predicted probabilities of each category, Indicates the first The original predicted scores for each category, It is a natural constant. This represents the total number of radio frequency interference categories. The Sigmoid activation function independently maps the predicted score for each category to a probability value, ensuring that the categories do not affect each other, making it suitable for multi-label classification scenarios.

[0126] During the reasoning phase, a threshold decision method is used to determine the existence or non-existence of each category:

[0127] in, Indicates the first The final decision result for each category is 1, indicating the presence of radio frequency interference (RFI) in that category; 0 indicates its absence; and 0.5 is the decision threshold. This thresholding method allows each sample to predict any number of labels (from 0 to all). (Multiple categories) enable the model to flexibly cope with complex scenarios in actual observations where a single time-spectrum map contains multiple radio frequency interference signals, and output radio frequency interference multi-label categories.

[0128] In this step, the three complementary features are gated and weighted to form an adaptive comprehensive feature representation. This representation is then processed by a nonlinear mapping and Sigmoid probability transformation of the classification head, ultimately outputting the independent predicted probabilities of various radio frequency interferences and the multi-label classification results. This method achieves high-precision simultaneous identification and classification of multiple radio frequency interference signals in dynamic radio astronomy data, supporting parallel detection of multiple interference types in a single time-spectrum image.

[0129] This invention also provides steps for training a time-frequency cross-attention fusion network, including: determining the loss function and training parameters, using the AdamW optimizer and cosine annealing strategy for error backpropagation, and obtaining a trained deep neural network, as described in detail below.

[0130] The dynamic spectral image with multi-label annotation is input into the classification network consisting of steps S1 to S5 for iterative training. The loss between the prediction result and the real label is calculated based on the multi-label binary cross-entropy loss function. The network parameters are updated using the AdamW optimizer and cosine annealing learning rate scheduling strategy to obtain the trained radio frequency interference multi-label classification model.

[0131] In this step, the 7-dimensional binary label corresponding to each dynamic spectrum image is used as a supervision signal. The k-th label has a value of 1, indicating that the image contains the k-th type of radio frequency interference, and a value of 0, indicating that it does not contain the k-th type of radio frequency interference. During training, the input image is subjected to random horizontal flipping, random rotation, and Gaussian noise injection for data augmentation. Then, training batches are formed according to a batch size of 32. The augmented image tensor and the corresponding multi-label tensor are input into the network described in steps S1 to S5, so that the network can simultaneously learn time-frequency fusion features, multi-scale features, backbone bypass features, and classification head parameters.

[0132] (1) Training data augmentation and batch organization.

[0133] A training batch is read from a dynamic spectral image with multi-label annotations. For each image within the batch, random horizontal flipping, random rotation, and Gaussian noise injection are performed independently. Random horizontal flipping is performed with a 50% probability; the random rotation angle is randomly selected from -10° to 10°; the standard deviation of the Gaussian noise is selected from 0 to 15 to simulate noise disturbances that may occur during radio telescope observations. Data augmentation is performed only during the training phase. During the validation and testing phases, the above random transformations are not performed; instead, the images are processed directly according to the resizing and normalization methods described in step S1.

[0134] (2) Calculation of multi-label loss.

[0135] Multi-label classification training is performed using the binary cross-entropy loss function (BCEWithLogitsLoss). The loss function calculates the binary cross-entropy loss independently for each class, and then averages it over all classes. The expression for the loss function is:

[0136] in, This indicates that when the loss function, This represents the true label of the k-th class, where K represents the number of classes. Represents the natural logarithm. This indicates the predicted label.

[0137] In actual calculations, the BCEWithLogitsLoss form, which combines the Sigmoid operation with the binary cross-entropy operation, is used to directly calculate the loss using the original prediction score z, thereby reducing the numerical instability caused by calculating the probability separately when the prediction score is large or small.

[0138] (3) Optimizer and learning rate scheduling configuration.

[0139] The AdamW optimizer is used to update the gradients of the trainable parameters in the network based on the current batch loss L_BCE. The initial learning rate of AdamW is set to η0 = 1 × 10^(-3), the weight decay coefficient is set to λ = 1 × 10^(-4), and the numerical stability constant is set to... =1×10^(-8).

[0140] The learning rate is adjusted according to a cosine annealing strategy during each training cycle, expressed as:

[0141] in, This represents the learning rate corresponding to the t-th training period. This represents the preset minimum learning rate. This represents the initial learning rate, and t represents the current training cycle number. =15 represents the total number of training epochs, and cos represents the cosine function. The cosine annealing strategy allows the learning rate to decrease smoothly during training, which is beneficial for the stable convergence of network parameters.

[0142] (4) Iterative training and model saving.

[0143] The total number of training epochs is set to 15. At the beginning of each training epoch, the network is set to training mode, and Dropout and batch normalization layers are enabled. Then, the training set is traversed batch by batch, and for each batch, forward propagation, loss calculation, gradient backpropagation, gradient zeroing, and AdamW parameter updates are performed sequentially until the current training set has been traversed. After each training epoch, the network is switched to evaluation mode, and the validation loss and multi-label classification evaluation metric are calculated on the validation set according to steps S1 to S5. The optimal model parameters are saved as model checkpoints based on the macro-average F1 score of the validation set.

[0144] After the above iterative training, the network parameters can jointly optimize the backbone feature extraction, SE channel recalibration, time-frequency cross-attention fusion, multi-scale feature fusion, gated weighted fusion, and classification head mapping processes under the action of multi-label supervision signals, making the output prediction probabilities of each category closer to the real multi-label corresponding to the dynamic spectrum image. After training, the model checkpoint with the best performance on the validation set is loaded, the dynamic spectrum image to be detected is preprocessed according to step S1, and the prediction probabilities of each category are obtained through inference in steps S2 to S5. Then, the radio frequency interference multi-label classification results are output according to the threshold decision rule in step S5-2.

[0145] To verify the effectiveness of this invention, a dataset was constructed using simulated observation data from a radio telescope at a certain observatory. The dataset contains 10,000 time-spectral images, each a 224×224 pixel RGB image, labeled with binary labels for 7 types of Radio Frequency Irregularities (RFIs). The dataset is divided into a training set (7,000 images), a validation set (1,500 images), and a test set (1,500 images) in a 7:1.5:1.5 ratio. The dataset exhibits high multi-label co-occurrence characteristics, with an average of 4.2 labels per sample, each sample containing at least one RFI type, and most samples simultaneously containing multiple RFI categories.

[0146] The proposed time-frequency cross-attention fusion network model was evaluated using metrics such as ExactMatch (EM), Hamming loss, macro-average F1 score, micro-average F1 score, macro-average AUC-ROC, and average number of predicted labels. The model was also compared with seven baseline models: CNN baseline, ResNet18, ResNet34, ResNet50, ResNet101, MobileNetV2, and EfficientNet-B0. The specific results are as follows: Table 1

[0147] Table 2

[0148] As shown in Table 2, the method of this invention achieved optimal performance on the independent test set, with an exact matching accuracy of 97.27%, a macro-average F1 score of 0.9966, and a Hamming loss of only 0.0041, significantly outperforming all baseline models. Compared to the best-performing lightweight baseline model MobileNetV2 (F1: 0.953), the macro-average F1 score of the method of this invention is improved by 4.6 percentage points. Furthermore, the number of predicted labels (4.18) and the number of true labels (4.20) of the method of this invention are highly consistent, indicating that the model has good multi-label distribution prediction capabilities.

[0149] In addition, to verify the effectiveness of each module, this invention also conducted ablation experiments, testing the independent contribution of each module by gradually adding TFCA, SE, and MSFF modules. The results are shown in the table below: Table 3

[0150] The ablation experiment results show that adding the SE module (B3+TFCA+SE) improves the performance of the validation set, and adding the MSFF module (B3+TFCA+MSFF) improves the performance of the test set. The complete model (B3+TFCA+SE+MSFF) achieves the best accuracy in exact matching, realizing the best overall performance balance and verifying the complementarity and necessity of each module.

[0151] All the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of the present invention, and will not be described in detail here.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A multi-label classification method for radio frequency interference based on a time-frequency cross-attention fusion network, characterized in that, Includes the following steps: Step S1: Obtain the dynamic spectrum data output by the radio telescope, and obtain the image tensor through size adjustment, pixel value normalization, channel copying and batch organization; Step S2: Input the image tensor into the backbone feature extraction network to obtain a high-dimensional feature map; The high-dimensional feature map is subjected to global adaptive average pooling, bottleneck structure processing, and channel-by-channel multiplication in sequence to obtain a recalibrated feature map. Step S3: The recalibrated feature map is processed by feature projection and position encoding to obtain the position-encoded feature map; The feature maps after position encoding are aggregated along the frequency dimension and along the time dimension to obtain time domain features and frequency domain features. Using the time-domain features and the frequency-domain features as mutual queries and keys, bidirectional cross-attention fusion, residual connection, layer normalization, and global pooling are performed to obtain time-frequency fusion features. Step S4: Based on the intermediate feature maps of multiple intermediate layers of different scales in the backbone feature extraction network, multi-scale features are obtained, and the recalibrated feature maps are globally pooled to obtain backbone bypass features. Step S5: Perform gated weighted fusion of the time-frequency fusion features, multi-scale features, and backbone bypass features to obtain gated fusion features; The gated fusion features are used to obtain the predicted probability of each radio frequency interference category through fully connected mapping and activation function, and the radio frequency interference multi-label category is obtained based on the predicted probability.

2. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 1, characterized in that, In step S1, the dynamic spectrum data is a time-frequency two-dimensional image, with the horizontal axis representing the time dimension and the vertical axis representing the frequency dimension. The brightness value of each pixel represents the signal strength at the corresponding time and frequency.

3. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 2, characterized in that, In step S2, the backbone feature extraction network is the EfficientNet-B3 network; In step S2, the expression for the global adaptive average pooling process is: in, Indicates the channel descriptor. and These represent the height and width of the input feature map, respectively. This represents summing over all positions in the spatial dimension. Indicates the spatial location of the input feature map The characteristic values ​​of all channels, Represents the real number field. Indicates batch size, This indicates the number of channels in the output feature map; The expression for bottleneck structure processing is: in, Represents the channel weight vector. This represents the Sigmoid activation function. This represents a 1×1 convolution operation. This indicates a modified linear unit activation function; The expression for the channel-by-channel multiplication process is: in, This indicates the recalibrated feature map. This indicates element-wise multiplication.

4. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 3, characterized in that, In step S3, the expression for processing the recalibrated feature map through feature projection and position encoding is: in, This represents the projected feature map. This indicates a batch normalization operation. The feature dimension for time-frequency cross-attention processing; in, This represents the feature map after position encoding. This indicates a two-dimensional position embedding.

5. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 4, characterized in that, In step S3, the steps of performing frequency-dimension aggregation and time-dimension aggregation on the position-encoded feature maps specifically include time-domain feature construction and frequency-domain feature construction, with the following expressions: in, Representing time-domain features, Presentation layer normalization operation, Indicates that the input dimension is Output dimension is Linear projection layer, This represents the feature map after reshaping along the frequency dimension; in, Represents frequency domain characteristics, This represents the feature map after being reshaped along the time dimension.

6. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 5, characterized in that, In step S3, the steps of performing bidirectional cross-attention fusion, residual connection, layer normalization, and global pooling processing using the time-domain features and frequency-domain features as mutual queries and keys specifically include: In the time-to-frequency direction, using time-domain features as queries and frequency-domain features as keys and values, a query-key-value matrix is ​​generated through linear projection. After reshaping into a multi-head form, the scaled dot product attention is calculated, expressed as: ， ， in, This represents the cross-attention output in the time-to-frequency direction. For time-domain query matrix, For frequency domain key matrix, It is a frequency-domain matrix. For the projection matrix, express transpose, Scaling factor This represents the normalized exponential function; In the frequency-to-time direction, using frequency domain features as the query and time domain features as the key and value, cross-attention calculation is performed, expressed as: ， ， in, This represents the cross-attention output from frequency to time. For frequency domain lookup matrix, The time-domain key matrix, It is a time-domain value matrix. The projection matrix; and After the output is processed through multi-head splicing and linear output projection, residual connection and layer normalization are performed. The expression is as follows: in, This indicates the features after fusion in the time direction. This indicates the features after frequency direction fusion. and These are the attention outputs from two directions after multi-head splicing and linear projection, respectively; Will and A joint time-frequency grid is formed by dimensional expansion, and then the channel dimensions are merged by linear projection until a time-frequency fusion feature is obtained after global average pooling. The expression is as follows: in, This represents the time-frequency fusion feature vector. This indicates a global average pooling operation. Indicates that the input dimension is Output dimension is Linear projection layer, and These are the temporal fusion features and frequency fusion features after dimensional expansion, respectively. This indicates a splicing operation along the channel dimension.

7. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 6, characterized in that, In step S4, the expression for obtaining multi-scale features based on intermediate feature maps of multiple intermediate layers at different scales in the backbone feature extraction network is as follows: in, Indicates the first The result of projecting the feature map in the middle of the layer. Indicates the first Intermediate layer feature map The spatial dimensions of the registered location at the corresponding scale; in, Indicates the first sampled number after upsampling. Layer feature map, This indicates the bilinear interpolation upsampling operation. Indicates target resolution ; in, Multi-scale features are represented. This represents the concatenation of three upsampled feature maps along the channel dimension; In step S4, the expression for the backbone bypass characteristics is: in, Represents the backbone bypass feature vector. Indicates the flattening operation. This indicates a global adaptive average pooling operation. Indicates that the input dimension is Output dimension is The linear projection layer.

8. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 7, characterized in that, In step S5, the expression for the gated fusion feature is: in, Represents the gated fusion feature vector. For the gated vector, As a time-frequency fusion feature, Characterized by a backbone bypass. Indicates that the input dimension is Output dimension is The first linear transformation, Indicates that the input dimension is Output dimension is The second linear transformation; In step S5, the expression for the predicted probability is: in, This represents the expanded feature vector. Indicates that the input dimension is Output dimension is The fully connected layer This represents the feature vector after activation and regularization. This indicates a Dropout operation with a drop rate of 0.

3. Represents the original predicted score vector for each category; Indicates the first The predicted probabilities of each category, Indicates the first The original predicted scores for each category, It is a natural constant. Total number of radio frequency interference categories; The expression for obtaining the radio frequency interference multi-tag category based on the predicted probability is: in, Indicates the first The final judgment results for each category.

9. The radio frequency interference multi-label classification method based on time-frequency cross-attention fusion network according to claim 8, characterized in that, Also includes: The steps for training a time-frequency cross-attention fusion network include: determining the loss function and training parameters, using the AdamW optimizer and cosine annealing strategy for error backpropagation, and obtaining a trained deep neural network. The expression for the loss function is as follows: in, This indicates that when the loss function, This represents the true label of the k-th class, where K represents the number of classes. Represents the natural logarithm. Indicates the predicted label; The learning rate is adjusted according to a cosine annealing strategy during each training cycle, expressed as: in, This represents the learning rate corresponding to the t-th training period. This represents the preset minimum learning rate. This represents the initial learning rate, and t represents the current training cycle number. denoted by , where represents the total number of training cycles, and cos represents the cosine function.

Images