A long short-term differentiable channel pruning method for radio frequency signal classification

Abstract

This invention discloses a short-term and long-term differentiable channel pruning method for radio frequency signal classification, belonging to the field of neural network model compression and optimization technology. Addressing the problems of existing pruning methods such as decreased model recognition performance at high compression rates, neglecting short-term and long-term dynamic changes between channels, and reliance on manual thresholds, this invention embeds short-term and long-term modules through a teacher-student model architecture and employs a three-stage training strategy. The short-term module calculates scores based on channel features and generates masks to dynamically suppress redundant channels, while the long-term module periodically clusters and fuses channels to achieve structural pruning. This invention can compress model size and computational load while maintaining classification accuracy, making it suitable for real-time signal recognition tasks on resource-constrained edge devices.

Description

Technical Field

[0001] This invention belongs to the field of neural network model compression and optimization technology, and particularly relates to a long and short-term differentiable channel pruning method for radio frequency signal classification. Background Technology

[0002] With the rapid development of UAV communication, the Internet of Things, and fifth- and sixth-generation wireless communication technologies, the airborne wireless spectrum environment has become unprecedentedly complex. The diversity of UAV signals, interference sources, and environmental noise presents severe challenges to radio frequency (RF) signal classification, including spectrum sharing, dynamic changes, and multi-source concurrency. Against this backdrop, RF signal classification, as a core technology for UAV monitoring, spectrum sensing, and air-to-ground communication security, is increasingly demonstrating its application value. Traditional RF signal classification methods primarily rely on manual feature extraction and shallow classifiers, which significantly degrade in performance under low signal-to-noise ratio, multipath interference, or complex electromagnetic environments, making it difficult to meet the practical requirements of modern UAV communication systems for high accuracy, low latency, and strong robustness.

[0003] In recent years, the introduction of deep learning has greatly promoted the development of radio frequency signal classification technology. Deep models such as convolutional neural networks and residual networks can automatically extract high-discrimination features from raw I / Q signals, achieving high-precision classification in complex environments. However, with the improvement of model performance, the number of parameters and computational load have also increased dramatically, resulting in long inference latency and high power consumption, making deployment on resource-constrained edge devices extremely difficult. Especially in scenarios requiring real-time processing of high-speed signal streams, such as UAV adaptive spectrum sensing, aerial target monitoring, and emergency communication response systems, high-complexity models often cause excessive latency and computational resource constraints, severely limiting the practical application effectiveness of the models.

[0004] To address the aforementioned issues, lightweight neural network design and model pruning techniques have gradually become research hotspots. Existing model pruning methods are mainly classified into three categories: weight pruning, channel pruning, and layer pruning. For example, Zhang et al. proposed a radio frequency signal classification model based on neural architecture search, dynamically optimizing the model structure through automated search. Chen et al. used the scaling factor of batch normalized layers to evaluate channel importance to guide pruning. Lu et al. further proposed a three-stage layer pruning method based on Fisher's optimal segmentation. Although these methods have achieved certain results, they still have significant limitations in practical applications.

[0005] First, existing methods, in pursuit of high compression rates, often lead to a significant decline in model recognition performance, especially when dealing with complex modulation patterns and deep network structures, resulting in weakened feature extraction capabilities or difficulty in model convergence. Second, traditional pruning methods typically process channels or hierarchical structures independently, ignoring long-term dependencies between different layers and short-term dynamic changes between channels, making it difficult to efficiently remove redundant features. Furthermore, many pruning methods rely on manually set thresholds or discrete screening mechanisms, making end-to-end adaptive optimization difficult and limiting the model's practical deployment capability and scalability in dynamic wireless environments. These shortcomings are particularly pronounced in resource-constrained UAV airborne signal processing scenarios, urgently requiring a pruning solution that balances model accuracy, inference efficiency, and adaptive capabilities. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a short-term and long-term differential channel pruning method for radio frequency signal classification, thereby resolving the issues present in the prior art.

[0007] In a first aspect, to achieve the above objective, the present invention provides a long-term and short-term differentiable channel pruning method for radio frequency signal classification, comprising the following steps:

[0008] S1. Obtain the teacher model and student model, where the teacher model is obtained by supervised training of a convolutional neural network on a labeled dataset, and the student model is a copy of the teacher model;

[0009] S2. Embed a short-term channel sensitivity evaluation module and a long-term importance memory module at the back end of each convolutional layer in the student model;

[0010] S3. The student model is trained in three stages, including a warm-up stage, a joint optimization stage, and a fine-tuning stage, to generate a pruned lightweight model.

[0011] Optionally, in S1, the process of obtaining the teacher model and the student model includes:

[0012] The given dataset is divided into a training set and a test set, with the training set comprising the majority of the dataset and the test set comprising a small portion.

[0013] The convolutional neural network model is trained in a supervised manner using a training set. The training objective is to minimize the cross-entropy loss function in order to obtain a teacher model.

[0014] The teacher model is copied to obtain the student model, which is then used as the object for channel pruning.

[0015] Optionally, in S2, the process of the short-term channel sensitivity assessment module includes:

[0016] The Score module constructs channel embedding vectors based on channel weight features, activation features, batch normalization parameters, and gradient information, and inputs them into a multilayer perceptron to calculate the importance score of each channel.

[0017] By using the Gate module to generate channel masks based on channel importance scores and learnable logit parameters, and then dynamically modulating the channel output, short-term channel suppression is achieved.

[0018] Optionally, in S2, the process of the long-term importance memory module includes:

[0019] During training, time smoothing of channel importance scores is performed periodically to calculate the long-term average score.

[0020] Based on the long-term average score, a hierarchical clustering algorithm is used to divide the channel into multiple clusters, where the number of clusters is determined based on a preset pruning rate.

[0021] The channel weights within each cluster are fused and calculated, and the information of redundant channels is compressed into representative channels, thereby reducing the number of output channels and achieving structural pruning.

[0022] Optionally, in S3, the preheating stage includes the following processes:

[0023] The short-term channel sensitivity assessment module in the student model is trained to learn stable channel importance scores without actual pruning.

[0024] The warm-up phase provides reliable prior information on the importance of channels for the subsequent joint optimization phase, avoiding misjudgments in early training.

[0025] Optionally, in S3, the process of the joint optimization phase includes:

[0026] A joint loss function is introduced, which consists of classification loss, distillation loss and sparse regularization loss. The classification loss is based on the cross-entropy between the student model prediction and the true label, the distillation loss is based on the softened output of the teacher model and the student model under the temperature coefficient, and the sparse regularization loss is used to constrain the sparsity of the channel score and the gate mask.

[0027] During training, the long-term importance memory module periodically performs channel clustering and fusion operations. It performs clustering based on the long-term average of channel importance scores and fuses the channel weights within the clusters to achieve structured pruning.

[0028] Secondly, the present invention also provides a long-short-term differentiable channel pruning system for radio frequency signal classification, for implementing a long-short-term differentiable channel pruning method for radio frequency signal classification, the system comprising:

[0029] The model management module is used to obtain teacher and student models. The teacher model is obtained by supervised training of a convolutional neural network on a labeled dataset, and the student model is a copy of the teacher model.

[0030] The short-term channel sensitivity assessment module is configured to be embedded in the back end of each convolutional layer of the student model. It is used to calculate the channel importance score based on the channel weights, activations, batch normalization parameters and gradient information, and generate a channel mask to dynamically suppress redundant channels.

[0031] The long-term importance memory module is configured to be embedded in the back end of each convolutional layer of the student model to periodically smooth and cluster the channel importance scores and fuse the weights of redundant channels to achieve structural pruning.

[0032] The training control module is used to perform three-stage training on the student model, including a warm-up stage, a joint optimization stage, and a fine-tuning stage, to generate a pruned lightweight model.

[0033] Thirdly, the present invention also provides a computer terminal device, comprising:

[0034] One or more processors;

[0035] A memory, coupled to the processor, for storing one or more programs;

[0036] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the long and short-term differentiable channel pruning method for radio frequency signal classification in the first aspect described above.

[0037] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the long-short-term microchannel pruning method for radio frequency signal classification described in the first aspect above.

[0038] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the long-short-term differentially pruning method for radio frequency signal classification described in the first aspect above.

[0039] Compared with the prior art, the present invention has the following advantages and technical effects:

[0040] This invention provides a long-short-term differentiable channel pruning method for radio frequency signal classification. This invention achieves refined channel pruning with long-short-term feature awareness. By introducing a collaborative mechanism between a short-term channel sensitivity evaluation module and a long-term importance memory module, it can accurately identify the feature contribution of channels at different time scales, effectively distinguishing between transiently ineffective channels and stable key channels. This strategy significantly reduces redundant computation channels, avoiding the performance collapse problem caused by the mis-pruning of key features in traditional static pruning methods, thus maintaining recognition accuracy comparable to the original model even at high compression ratios. Simultaneously, by introducing long-short-term feature adaptive updating and differentiable sparse optimization mechanisms during the pruning process, this invention enables the model to continuously adjust channel weights during the training phase, ensuring its robustness under different signal-to-noise ratios and dynamic channel conditions. This effectively reduces redundant computation and latency fluctuations in convolutional layers, ultimately enabling the lightweight model to achieve low-latency, high-stability online modulation recognition inference on edge terminals. Attached Figure Description

[0041] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0042] Figure 1 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0045] In next-generation wireless communication networks, edge intelligent terminals (such as airborne signal processing units, UAV communication modules, or IoT nodes) are undertaking critical tasks ranging from radio frequency (RF) signal acquisition and feature extraction to classification and adaptive link control. However, these devices typically have limited computing power and storage resources, while needing to perform millisecond-level signal recognition and decision-making under dynamic channel conditions and low signal-to-noise ratios. Therefore, effectively compressing model size and improving inference efficiency while ensuring RF signal classification accuracy has become one of the core challenges in the field of RF signal classification.

[0046] For this resource-constrained real-time recognition scenario, the long and short-term differentiable channel pruning framework proposed in this invention can simultaneously model the short-term dynamics and long-term stable contributions of channel features during the model training phase, and achieve end-to-end adaptive compression of the structure through a differentiable optimization mechanism.

[0047] Specifically, this invention introduces a short-term channel sensitivity assessment module and a long-term importance memory unit: the former captures the instantaneous contribution of channels based on the current gradient and activation response, thereby quickly adapting to changes in feature distribution under different signal environments; the latter estimates the global importance of channels during multi-batch training through a sliding weighting and temporal accumulation mechanism, avoiding the structural degradation and performance instability problems caused by traditional one-time pruning strategies. The synergistic effect of these two modules enables pruning decisions to possess both rapid local responsiveness and maintain global structural stability, thus achieving high compression ratios and low computational latency without sacrificing recognition performance.

[0048] Example 1

[0049] like Figure 1 As shown, this embodiment provides a long-term and short-term differentiable channel pruning method for radio frequency signal classification, including:

[0050] S1. Obtain the teacher model and student model, where the teacher model is obtained by supervised training of a convolutional neural network on a labeled dataset, and the student model is a copy of the teacher model;

[0051] S2. Embed a short-term channel sensitivity evaluation module and a long-term importance memory module at the back end of each convolutional layer in the student model;

[0052] S3. The student model is trained in three stages, including a warm-up stage, a joint optimization stage, and a fine-tuning stage, to generate a pruned lightweight model.

[0053] The above method includes the following steps:

[0054] S1: Given a dataset and a convolutional neural network model, divide the dataset into a training set and a test set. The training set consists of a large amount of labeled data for supervised training and fine-tuning, while the test set consists of a small batch of labeled data for performance evaluation. Use the training set to perform supervised training on the convolutional neural network model to obtain a teacher model, which serves as the source of knowledge distillation during the pruning process. Copy the teacher model to obtain the student model.

[0055] S2: Initialize the student model from S1 and prepare for channel pruning. First, embed a short-term channel sensitivity evaluation module (including a Score module and a Gate module) and a long-term importance memory module at the back end of each convolutional layer of the student model to construct the channel pruning preparation stage. Specifically, the Score module calculates channel importance scores based on channel embedding information, providing a quantitative basis for subsequent channel selection; the Gate module controls the activation state of channels in a differentiable manner to achieve short-term dynamic suppression of redundant channels; and the long-term importance memory module periodically performs channel clustering and fusion operations during training to complete long-term structured pruning, while using feature fusion techniques to maintain key representation capabilities and reduce accuracy loss caused by pruning.

[0056] S3: The student model from S2 is trained using a three-stage strategy, including a warm-up phase, a joint optimization phase, and a fine-tuning phase. First, in the warm-up phase, the Score module is pre-trained to learn stable and discriminative channel scores without actual pruning, providing reliable prior information for subsequent joint training. Then, in the joint optimization phase, a joint loss function is introduced, consisting of classification loss, distillation loss, and sparse regularization loss. The classification loss maintains the model's discriminative ability on the target task, the distillation loss guides the student model to inherit the knowledge representation of the teacher model, and the sparse regularization loss constrains the sparsity of channel scores and gate masks, enhancing the pruning effect. The long-term importance memory module performs temporal smoothing and cluster analysis on the channel scores, grouping channels with similar or redundant scores into the same cluster and performing weight fusion on channels within the cluster, thus achieving structural pruning. Finally, in the fine-tuning phase, the fused, compact model is finely adjusted with a small learning rate to restore the model's performance stability and classification accuracy. This three-stage training strategy achieves channel soft suppression in the short term through the short-term channel sensitivity assessment module, and in the long term through the long-term importance memory module to complete channel clustering fusion and structural pruning, ultimately obtaining a lightweight neural network model that combines high efficiency and accuracy.

[0057] As one implementation method in this embodiment, in S1, the process of obtaining the teacher model and the student model includes:

[0058] The given dataset is divided into a training set and a test set, with the training set comprising the majority of the dataset and the test set comprising a small portion.

[0059] The convolutional neural network model is trained in a supervised manner using a training set. The training objective is to minimize the cross-entropy loss function in order to obtain a teacher model.

[0060] The teacher model is copied to obtain the student model, which is then used as the object for channel pruning.

[0061] Furthermore, in step S1, a dataset is typically split into a training set and a test set, represented as follows: and ,in Therefore, 80% of the dataset will be used as the training set. , represented as Use 20% of the dataset as the test set. ,in Indicates a sample, The label is represented by the convolutional neural network model. Include The first convolutional layer. The weight tensor of each convolutional layer can be represented as: in . and Indicates the first The number of output channels and input / output channels of each convolutional layer and This represents the height and width of the convolution kernel. (Using the training set) For the convolutional neural network Supervised training is performed with the objective of minimizing the cross-entropy loss function to obtain the teacher model. ; Replicating the teacher model To obtain student models .

[0062] As one implementation method in this embodiment, in S2, the process of the short-term channel sensitivity assessment module includes:

[0063] The Score module constructs channel embedding vectors based on channel weight features, activation features, batch normalization parameters, and gradient information, and inputs them into a multilayer perceptron to calculate the importance score of each channel.

[0064] By using the Gate module to generate channel masks based on channel importance scores and learnable logit parameters, and then dynamically modulating the channel output, short-term channel suppression is achieved.

[0065] As one implementation method in this embodiment, in S2, the process of the long-term importance memory module includes:

[0066] During training, time smoothing of channel importance scores is performed periodically to calculate the long-term average score.

[0067] Based on the long-term average score, a hierarchical clustering algorithm is used to divide the channel into multiple clusters, where the number of clusters is determined based on a preset pruning rate.

[0068] The channel weights within each cluster are fused and calculated, and the information of redundant channels is compressed into representative channels, thereby reducing the number of output channels and achieving structural pruning.

[0069] Furthermore, in step S2, the student model initialization process is as follows:

[0070] The short-term channel sensitivity assessment module includes a Score module and a Gate module. The Score module constructs channel embeddings based on channel weights, activation features, batch normalization parameters, and gradient information, and calculates channel importance scores using a multilayer perceptron. Specifically, for the... The first convolutional layer Output Channel The channel weight feature can be defined as its L2 norm:

[0071] (1)

[0072] This channel is for input samples The activation output value is The activation feature is defined as its mean:

[0073] (2)

[0074] Batch normalization after channelization, with scaling parameters as follows: Then the BN feature is defined as:

[0075] (3)

[0076] The gradient of the channel weight is:

[0077] (4)

[0078] The gradient feature is then defined as:

[0079] (5)

[0080] The above four types of features are concatenated to obtain the channel embedding. :

[0081] (6)

[0082] Embed the channels into the multilayer perceptron (MLP):

[0083] (7)

[0084] in That is, the first Layer Importance score of the channel.

[0085] The Gate module generates channel masks in a differentiable manner based on channel scores and learnable logit parameters. These masks are used to dynamically control channel activity and achieve short-term channel suppression. Each channel is assigned a learnable parameter. This parameter represents the "prior switching probability" of this channel, and it is continuously updated during training. A mask between 0 and 1 is then generated using the sigmoid function. Defined as:

[0086] (8)

[0087] in The scaling factor controls the sensitivity of the channel score to the mask; This is the sigmoid function. After modulation by the Gate module, the new output is:

[0088] (9)

[0089] in It is the first The first convolutional layer The raw output of each channel. During forward propagation, the network can dynamically adjust the activity of each channel; during backpropagation, the gradient can be propagated back to... and the Score module.

[0090] The long-term importance memory module periodically performs cluster analysis on channel scores during training, grouping similar or redundant channels into the same cluster. It then employs model fusion technology to calculate the channel weights within the same cluster, compressing information from redundant channels into representative channels to achieve information reconstruction and compensation. In this way, redundant channels are gradually replaced by representative channels, and their feature contributions are absorbed through fusion, thus achieving long-term channel fusion and simplification while maintaining the model's expressive power. First, the importance score of each convolutional layer channel is temporally smoothed. Within a set period T, the importance score of the first channel is calculated... Layer Long-term average score of each channel:

[0091] (10)

[0092] in This represents the instantaneous score output by the Score module at the t-th iteration. Then, the long-term score vector for all channels is obtained. by As feature input, a hierarchical clustering algorithm is used to perform similarity analysis on the channels. The hierarchical clustering algorithm is then applied to divide the channels into groups based on a set pruning rate. The group, the formula is as follows:

[0093] (11)

[0094] In the formula, This is expressed as the pruning rate. Specifically, the hierarchical clustering algorithm first... All channels of the layer are treated as a single sample set. Then each sample Initialize as an independent cluster At this time, there are a total of Given an initial cluster, the average linking method is used to calculate any two clusters. and The distance between them is defined as:

[0095] (12)

[0096] Find the two clusters with the minimum distance in the current distance matrix. and Merge two clusters ,Will The subsequent clusters are shifted one position to the left to ensure consecutive cluster numbering. (Delete distance matrix) Zhongyu Relevant columns and recalculate. Calculate the average link distance to other clusters, update the corresponding column in the distance matrix, and decrement the cluster count by one. Repeat this iteration until the cluster count is [value missing]. When the time is reached, the clustering process terminates, and the final set of clusters is output. .

[0097] Since channels within the same cluster exhibit high similarity in feature representation, indicating strong information redundancy, model fusion techniques are employed to fuse and calculate the channel weight parameters within the same cluster, compressing the information from redundant channels into representative channels. In this context, the fusion formula is as follows:

[0098] (13)

[0099] After channel fusion operation, the first The number of output channels of each convolutional layer is increased from the original Reduce to Then, the same pruning operation is performed on the input channels of the convolutional layer to construct a preliminary channel pruning model.

[0100] In the overall channel pruning framework, the short-term channel sensitivity assessment module achieves short-term soft pruning through a differentiable channel masking mechanism, flexibly suppressing unimportant channels during training to ensure gradient propagation. Meanwhile, the long-term importance memory module prepares for hard pruning through long-term score clustering and weight fusion, gradually absorbing redundant channels into representative channels and removing them at the structural level. The two modules work together to achieve a gradual channel pruning process from short-term suppression to long-term structural simplification.

[0101] As one implementation method in this embodiment, in S3, the preheating stage includes the following process:

[0102] The short-term channel sensitivity assessment module in the student model is trained to learn stable channel importance scores without actual pruning.

[0103] The warm-up phase provides reliable prior information on the importance of channels for the subsequent joint optimization phase, avoiding misjudgments in early training.

[0104] As one implementation method in this embodiment, in S3, the process of the joint optimization phase includes:

[0105] A joint loss function is introduced, which consists of classification loss, distillation loss and sparse regularization loss. The classification loss is based on the cross-entropy between the student model prediction and the true label, the distillation loss is based on the softened output of the teacher model and the student model under the temperature coefficient, and the sparse regularization loss is used to constrain the sparsity of the channel score and the gate mask.

[0106] During training, the long-term importance memory module periodically performs channel clustering and fusion operations. It performs clustering based on the long-term average of channel importance scores and fuses the channel weights within the clusters to achieve structured pruning.

[0107] Furthermore, in step S3, the training process of the three-stage strategy is as follows:

[0108] First, in the preparation and warm-up phase, the teacher and student models are loaded, with a focus on warming up the short-term channel sensitivity assessment module. The purpose of this phase is to enable the scorer to learn meaningful channel importance scores without substantial pruning, laying a reliable foundation for subsequent joint optimization and avoiding irreversible performance loss due to early misjudgments.

[0109] Subsequently, the core joint training phase begins. In this phase, knowledge distillation, reconstruction loss, and sparsity constraints are jointly introduced. Specifically, the classification loss is used to maintain the discriminative ability of the student model on the target task, assuming the training samples are... The predicted probability distribution of the student model is as follows: The classification loss is then defined as the cross-entropy function:

[0110] (14)

[0111] in, For the sample size, The true label for the sample.

[0112] Distillation loss is used to guide the student model to inherit the knowledge representation of the teacher model. Let the output distribution of the teacher model be... Using temperature coefficient If the softened output is obtained, then the distillation loss is:

[0113] (15)

[0114] in, For the number of categories, Indicates the temperature after passing through The smoothed softmax distribution. Sparse regularization loss is used to promote sparsity in channel scores and gate masks, encouraging the model to automatically suppress unimportant channels. The sparse regularization term is defined as:

[0115] (16)

[0116] in, is the sparsity coefficient.

[0117] In summary, the joint loss function is:

[0118] (17)

[0119] in, and These are the weight coefficients for distillation loss and sparse regularization loss, used to balance the three types of objectives. During training, the long-term importance memory module periodically performs channel clustering and fusion operations: based on the periodic average score output by the scorer, channels are clustered, and weight fusion is performed on the channels within the cluster (according to equations (10)-(13)), thereby achieving structured pruning.

[0120] Finally, in the fine-tuning stage, the obtained compact model is finely tuned for one cycle to restore and stabilize its final performance with a small learning rate, thus obtaining the final pruned model. This maintains the original high recognition accuracy as much as possible while significantly reducing the number of parameters and computation.

[0121] In summary, the entire process employs a three-stage strategy: short-term channel sensitivity assessment module enables soft pruning, long-term importance memory enables structural fusion and hard pruning, allowing the student model to gradually evolve into a compact and efficient channel pruning model under the guidance of knowledge distillation.

[0122] Based on this, the present invention provides a long-term and short-term differentiable channel pruning method for radio frequency signal classification. The beneficial effects of the present invention are as follows:

[0123] (1) Achieving refined channel pruning with long-term and short-term feature awareness: This invention introduces a mechanism for decoupling long-term and short-term features and evaluating the importance of differentiable channels, which can accurately identify the feature contribution of channels at different time scales, thereby effectively distinguishing between transiently invalid channels and stable key channels. This strategy not only significantly reduces redundant computational channels, but also avoids the performance collapse problem caused by the mispruning of key features in traditional static pruning methods. Under high compression ratios, it can still maintain recognition accuracy comparable to the original model, achieving a dynamic balance between model performance and structural compression.

[0124] (2) Improve the real-time performance of modulation recognition: By introducing adaptive updating of long-term and short-term features and differentiable sparse optimization mechanism during the pruning process, this method can continuously adjust the channel weights during the training phase to ensure the robustness of the model under different signal-to-noise ratios and dynamic channel conditions. Compared with traditional channel pruning methods, this invention can effectively and quickly reduce redundant computation and latency fluctuations in convolutional layers in the rapidly changing scenario of UAV wireless signals, enabling the model to achieve low-latency and high-stability online modulation recognition inference on edge terminals.

[0125] The following description, in conjunction with the accompanying drawings, uses the airborne signal processing unit in an unmanned aerial vehicle (UAV) wireless communication system as an application scenario and the DroneRF UAV radio frequency dataset to provide a systematic and clear description of the "long-short-term differentiable channel pruning framework for radio frequency signal classification" of the present invention.

[0126] In low-altitude airspace management and UAV safety communication systems, radio frequency signal identification is a crucial link in achieving target monitoring, interference defense, and spectrum situational awareness. However, real-world UAV communication environments often involve a variety of complex factors: severe frequency band overlap, strong signal interference, limited bandwidth resources, and limited computing power of edge sensing nodes (such as embedded GPU devices equipped with only 0.5GB~2GB of video memory). Although traditional deep neural network models achieve high recognition accuracy on standard datasets, their large number of parameters and high inference latency make them unstable for deployment in resource-constrained UAV communication nodes, resulting in a significant decrease in real-time recognition performance.

[0127] Under these limited hardware conditions, the system deploys an optimized lightweight modulation recognition model using the long and short term differentiable channel pruning method of this invention for real-time signal recognition of radio frequency signals captured in the UAV communication link. Its low inference latency ensures real-time signal recognition capability, and the low computational load translates into lower device power consumption and thermal load, effectively extending the UAV's flight time.

[0128] Reference Figure 1 A short- and long-term differentiable channel pruning framework for radio frequency signal classification includes the following steps:

[0129] S1: Given a dataset and a neural network model, divide the dataset into a training set and a test set.

[0130] In this embodiment, given the DroneRF dataset, the neural network model is determined to be the SigNet50 convolutional neural network.

[0131] DroneRF is an authoritative open-source dataset designed specifically for radio frequency (RF) sensing research. Its core value lies in providing benchmark data for drone detection and classification algorithms using RF signals collected in real-world environments. The dataset contains four types of signal sources (background noise and three types of drones: Bebop, AR, and Phantom). Each type of drone signal is collected under four operating modes (power-on standby, hovering, flight without video, and flight with video), fully covering typical application scenarios. Data acquisition employs a dual-band synchronous strategy, capturing signals in the 2400-2440MHz low-frequency band and the 2440-2480MHz high-frequency band using two 40MHz bandwidth receivers, resulting in a total of 454 records (227 low-frequency records and 227 high-frequency records). All data is saved in its original I / Q sampling format, with drone recordings lasting 5.25 seconds and background recordings lasting 10.25 seconds, fully preserving the amplitude and phase characteristics of the signals. This dataset, by subdividing long recordings into millisecond-level segments (e.g., 25ms), constructs a large-scale sample library and has become a core benchmark for evaluating the performance of drone RF classification algorithms. The DroneRF dataset is divided into a training set and a test set, denoted as follows: and ,in The ratio is 4:1. The SigNet50 model consists of 6 residual blocks. Each residual block contains 5 convolutional layers and 1 pooling layer. The training set is used... For the SigNet50 model Supervised training is performed with the objective of minimizing the cross-entropy loss function to obtain the teacher model. ; Replicating the teacher model To obtain student models Finally, a reference was obtained. Figure 1 The student model on the top side of the middle.

[0132] S2: Student Model Initialization. First, embed a short-term channel sensitivity evaluation module (including a Score module and a Gate module) and a long-term importance memory module at the back end of each convolutional layer of the student model to construct the channel pruning preparation stage.

[0133] In this embodiment, in the calculation of the short-term channel sensitivity assessment module, the Score module constructs the channel embedding based on the channel weights, activation features, batch normalization parameters, and gradient information, and calculates the channel importance score through a multilayer perceptron, as shown in formulas (1) to (6). The Gate module generates a channel mask in a differentiable manner based on the channel score and the learnable logit parameter, which is used to dynamically control the activity of the channel and realize short-term channel suppression, as shown in formulas (7) to (8).

[0134] Subsequently, during training, the long-term importance memory module periodically performs cluster analysis on channel scores, grouping similar or redundant channels into the same cluster. Model fusion technology is then used to calculate the channel weight parameters within the same cluster, compressing the information from redundant channels into representative channels to achieve information reconstruction and compensation. Specific methods are shown in formulas (9) to (12). (Refer to the reference.) Figure 1 The right-hand side of the module is embedded.

[0135] S3: The training process of the three-stage strategy in this embodiment. First, in the preparation and warm-up stage, the teacher model and student model are loaded, and the short-term channel sensitivity assessment module is warmed up and trained to avoid irreversible performance loss due to early misjudgment. Then, the core joint training stage is entered. In this stage, knowledge distillation, reconstruction loss and sparsity constraints are introduced together. The specific loss functions are as shown in formulas (13)-(16) with reference to the following. Figure 1 The left side of the middle section. Finally, in the fine-tuning stage, the obtained compact model undergoes one cycle of fine-tuning to restore and stabilize its final performance with a small learning rate, resulting in the final pruned model. This significantly reduces the number of parameters and computational load while maintaining the original high recognition accuracy as much as possible. Finally, the final model is deployed to edge drone devices.

[0136] Final testing showed that the pruned lightweight model reduced the number of parameters by 76.3%, computational cost (FLOPs) by 79.93%, and real-time inference latency by approximately 40% on the DroneRF dataset. The optimized model requires only about 300MB of GPU memory to load and run stably, making it perfectly suited for drone RFID nodes equipped with embedded GPUs. Under these conditions, the model maintained an overall accuracy of 97.33% in drone communication and identification tasks, achieving an optimal balance between accuracy and lightweight design.

[0137] This embodiment proposes a short-term and long-term differentiable channel pruning framework for radio frequency signal classification. This method achieves end-to-end optimizability of the pruning process through the collaborative design of a short-term channel sensitivity assessment module and a long-term importance memory module, combined with a differentiable channel scoring mechanism and a three-stage training strategy. While maintaining model performance, it significantly reduces the number of parameters and computational load, thereby making the signal model more adaptable to the deployment requirements of edge devices and exhibiting good performance.

[0138] Example 2

[0139] In this embodiment, a computer terminal device is provided, including:

[0140] One or more processors;

[0141] A memory, coupled to the processor, for storing one or more programs;

[0142] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described long and short-term differentiable channel pruning method for radio frequency signal classification.

[0143] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described method for long- and short-term microchannel pruning for radio frequency signal classification.

[0144] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the above-described long and short-term microchannel pruning method for radio frequency signal classification.

[0145] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described long and short-term differentially pruning method for radio frequency signal classification.

[0146] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0147] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0148] This embodiment provides such an apparatus or system. The system, referred to as a long-short-term microchannel pruning system for radio frequency signal classification, includes:

[0149] The model management module is used to obtain teacher and student models. The teacher model is obtained by supervised training of a convolutional neural network on a labeled dataset, and the student model is a copy of the teacher model.

[0150] The short-term channel sensitivity assessment module is configured to be embedded in the back end of each convolutional layer of the student model. It is used to calculate the channel importance score based on the channel weights, activations, batch normalization parameters and gradient information, and generate a channel mask to dynamically suppress redundant channels.

[0151] The long-term importance memory module is configured to be embedded in the back end of each convolutional layer of the student model to periodically smooth and cluster the channel importance scores and fuse the weights of redundant channels to achieve structural pruning.

[0152] The training control module is used to perform three-stage training on the student model, including a warm-up stage, a joint optimization stage, and a fine-tuning stage, to generate a pruned lightweight model.

[0153] As one implementation method in this embodiment, the model management module includes:

[0154] A dataset partitioning unit is used to divide a given dataset into a training set and a test set, where the training set accounts for the majority of the dataset and the test set accounts for a small portion.

[0155] The teacher model training unit is used to perform supervised training on the convolutional neural network model using the training set. The training objective is to minimize the cross-entropy loss function to obtain the teacher model.

[0156] The student model replication unit is used to replicate the teacher model to obtain the student model.

[0157] As one implementation method in this embodiment, the short-term channel sensitivity assessment module includes:

[0158] Channel embedding building units are used to construct channel embedding vectors based on channel weight features, activation features, batch normalization parameters, and gradient information.

[0159] The importance score calculation unit is used to input the channel embedding vector into the multilayer perceptron to calculate the importance score of each channel;

[0160] The channel mask generation unit is used to generate channel masks using the sigmoid function based on channel importance scores and learnable logit parameters, dynamically modulating the channel output to achieve short-term channel suppression.

[0161] As one implementation method in this embodiment, the long-term importance memory module includes:

[0162] The time smoothing unit is used to periodically perform time smoothing of channel importance scores during training and calculate the long-term average score.

[0163] The channel clustering unit is used to divide the channel into multiple clusters based on the long-term average score using a hierarchical clustering algorithm, wherein the number of clusters is determined based on a preset pruning rate;

[0164] The weight fusion unit is used to perform fusion calculations on the channel weights within each cluster, compressing the information of redundant channels into representative channels, thereby reducing the number of output channels and achieving structural pruning.

[0165] As one implementation method in this embodiment, the training control module includes the following in the preheating phase:

[0166] The warm-up training unit is used to train the short-term channel sensitivity assessment module in the student model, enabling the Score module to learn stable channel importance scores without actual pruning.

[0167] The preheating phase provides reliable prior information on the importance of channels for the subsequent joint optimization phase.

[0168] As one implementation method in this embodiment, the training control module includes the following in the joint optimization phase:

[0169] The loss calculation unit is used to calculate the joint loss function, which consists of classification loss, distillation loss and sparse regularization loss. The classification loss is based on the cross-entropy between the student model prediction and the true label, the distillation loss is based on the softened output of the teacher model and the student model under the temperature coefficient, and the sparse regularization loss is used to constrain the sparsity of the channel score and the gate mask.

[0170] The pruning execution unit is used to periodically perform channel clustering and fusion operations during training. It performs clustering based on the long-term average of channel importance scores and fuses the channel weights within the cluster to achieve structured pruning.

[0171] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0172] The above implementation method solves the problem of short-term and long-term microchannel pruning for radio frequency signal classification in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0173] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for long-term and short-term differentiable channel pruning for radio frequency signal classification, characterized in that, For application in resource-constrained edge computing devices, the following steps are included: S1. Obtain the teacher model and student model. The teacher model is obtained by supervised training of a convolutional neural network on a labeled radio frequency signal dataset. The student model is a copy of the teacher model. The labeled radio frequency signal dataset is a drone detection and classification dataset. The labeled radio frequency signal dataset contains background noise and three types of drones: Bebop, AR, and Phantom, totaling four signal sources. Each type of drone signal is collected in four operating modes. S2. Embed a short-term channel sensitivity evaluation module and a long-term importance memory module at the back end of each convolutional layer in the student model; In S2, the process of the short-term channel sensitivity assessment module includes: The Score module constructs channel embedding vectors based on channel weight features, activation features, batch normalization parameters, and gradient information, and inputs them into a multilayer perceptron to calculate the importance score of each channel. The Gate module generates a channel mask using the sigmoid function based on the channel importance score and the learnable logit parameter, and dynamically modulates the channel output to achieve short-term channel suppression. In S2, the process of the long-term importance memory module includes: During training, time smoothing of channel importance scores is performed periodically to calculate the long-term average score. Based on the long-term average score, a hierarchical clustering algorithm is used to divide the channel into multiple clusters, where the number of clusters is determined based on a preset pruning rate. The channel weights within each cluster are fused and calculated to compress the information of redundant channels into representative channels, thereby reducing the number of output channels and achieving structural pruning. S3. Perform three-stage training on the student model, including a warm-up stage, a joint optimization stage, and a fine-tuning stage, to generate a pruned, lightweight model adapted to the resource-constrained edge computing device.

2. The method according to claim 1, characterized in that, In S1, the process of obtaining the teacher model and the student model includes: The given dataset is divided into a training set and a test set, with the training set comprising the majority of the dataset and the test set comprising a small portion. The convolutional neural network model is trained in a supervised manner using a training set. The training objective is to minimize the cross-entropy loss function in order to obtain a teacher model. The teacher model is copied to obtain the student model, which is then used as the object for channel pruning.

3. The method according to claim 1, characterized in that, In S3, the preheating stage includes the following processes: The short-term channel sensitivity assessment module in the student model is trained to learn stable channel importance scores without actual pruning. The warm-up phase provides reliable prior information on the importance of channels for the subsequent joint optimization phase, avoiding misjudgments in early training.

4. The method according to claim 1, characterized in that, In S3, the process of the joint optimization phase includes: A joint loss function is introduced, which consists of classification loss, distillation loss and sparse regularization loss. The classification loss is based on the cross-entropy between the student model prediction and the true label, the distillation loss is based on the softened output of the teacher model and the student model under the temperature coefficient, and the sparse regularization loss is used to constrain the sparsity of the channel score and the gate mask. During training, the long-term importance memory module periodically performs channel clustering and fusion operations. It performs clustering based on the long-term average of channel importance scores and fuses the channel weights within the clusters to achieve structured pruning.

5. A short- and long-term microchannel pruning system for radio frequency signal classification, characterized in that, The system for implementing the method according to any one of claims 1-4 comprises: The model management module is used to obtain teacher and student models. The teacher model is obtained by supervised training of a convolutional neural network on a labeled dataset, and the student model is a copy of the teacher model. The short-term channel sensitivity assessment module is configured to be embedded in the back end of each convolutional layer of the student model. It is used to calculate the channel importance score based on the channel weights, activations, batch normalization parameters and gradient information, and generate a channel mask to dynamically suppress redundant channels. The long-term importance memory module is configured to be embedded in the back end of each convolutional layer of the student model to periodically smooth and cluster the channel importance scores and fuse the weights of redundant channels to achieve structural pruning. The training control module is used to perform three-stage training on the student model, including a warm-up stage, a joint optimization stage, and a fine-tuning stage, to generate a pruned lightweight model.

6. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images