Two-stage recoverable model pruning method for automatic modulation identification

Abstract

The invention discloses a two-stage recoverable model pruning method for automatic modulation identification, belongs to the crossing field of communication signal processing and artificial intelligence, and solves the problems that an existing pruning method is difficult to maintain the model performance under a high compression ratio and the redundancy layer identification is inaccurate. According to the method, redundant channels are removed based on channel similarity clustering fusion in the channel pruning stage, a degradation layer is removed based on linear detection diagnosis in the layer pruning stage, the model performance is recovered in combination with fine adjustment, high recognition precision is kept while model compression is achieved, and the method is mainly used for automatic modulation recognition deployment in resource-constrained equipment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of communication signal processing and artificial intelligence, and particularly relates to a two-stage recoverable model pruning method for automatic modulation recognition. BACKGROUND

[0002] Automatic modulation recognition, as a core technology of wireless communication intelligent demodulation, spectrum management and security monitoring, plays an important role in the fifth generation and future sixth generation mobile communication, Internet of Things and other complex electromagnetic environments. Traditional methods rely on artificial feature extraction and shallow machine learning models, which have limited recognition accuracy and insufficient generalization ability in complex environments, making it difficult to meet the requirements of modern communication systems for high accuracy and real-time performance. The rise of deep learning provides a new way to solve this problem. Convolutional neural networks, deep residual networks and other models significantly improve recognition accuracy with end-to-end feature learning capabilities. However, these models usually have a large number of parameters and high computational complexity, resulting in slow inference speed and high resource occupation. Signal terminal processing devices such as base stations, Internet of Things nodes and mobile terminals often have limited computing, storage and energy resources, while real-time signal processing systems need to complete inference under strict time delay constraints. The high parameter quantity and computational complexity of traditional models can cause high signal processing link delay, affecting communication quality. These contradictions make it difficult to deploy high-performance deep learning models in practical applications.

[0003] To solve the problem of limited resources of signal terminal processing devices, lightweight neural networks have become the focus of automatic modulation recognition research, and model pruning has attracted widespread attention due to its versatility and high adaptability in model structure optimization. Existing neural network model pruning methods mainly include weight pruning, channel pruning and layer pruning, each with its own characteristics in terms of pruning granularity, compression efficiency and actual deployment effect. Although existing methods have achieved certain results, they are difficult to achieve significant model compression rate while maintaining accuracy. Specifically, existing pruning methods often struggle to balance model performance while pursuing high compression rates, and may experience a sharp drop in accuracy or even model collapse when processing complex modulation signals and deep neural networks. In particular, during the channel pruning stage, as the pruning ratio increases, the model may lose a large number of key channels, significantly weakening the feature extraction capability. In multi-layer networks, redundant layers and degenerative layers are difficult to accurately identify, leading to long-term accumulation of structural redundancy, not only wasting computing resources, but also weakening the actual inference acceleration effect of the pruned model, limiting the deployment ability of the model in resource-constrained environments. These problems are particularly pronounced in practical application scenarios such as 5G / 6G base stations, embedded communication terminals and Internet of Things nodes, and a new pruning method that can maintain model performance at high compression rates is urgently needed. SUMMARY

[0004] To solve the above technical problems, the present application provides a two-stage recoverable model pruning method for automatic modulation recognition to solve the problems existing in the prior art.

[0005] In the first aspect, to achieve the above object, the present application provides a two-stage recoverable model pruning method for automatic modulation recognition, comprising the following steps:

[0006] S1. Under a given data set and a convolutional neural network model, the data set is divided into a training set and a test set, and the training set is used to supervise the training of the model to obtain a pre-trained model;

[0007] S2. Channel pruning is performed on the pre-trained model, including calculating the similarity of the convolutional layer channels, performing hierarchical clustering based on the similarity to obtain channel clustering division, determining the target cluster number according to the pruning rate, performing parameter fusion on the channels in the same cluster and removing redundant channels to obtain a channel pruning model, and fine-tuning the channel pruning model;

[0008] S3. Layer evaluation is performed on the fine-tuned channel pruning model, including freezing the convolutional layer parameters, introducing a linear layer after the convolutional layer and training the linear layer using the training set to evaluate the performance of each convolutional layer;

[0009] S4. Layer pruning is performed based on the layer evaluation results, including comparing the performance of adjacent layers to identify lower performance layers, removing these layers to obtain a layer pruning model, fine-tuning the layer pruning model, and evaluating the performance of the final model using the test set.

[0010] Optionally, in S1, the process of dividing the data set into a training set and a test set includes: 80% of the data set is used as the training set and 20% is used as the test set; the process of using the training set to supervise the training of the model includes: minimizing the cross-entropy loss function to obtain the pre-trained model.

[0011] Optionally, in S2, the process of performing channel pruning on the pre-trained model includes: for the output channels of each convolutional layer, vectorizing the weight tensor and calculating the cosine similarity between the channels, converting the similarity into a correlation distance matrix; applying a hierarchical clustering algorithm to divide the channels into multiple clusters according to the pruning rate; performing weighted fusion on the channel weights within the same cluster to retain representative channels and remove redundant channels; fine-tuning the pruned model.

[0012] Optionally, in S3, the process of layer evaluation on the channel pruning model includes: freezing the convolutional layer parameters of the channel pruning model; extracting the feature representation of each convolutional layer from the training set; connecting a linear classification layer after each convolutional layer, training the linear classification layer using the training set, and verifying the classification accuracy of the linear classification layer using the test set as the layer performance indicator.

[0013] Optionally, in S4, the process of performing layer pruning based on the layer evaluation results includes: comparing the performance indicators of adjacent layers; if the performance difference is lower than a preset redundancy threshold, the layer is determined to be a weak layer; removing the weak layer to obtain the layer pruning model; fine-tuning the layer pruning model; and evaluating the performance of the final pruning model using a test set.

[0014] Optionally, in S4, the process of comparing the performance of adjacent layers includes: calculating the performance index difference between the i-th layer and the (i+1)-th layer; if the difference is less than the redundancy threshold, then the i-th layer is determined to be a weaker layer.

[0015] Secondly, the present invention also provides a two-stage recoverable model pruning system for automatic modulation identification, for implementing a two-stage recoverable model pruning method for automatic modulation identification, the system comprising:

[0016] The model training module is used to divide the dataset into training and test sets given a dataset and a convolutional neural network model, and to use the training set to perform supervised training on the model to obtain a pre-trained model.

[0017] The channel pruning module is used to prune channels in the pre-trained model, including calculating the similarity of convolutional layer channels, performing hierarchical clustering based on similarity to obtain channel clustering, determining the number of target clusters based on the pruning rate, performing parameter fusion on channels in the same cluster and removing redundant channels to obtain the channel pruning model, and fine-tuning the channel pruning model.

[0018] The layer evaluation module is used to evaluate the fine-tuned channel pruning model by freezing the convolutional layer parameters, introducing a linear layer after the convolutional layer and training the linear layer with the training set to evaluate the performance of each convolutional layer.

[0019] The layer pruning module is used to perform layer pruning based on layer evaluation results. This includes comparing the performance of adjacent layers to identify low-performing layers, removing these layers to obtain a layer pruning model, fine-tuning the layer pruning model, and evaluating the performance of the final model using a test set.

[0020] Thirdly, the present invention also provides a computer terminal device, comprising:

[0021] One or more processors;

[0022] A memory, coupled to the processor, for storing one or more programs;

[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the two-stage recoverable model pruning method for automatic modulation recognition in the first aspect described above.

[0024] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the two-stage recoverable model pruning method for automatic modulation recognition described in the first aspect above.

[0025] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the two-stage recoverable model pruning method for automatic modulation recognition described in the first aspect above.

[0026] Compared with the prior art, the present invention has the following advantages and technical effects:

[0027] This invention provides a two-stage recoverable model pruning method for automatic modulation recognition. This invention, through a two-stage recoverable model pruning framework, effectively removes redundant channels and layers in the neural network, significantly reducing the number of model parameters and computational complexity. Simultaneously, it maintains high automatic modulation recognition accuracy under high compression ratios, avoiding performance degradation due to the loss of key features or layer collapse. This invention improves the real-time performance of modulation recognition, reduces inference latency, meets the low-latency signal processing requirements of resource-constrained devices, and enhances the model's deployment adaptability in embedded terminals and edge computing platforms through a layer diagnostic mechanism, ensuring that the compressed model still possesses stable feature extraction capabilities and recognition performance. Attached Figure Description

[0028] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0029] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0032] To address the deployment difficulties of deep learning models in practical applications, this invention proposes a two-stage recoverable model pruning method for automatic modulation recognition. This method is particularly suitable for scenarios such as communication devices, embedded systems, and edge computing nodes, effectively alleviating technical challenges such as excessively large model size, slow inference speed, and high computational and storage resource consumption. This method can effectively reduce the complexity of model deployment on existing hardware platforms, improve inference speed and energy efficiency, thereby enhancing the operational performance of devices in practical applications.

[0033] Specifically, this invention combines channel-level pruning with hierarchical collapse diagnosis to achieve extremely high model compression ratio, excellent performance, and inference efficiency. In the first stage, hierarchical clustering algorithms are used to cluster the channel weights of each layer based on the cosine similarity between channels. Subsequently, model merging techniques are used to fuse parameters from the same cluster to achieve standardized channel pruning. In the second stage, the layer collapse diagnosis module uses linear probing techniques to identify layer collapse and layer folding phenomena caused by high compression ratios, and promptly removes degenerate layers.

[0034] Example 1

[0035] like Figure 1 As shown, this embodiment provides a two-stage recoverable model pruning method for automatic modulation recognition, including:

[0036] S1. Given a dataset and a convolutional neural network model, divide the dataset into a training set and a test set, and use the training set to perform supervised training on the model to obtain a pre-trained model.

[0037] S2. Perform channel pruning on the pre-trained model, including calculating the similarity of convolutional layer channels, performing hierarchical clustering based on similarity to obtain channel clustering, determining the number of target clusters based on the pruning rate, performing parameter fusion on channels in the same cluster and removing redundant channels to obtain the channel pruning model, and fine-tuning the channel pruning model.

[0038] S3. Perform layer evaluation on the fine-tuned channel pruning model, including freezing the convolutional layer parameters, introducing a linear layer after the convolutional layer and training the linear layer using the training set to evaluate the performance of each convolutional layer.

[0039] S4. Perform layer pruning based on the layer evaluation results, including comparing the performance of adjacent layers to identify layers with lower performance, removing these layers to obtain a layer pruning model, fine-tuning the layer pruning model, and evaluating the performance of the final model using a test set.

[0040] Specifically, S1: Given a dataset and a convolutional neural network model, the dataset is divided into a training set and a test set. The training set consists of a large amount of labeled data for supervised training and fine-tuning, while the test set consists of a small batch of labeled data for performance evaluation. The training set is used to perform supervised training on the convolutional neural network model to obtain a pre-trained model.

[0041] S2: Perform channel pruning on the pre-trained model obtained in S1. First, calculate the similarity of the convolutional layer channels of the pre-trained model, and perform hierarchical clustering based on the similarity to obtain channel clustering. Then, determine the number of target clusters according to the set pruning rate, perform parameter fusion on channels belonging to the same cluster, and remove redundant channels to obtain a preliminary pruned channel model. Finally, fine-tune the channel pruned model to restore or improve its performance.

[0042] S3: Perform layer evaluation on the channel pruning model obtained after fine-tuning in S2. First, freeze the parameters of the channel pruning model to keep its convolutional layer parameters unchanged; then, introduce a linear layer after the convolutional layer of the channel pruning model, and train the linear layer using the training set to achieve performance evaluation of each convolutional layer.

[0043] S4: Analyze the performance evaluation results of the layers obtained in S3 to perform layer pruning. First, perform a performance comparison analysis between adjacent layers to identify convolutional layers with relatively low performance. By removing these weaker convolutional layers, a more compact layer pruning model is constructed. This layer pruning model is then fine-tuned to effectively recover the accuracy loss caused by structural changes. Finally, the performance of the fine-tuned layer pruning model is evaluated using a test set to obtain the corresponding performance indicators.

[0044] As one implementation method in this embodiment, in S1, the process of dividing the dataset into a training set and a test set includes: using 80% of the dataset as the training set and 20% as the test set; the process of using the training set to perform supervised training on the model includes: minimizing the cross-entropy loss function to obtain a pre-trained model.

[0045] Furthermore, in step S1, a dataset is typically split into a training set and a test set, represented as follows: and ,in Therefore, 80% of the dataset will be used as the training set. , represented as Use 20% of the dataset as the test set. ,in Indicates a sample, The label is represented by the convolutional neural network model. Include The first convolutional layer. The weight tensor of each convolutional layer can be represented as: in . and Indicates the first The number of output channels and input channels of each convolutional layer and This represents the height and width of the convolution kernel. (Using the training set) For the convolutional neural network Supervised training is performed with the objective of minimizing the cross-entropy loss function to obtain a pre-trained model. .

[0046] As one implementation method in this embodiment, in S2, the process of pruning the channels of the pre-trained model includes: for the output channel of each convolutional layer, vectorizing the weight tensor and calculating the cosine similarity between channels, and converting the similarity into a correlation distance matrix; applying a hierarchical clustering algorithm to divide the channels into multiple clusters according to the pruning rate; performing weighted fusion on the channel weights within the same cluster to retain representative channels and remove redundant channels; and fine-tuning the pruned model.

[0047] Furthermore, in step S2, the channel pruning process is as follows:

[0048] In the output channel similarity calculation, for the th The first convolutional layer Output Channel , Transform its weight tensor into a row vector. ,in .

[0049] The vectorization process is as follows:

[0050] set up:

[0051] (1)

[0052] but:

[0053] (2)

[0054] Then, the cosine similarity is used to calculate the first... With the The similarity of the output channels is calculated using the following formula:

[0055] (3)

[0056] In the formula, Representing the The similarity of each channel is calculated, and then the channel similarity is converted into relevance distance using the following formula:

[0057] (4)

[0058] By traversing all channels, the correlation distance matrix can be obtained. After obtaining the correlation distance matrix, a hierarchical clustering algorithm is applied to divide the channels into groups according to a set pruning rate. The group, the formula is as follows:

[0059] (5)

[0060] In the formula, This is expressed as the pruning rate. Specifically, the hierarchical clustering algorithm first... All channels of a layer are treated as a single sample set. Then each sample Initialize as an independent cluster At this time, there are a total of Given an initial cluster, the average linking method is used to calculate any two clusters. and The distance between them is defined as:

[0061] (6)

[0062] Find the two clusters with the minimum distance in the current distance matrix. and Merge two clusters ,Will The subsequent clusters are shifted one position to the left to ensure consecutive cluster numbers. (Delete distance matrix) Zhongyu Relevant rows and columns, and recalculate. Average link distance to other clusters, updated For each row and column in the corresponding data set, decrement the cluster count by one. Repeat this iteration until the cluster count reaches a certain value. When the time is reached, the clustering process terminates, and the final set of clusters is output. .

[0063] Since channels within the same cluster exhibit high similarity in feature representation, indicating strong information redundancy, model fusion techniques are employed to fuse and calculate the channel weight parameters within the same cluster, compressing the information from redundant channels into representative channels. In this context, the fusion formula is as follows:

[0064] (7)

[0065] After channel fusion operation, the first The number of output channels of each convolutional layer is increased from the original Reduce to Then, the same pruning operation is performed on the input channels of the convolutional layer to construct a preliminary channel pruning model. Next, the channel pruning model is fine-tuned a few times using the training set to obtain the fine-tuned channel pruning model. .

[0066] As one implementation method in this embodiment, in S3, the process of performing layer evaluation on the channel pruning model includes: freezing the convolutional layer parameters of the channel pruning model; extracting the feature representation of each convolutional layer from the training set; connecting a linear classification layer after each convolutional layer; training the linear classification layer using the training set; and verifying the classification accuracy of the linear classification layer using the test set as a layer performance indicator.

[0067] Furthermore, in step S3, the layer evaluation process is as follows:

[0068] During the layer evaluation process, for the channel pruning model The For each candidate layer, we extract its feature representation from the training set. The formula is as follows:

[0069] (8)

[0070] In the formula, express The layer, Represents function composition. This represents the data input. The feature representations of each convolutional layer are then obtained. Then, freeze The parameters are used, and a linear classification layer is connected to each layer using the training set. The linear classifier is trained and then validated using a test set to obtain its classification accuracy. The training objective formula for the linear classification layer is as follows:

[0071] (9)

[0072] In the formula, Represents the loss function. The parameters representing the linear layer, This indicates the actual label.

[0073] As one implementation method in this embodiment, in S4, the process of performing layer pruning based on the layer evaluation results includes: comparing the performance indicators of adjacent layers; if the performance difference is lower than a preset redundancy threshold, the layer is determined to be a weak layer; removing the weak layer to obtain the layer pruning model; fine-tuning the layer pruning model; and using a test set to evaluate the performance of the final pruning model.

[0074] As one implementation method in this embodiment, in S4, the process of comparing the performance of adjacent layers includes: calculating the first... Layer and First If the performance metric difference of a layer is less than the redundancy threshold, then the layer is considered to be... The layer is a layer with relatively weak performance.

[0075] In step S4, the layer pruning process is as follows:

[0076] By comparing the first in the frozen state Layer and First The characteristics of each layer can identify the first layer, which has a smaller impact on overall performance, at a lower computational cost. Layer. Define the first The formula for the impact of layers on model performance is as follows:

[0077] (10)

[0078] In the formula, This is a preset redundancy threshold. Then, layers with weaker discriminative abilities are removed to obtain a hierarchical pruning model. This model is then fine-tuned to obtain the optimized pruning model. , Finally, the performance of the final pruned model is evaluated using the test set to obtain its test accuracy metrics on the target task.

[0079] Based on this, the two-stage recoverable model pruning method for automatic modulation recognition provided by the embodiments of the present invention has the following beneficial effects:

[0080] This method effectively removes redundant channels and layers, significantly reducing both model parameters and computational cost while maintaining high recognition accuracy under high compression ratios. Specifically, even with model compression ratios as high as 80%–90%, this invention maintains recognition performance comparable to the original model. Compared to traditional methods, it does not significantly degrade performance due to the loss of key feature channels in high compression scenarios. By introducing a layer collapse diagnosis mechanism, this method can promptly detect and remove degenerate layers, thereby ensuring stable recognition accuracy and achieving a balance between high compression ratios and high performance.

[0081] (1) Improve the real-time performance of modulation recognition: By channel-level clustering pruning and layer diagnostic pruning, the redundant computation of convolutional layers is effectively reduced, enabling the modulation recognition model to achieve low-latency inference on 5G / 6G base stations and edge devices, meeting the needs of real-time signal processing, and avoiding the impact on communication link quality due to excessive inference delay.

[0082] (2) Enhanced model deployment adaptability: Through two-stage pruning, this invention maintains good portability while significantly reducing the number of parameters and computational load, enabling the model to be successfully deployed in embedded communication terminals, IoT nodes and lightweight edge computing platforms, solving the problem that existing deep learning models are difficult to apply due to limited resources.

[0083] Furthermore, taking a miniature signal sensing node deployed in a smart Internet of Things as an application scenario, the implementation process of this invention will be described in detail. The edge device of this node (equipped with an embedded edge AI chip, with 2GB of available video memory) needs to simultaneously handle multiple tasks, including real-time signal stream capture (occupying approximately 0.5GB of video memory), signal preprocessing (occupying approximately 0.25GB of video memory), and protocol stack and data reporting tasks (occupying approximately 0.25GB of video memory). Therefore, its video memory resources experience real-time dynamic fluctuations (fluctuation range 0.5GB-1GB). The system deploys an adapted, pruned, and optimized lightweight model using the method of this invention to perform real-time modulation identification of the detected radio signals (such as identifying abnormal modulation) and uploads the classification results to the cloud management platform.

[0084] This method effectively overcomes the shortcomings of existing large-scale modulation recognition models, which are either completely undeployable or suffer from excessive power consumption during operation in resource-constrained micro-terminal devices. It preserves the most critical feature extraction capabilities within a limited resource budget. Lower inference latency ensures real-time signal processing; less computation directly translates to lower device power consumption, extending terminal battery life; and lower training costs enable the model to quickly adapt to different devices and scenarios.

[0085] Reference Figure 1 A two-stage recoverable model pruning method for automatic modulation recognition includes the following steps:

[0086] S1: Given a dataset and a neural network model, divide the dataset into a training set and a test set.

[0087] In this embodiment, given the dataset RML2016.10a, the neural network model is determined to be the SigNet50 convolutional neural network used for signal modulation recognition.

[0088] The RML2016.10a dataset contains 11 modulation categories: 8BPSK, QPSK, 8PSK, 16QAM, 64QAM, BFSK, CPFSK, and PAM4, as well as 3 analog modulations: WB-FM, AM-SSB, and AM-DSB. Each sample is an IQ dual-channel dataset, and the modulation categories are uniformly distributed within the range of -20dB to 18dB. The RML2016.10a dataset is divided into training and test sets, denoted as follows: and ,in The ratio is 4:1, with a training set of 176,000 signals and a test set of 44,000 signals, both having the same number of modulation categories. The SigNet50 model consists of 6 residual blocks. Each residual block contains 5 convolutional layers and 1 pooling layer. The training set... For the SigNet50 model Supervised training is used to obtain a pre-trained model. This allows us to obtain baseline model parameters that can be used for subsequent model pruning. Finally, a reference model is obtained. Figure 1 The model on the upper left side of the image.

[0089] S2: Model Perform channel pruning. First, prune the pre-trained model. For each convolutional layer channel, cosine similarity is calculated, and the similarity is converted into correlation distance to perform hierarchical clustering to obtain channel clustering. Then, the number of target clusters is determined according to the set pruning rate, parameter fusion is performed on channels belonging to the same cluster, and redundant channels are removed to obtain a preliminary pruning channel pruning model. Finally, the channel pruning model is fine-tuned to restore or improve its performance.

[0090] This embodiment addresses the issue of channel similarity calculation for the first... For each output channel of a convolutional layer, vectorize its weight tensor. ,in , The similarity between any two output channels is calculated using cosine similarity. Based on the cosine similarity results between all channel pairs, a similarity matrix is ​​constructed and further converted into a relevance distance matrix.

[0091] The specific method for obtaining the correlation distance matrix is ​​shown in formulas (1) to (4). Then, a hierarchical clustering algorithm is applied to the correlation distance matrix to divide the channels into groups according to a set pruning rate. The channel weight parameters of the same cluster are fused and calculated to compress the information of redundant channels into representative channels. The parameters of the representative channels are saved to construct a preliminary channel pruning model. The specific method to obtain the channel pruning model is shown in formulas (5) to (7). Finally, the channel pruning model is fine-tuned a few times using the training set to obtain the fine-tuned channel pruning model. Reference for reference Figure 1 The pruning section in the lower left channel.

[0092] S3: Channel pruning model Layer evaluation is performed. First, the parameters of the channel pruning model are frozen to keep the parameters of its convolutional layers unchanged. Then, a linear layer is introduced after the convolutional layer of the channel pruning model, and the linear layer is trained using the training set, thereby realizing the performance evaluation of each convolutional layer.

[0093] In this embodiment, during the layer evaluation process, the channel pruning model is... The For each candidate layer, we extract its feature representation from the training set. In obtaining the feature representations of each convolutional layer Then, freeze The parameters are used, and a linear classification layer is connected to each layer using the training set. Then, the training set is used to train the linear classification layer. The linear classifier is trained over a short period of time, and finally validated using a test set to obtain its classification accuracy. .

[0094] The specific methods for obtaining the layer evaluation results are shown in formulas (8) and (9). (Refer to the reference.) Figure 1 The lower right layer is pruned. The layer evaluation results are obtained and saved.

[0095] S4: Channel Pruning Model Layer pruning is performed. First, the performance evaluation results are compared and analyzed between adjacent layers to identify convolutional layers with relatively low performance. By removing these weaker convolutional layers, a more compact layer pruning model is constructed. This layer pruning model is then fine-tuned to effectively recover the accuracy loss caused by structural changes. Finally, the performance of the fine-tuned layer pruning model is evaluated using a test set to obtain the corresponding performance indicators.

[0096] In this embodiment, during the layer pruning process, by comparing the features of adjacent layers in the frozen state, the layers with minimal impact on overall performance can be identified with low computational cost. Layer. Then, load the channel pruning model. Layers with weaker discriminative abilities are removed to obtain a hierarchical pruning model. The specific method for obtaining the hierarchical pruning model result is shown in formula (10). The model is then fine-tuned to finally obtain the optimized pruning model. Finally, the performance of the final pruned model is evaluated using the test set to obtain its test accuracy metrics on the target task. (Reference: [reference needed]) Figure 1 Prune the lower right layer and the right side section. Finally, create the final model. Edge segment devices deployed to this node.

[0097] Finally, after testing, the pruned and optimized lightweight model obtained in this embodiment reduces the number of parameters by 96.3%, computational cost (FLOPs) by 95.93%, and real-time inference latency by approximately 40% compared to the original model. This model requires only about 100MB of GPU memory to load, enabling stable operation on edge sensing nodes with a minimum available GPU memory fluctuation of 0.5GB, completely resolving the deployment issue caused by memory overflow in the original model. Under these stringent resource constraints, the model still achieves an overall recognition accuracy of 62.33% on the RML2016.10a test set, meeting the requirements for modulation recognition accuracy in practical applications. This represents a loss of only 1.08 percentage points compared to the original model (63.41%), maintaining excellent performance.

[0098] This embodiment presents a two-stage recoverable model pruning framework for automatic modulation recognition. Compared to existing signal model pruning methods, a two-stage pruning method from fine to coarse is introduced. In the first stage, this method uses hierarchical clustering to cluster the weights of all channels in each convolutional layer, and then uses model merging technology to fuse the parameters within the same cluster to achieve pruning in the width dimension. However, when performing channel pruning, the performance of the pruned model will collapse due to the excessively high channel pruning rate. To solve this problem, a layer diagnostic method is introduced in the second stage of this method. This method uses linear probing to diagnose and remove invalid layers to obtain the optimal model, significantly reducing the number of parameters and computational load while maintaining model performance, thereby making the signal model more adaptable to the deployment requirements of edge devices and exhibiting good performance.

[0099] Example 2

[0100] In this embodiment, a computer terminal device is provided, including:

[0101] One or more processors;

[0102] A memory, coupled to the processor, for storing one or more programs;

[0103] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the two-stage recoverable model pruning method for automatic modulation recognition described above.

[0104] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the two-stage recoverable model pruning method for automatic modulation recognition described above.

[0105] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the two-stage recoverable model pruning method for automatic modulation recognition described above.

[0106] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the two-stage recoverable model pruning method for automatic modulation recognition described above.

[0107] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0108] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0109] This embodiment provides such an apparatus or system. The system, referred to as a two-stage recoverable model pruning system for automatic modulation recognition, includes:

[0110] The model training module is used to divide the dataset into training and test sets given a dataset and a convolutional neural network model, and to use the training set to perform supervised training on the model to obtain a pre-trained model.

[0111] The channel pruning module is used to prune channels in the pre-trained model, including calculating the similarity of convolutional layer channels, performing hierarchical clustering based on similarity to obtain channel clustering, determining the number of target clusters based on the pruning rate, performing parameter fusion on channels in the same cluster and removing redundant channels to obtain the channel pruning model, and fine-tuning the channel pruning model.

[0112] The layer evaluation module is used to evaluate the fine-tuned channel pruning model by freezing the convolutional layer parameters, introducing a linear layer after the convolutional layer and training the linear layer with the training set to evaluate the performance of each convolutional layer.

[0113] The layer pruning module is used to perform layer pruning based on layer evaluation results. This includes comparing the performance of adjacent layers to identify low-performing layers, removing these layers to obtain a layer pruning model, fine-tuning the layer pruning model, and evaluating the performance of the final model using a test set.

[0114] As one implementation method in this embodiment, the model training module includes:

[0115] Data partitioning units are used to divide the dataset into 80% training set and 20% test set;

[0116] The model training unit is used to perform supervised training on the model using the training set, minimizing the cross-entropy loss function to obtain a pre-trained model.

[0117] As one implementation method in this embodiment, the channel pruning module includes:

[0118] The similarity calculation unit is used to vectorize the weight tensor and calculate the cosine similarity between channels for the output channels of each convolutional layer, and convert the similarity into a correlation distance matrix.

[0119] Clustering units are used to divide channels into multiple clusters based on the pruning rate using hierarchical clustering algorithms.

[0120] The fusion pruning unit is used to weight and fuse the channel weights within the same cluster to retain representative channels and remove redundant channels.

[0121] The fine-tuning unit is used to fine-tune the pruned model.

[0122] As one implementation method in this embodiment, the layer evaluation module includes:

[0123] The feature extraction unit is used to freeze the convolutional layer parameters of the channel pruning model and extract the feature representations of each convolutional layer from the training set.

[0124] A linear training unit is used to connect a linear classification layer after each convolutional layer. The linear classification layer is trained using the training set and validated using the test set as a performance metric for the layer.

[0125] As one implementation method in this embodiment, the layer pruning module includes:

[0126] The performance comparison unit is used to compare the performance indicators of adjacent layers. If the performance difference is lower than the preset redundancy threshold, the layer is determined to be a weaker layer.

[0127] Layer removal unit, used to remove weaker layers to obtain a layer pruning model;

[0128] The fine-tuning unit is used to fine-tune the layer pruning model;

[0129] Evaluation unit, used to evaluate the performance of the final pruning model using a test set.

[0130] As one implementation method in this embodiment, the performance comparison unit is specifically used to: calculate the performance index difference between the i-th layer and the (i+1)-th layer; if the difference is less than the redundancy threshold, then the i-th layer is determined to be a weaker layer.

[0131] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0132] The above implementation method solves the problem of two-stage recoverable model pruning for automatic modulation recognition in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0133] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A two-stage recoverable model pruning method for automatic modulation recognition, characterized in that, Includes the following steps: S1. Given a dataset and a convolutional neural network model, divide the dataset into a training set and a test set, and use the training set to perform supervised training on the model to obtain a pre-trained model. S2. Perform channel pruning on the pre-trained model, including calculating the similarity of convolutional layer channels, performing hierarchical clustering based on similarity to obtain channel clustering, determining the number of target clusters based on the pruning rate, performing parameter fusion on channels in the same cluster and removing redundant channels to obtain the channel pruning model, and fine-tuning the channel pruning model. S3. Perform layer evaluation on the fine-tuned channel pruning model, including freezing the convolutional layer parameters, introducing a linear layer after the convolutional layer and training the linear layer using the training set to evaluate the performance of each convolutional layer. S4. Perform layer pruning based on the layer evaluation results, including comparing the performance of adjacent layers to identify layers with lower performance, removing these layers to obtain a layer pruning model, fine-tuning the layer pruning model, and evaluating the performance of the final model using a test set.

2. The method according to claim 1, characterized in that, In S1, the process of dividing the dataset into a training set and a test set includes: using 80% of the dataset as the training set and 20% as the test set; the process of using the training set to perform supervised training on the model includes: minimizing the cross-entropy loss function to obtain a pre-trained model.

3. The method according to claim 1, characterized in that, In S2, the process of pruning the channels of the pre-trained model includes: for the output channel of each convolutional layer, vectorizing the weight tensor and calculating the cosine similarity between channels, and converting the similarity into a correlation distance matrix; applying a hierarchical clustering algorithm to divide the channels into multiple clusters according to the pruning rate; performing weighted fusion on the channel weights within the same cluster to retain representative channels and remove redundant channels; and fine-tuning the pruned model.

4. The method according to claim 1, characterized in that, In S3, the process of evaluating the channel pruning model includes: freezing the convolutional layer parameters of the channel pruning model; extracting the feature representation of each convolutional layer from the training set; connecting a linear classification layer after each convolutional layer; training the linear classification layer using the training set; and verifying the classification accuracy of the linear classification layer using the test set as a layer performance indicator.

5. The method according to claim 1, characterized in that, In S4, the process of performing layer pruning based on layer evaluation results includes: comparing the performance indicators of adjacent layers; if the performance difference is lower than a preset redundancy threshold, the layer is determined to be a weak layer; removing the weak layer to obtain a layer pruning model; fine-tuning the layer pruning model; and using a test set to evaluate the performance of the final pruning model.

6. The method according to claim 5, characterized in that, In S4, the process of comparing the performance of adjacent layers includes: calculating the performance index difference between the i-th layer and the (i+1)-th layer; if the difference is less than the redundancy threshold, the i-th layer is determined to be a weaker layer.

7. A two-stage recoverable model pruning system for automatic modulation recognition, characterized in that, The system for implementing the method of any one of claims 1-6 comprises: The model training module is used to divide the dataset into training and test sets given a dataset and a convolutional neural network model, and to use the training set to perform supervised training on the model to obtain a pre-trained model. The channel pruning module is used to prune channels in the pre-trained model, including calculating the similarity of convolutional layer channels, performing hierarchical clustering based on similarity to obtain channel clustering, determining the number of target clusters based on the pruning rate, performing parameter fusion on channels in the same cluster and removing redundant channels to obtain the channel pruning model, and fine-tuning the channel pruning model. The layer evaluation module is used to evaluate the fine-tuned channel pruning model by freezing the convolutional layer parameters, introducing a linear layer after the convolutional layer and training the linear layer with the training set to evaluate the performance of each convolutional layer. The layer pruning module is used to perform layer pruning based on layer evaluation results. This includes comparing the performance of adjacent layers to identify low-performing layers, removing these layers to obtain a layer pruning model, fine-tuning the layer pruning model, and evaluating the performance of the final model using a test set.

8. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images