An automatic modulation recognition core set selection method based on a forgetting quality score

Abstract

This invention discloses an automatic modulation recognition core set selection method based on Forgot Quality Score (FoQuS), belonging to the interdisciplinary field of wireless communication signal processing and deep learning. Addressing the shortcomings of existing methods in automatic modulation recognition tasks, such as neglecting the time-frequency structure and signal-to-noise ratio distribution of signals and using a single evaluation metric, this invention proposes a method that records the prediction trajectories of samples during multiple training rounds, calculates the number of changes in fusion correctness, the number of rounds of persistent errors, and the ratio of accuracy to loss value using the Forgot Quality Score (FoQuS), and dynamically selects a core set based on this score in a hierarchical manner. Retraining the model using the core set significantly reduces the training data size while maintaining recognition performance, demonstrating strong generalization ability and high practical value.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of wireless communication signal processing and deep learning, and particularly relates to an automatic modulation recognition core set selection method based on forgetting quality scoring. Background Technology

[0002] With the widespread application of deep learning technology in wireless communication, deep learning-based automatic modulation recognition technology has become a mainstream solution in scenarios such as spectrum monitoring, cognitive radio, and electronic countermeasures. This method can directly learn discriminative features from raw signal data, achieving high recognition accuracy in complex environments. However, the excellent performance of deep learning models depends on large-scale labeled datasets. The training process requires multiple iterations through the entire training set and tedious hyperparameter tuning, resulting in huge computational resource consumption and long model development cycles. This problem is particularly prominent in scenarios with limited storage and computing power, such as repeated iterations in scientific research and the deployment of edge devices.

[0003] To reduce training costs, core set selection techniques have been proposed, aiming to select a representative subset from the full dataset. Common strategies include random uniform sampling, gradient norm-based methods, sample forgetting frequency-based methods, and feature space coverage-based methods. These methods perform well in tasks such as image classification, but their direct transfer to automatic modulation recognition tasks has significant limitations. First, radio frequency signals have unique time-frequency structures and physical properties, often accompanied by channel effects such as noise, multipath propagation, and interference, which are fundamentally different from the spatial locality and semantic continuity of natural images. Second, the signal-to-noise ratio (SNR) carried by signal samples has a decisive impact on the model's generalization ability, and the feature distribution of samples differs significantly under different SNR conditions. Existing methods do not fully consider the characteristics of SNR distribution, relying solely on single indicators such as loss value or gradient magnitude to select samples. This easily leads to over-selection of simple samples with high SNR while ignoring difficult samples with low SNR, resulting in a lack of diversity in the selected core set and causing the trained model's performance to deteriorate sharply under complex channel conditions.

[0004] Some existing data filtering methods focus primarily on feature-level selection rather than sample-level importance assessment, leading to significant biases in the filtering results due to differences in feature selection. Other methods, while considering the weighted importance of multiple features, employ a greedy strategy of sequentially eliminating low-importance features, easily overlooking differentiated features and sacrificing dataset diversity. More importantly, these methods fail to incorporate dynamic training information, failing to identify key samples that are repeatedly forgotten or consistently poorly fitted during training. Consequently, the selected core set lacks sufficient support for the model's generalization ability.

[0005] Therefore, facing the dual challenges of the unique properties of signal data and training efficiency in automatic modulation recognition tasks, there is an urgent need for a solution that can address these issues. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes an automatic modulation recognition core set selection method based on forgetting quality scoring, thereby resolving the issues present in the prior art.

[0007] In a first aspect, to achieve the above objective, the present invention provides an automatic modulation recognition core set selection method based on forgetting quality scoring, comprising the following steps:

[0008] S1. Preprocess the original training dataset, including signal amplitude normalization and hierarchical random partitioning, to obtain the training set and test set;

[0009] S2. Use a deep learning model to train on the training set for multiple rounds, and record the predicted label, correctness indicator and cross-entropy loss for each sample in each training round.

[0010] S3. Based on the recorded training trajectory, calculate the forgetting quality score FoQuS for each sample. FoQuS is obtained by fusing three complementary statistics, including the number of changes in the sample prediction accuracy, the number of rounds of continuous prediction errors, and the combination of the sample prediction accuracy and the loss value.

[0011] S4. Sort all samples in descending order according to the FoQuS and dynamically divide them into multiple selection intervals. Adaptively determine the number of samples to be selected in each selection interval according to the target sampling rate. Select samples from each selection interval to form a core set.

[0012] S5. Retrain the model using the core set and evaluate the model performance on the test set.

[0013] Optionally, in S1, the preprocessing procedure includes:

[0014] The IQ data of each signal sample is normalized by amplitude normalization, and standardization is achieved by calculating the mean and standard deviation of the sample.

[0015] A hierarchical random sampling strategy is adopted to divide the dataset into training and test sets to ensure that the distribution of each modulation category and each signal-to-noise ratio level in the training and test sets is consistent with that in the original dataset.

[0016] Optionally, in S2, the process of training and recording trajectories includes:

[0017] Perform multiple rounds of training, and record the model's predicted label for each sample in each training round;

[0018] A correctness indicator for a sample is calculated based on the predicted label and the true label. The correctness indicator is a binary variable used to indicate whether the prediction result is correct.

[0019] Record the cross-entropy loss of the sample in the current training round;

[0020] The sum of the number of correct predictions for each sample and the cross-entropy loss.

[0021] Optionally, in S3, the process of calculating FoQuS includes:

[0022] Calculate the first statistic, which is the number of times the sample prediction accuracy changes, counting the number of times the sample changes from a correct prediction to an incorrect prediction;

[0023] Calculate the second statistic, which is the number of consecutive incorrect predictions of the sample. The second statistic is the maximum number of consecutive incorrect predictions of the sample.

[0024] Calculate the third statistic, which is a combination of the sample prediction accuracy and the loss value, based on the cumulative number of correct predictions and the average loss of the sample.

[0025] Normalize the first, second, and third statistics respectively;

[0026] The three normalized statistics are summed to obtain the FoQuS value.

[0027] Optionally, in S4, the process of dynamically selecting the core set includes:

[0028] Sort all samples in descending order based on their FoQuS values;

[0029] The sorted sample sequence is divided into three selection intervals, including the first selection interval, the second selection interval, and the third selection interval;

[0030] The number of samples selected for each selection interval is adaptively determined based on the target sampling rate, wherein the sampling ratio of each selection interval is different when the target sampling rate is different;

[0031] Samples are selected from each selection interval according to a predetermined sample selection number to form the core set.

[0032] Optionally, in S5, the model training and performance verification process includes:

[0033] Retrain the deep learning model using the selected core set;

[0034] Evaluate the performance of the retrained model on the test set;

[0035] The effectiveness of the core set selection method was verified through comparative experiments.

[0036] Secondly, the present invention also provides an automatic modulation recognition core set selection system based on forgetting quality score, for implementing an automatic modulation recognition core set selection method based on forgetting quality score, the system comprising:

[0037] The preprocessing module is used to preprocess the original training dataset, including signal amplitude normalization and hierarchical random partitioning, to obtain the training set and the test set;

[0038] The training trajectory recording module is used to perform multiple rounds of training on the training set using a deep learning model and record the predicted label, correctness indication, and cross-entropy loss for each sample in each training round.

[0039] The FoQuS calculation module is used to calculate the forgetting quality score FoQuS for each sample based on the recorded training trajectory. The FoQuS is obtained by fusing three complementary statistics, including the number of changes in the sample prediction accuracy, the number of rounds of continuous prediction errors, and the combination of the sample prediction accuracy and the loss value.

[0040] The core set selection module is used to sort all samples in descending order according to the FoQuS and dynamically divide them into multiple selection intervals. It adaptively determines the number of samples to be selected in each selection interval according to the target sampling rate and selects samples from each selection interval to form the core set.

[0041] The model training and validation module is used to retrain the model using the core set and evaluate the model performance on the test set.

[0042] Thirdly, the present invention also provides a computer terminal device, comprising:

[0043] One or more processors;

[0044] A memory, coupled to the processor, for storing one or more programs;

[0045] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the automatic modulation recognition core set selection method based on forgetting quality score in the first aspect above.

[0046] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the automatic modulation recognition core set selection method based on forgetting quality score in the first aspect described above.

[0047] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the automatic modulation recognition core set selection method based on forgetting quality score in the first aspect described above.

[0048] Compared with the prior art, the present invention has the following advantages and technical effects:

[0049] This invention provides an automatic modulation recognition core set selection method based on forgetting quality scoring, which effectively overcomes the limitations of existing core set selection methods in automatic modulation recognition tasks. This invention fully considers the time-frequency structure characteristics and signal-to-noise ratio distribution of signal data and is specifically designed for automatic modulation recognition tasks. By fusing three complementary statistics—the number of changes in sample prediction accuracy, the number of consecutive prediction errors, and the prediction accuracy versus loss value—a multi-dimensional and comprehensive evaluation of sample importance is achieved, avoiding the one-sidedness of evaluation by a single statistical measure. The hierarchical dynamic sampling strategy ensures that the selected core set simultaneously includes key samples near the decision boundary, difficult-to-fit samples, and stable samples that are easy to learn, guaranteeing a balanced distribution and overall diversity of samples under different signal-to-noise ratio conditions. This method significantly reduces the data scale and computational resource overhead required for model training while maintaining recognition performance similar to that of training with full data, demonstrating good cross-model architecture generalization ability and possessing significant practical value in resource-constrained edge computing environments. Attached Figure Description

[0050] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0051] Figure 1 This is a flowchart of the AMR core set selection based on forgetting quality score according to an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of FoQuS score calculation according to an embodiment of the present invention. Detailed Implementation

[0053] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0054] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0055] Example 1

[0056] This invention provides a data processing-based method. The core idea is to obtain three complementary intermediate statistics by recording the prediction trajectory of each sample during training: a first statistic on the number of changes in sample prediction accuracy, a second statistic on the number of rounds of continuous prediction errors, and a third statistic on the sample prediction accuracy and loss value. Based on these statistics, this method employs a dynamic sample selection strategy to form a diverse core set. This method can significantly reduce the size of the required training data while maintaining near-full-data model performance.

[0057] The mathematical goal of core set selection is to construct a compact and efficient subset. (in This allows the model trained on this subset to achieve performance similar to that trained on the full dataset D. The optimization problem can be formally expressed as:

[0058] (1)

[0059] (2)

[0060] (3)

[0061] in, and These represent the model parameters obtained by training on the full dataset D and the core set S, respectively. This represents the loss function; k is the data sampling rate. ; This represents the acceptable performance loss threshold.

[0062] like Figure 1 As shown, in this embodiment, an automatic modulation recognition core set selection method based on forgetting quality scoring is provided, including:

[0063] S1. Preprocess the original training dataset, including signal amplitude normalization and hierarchical random partitioning, to obtain the training set and test set;

[0064] S2. Use a deep learning model to train on the training set for multiple rounds, and record the predicted label, correctness indicator and cross-entropy loss for each sample in each training round.

[0065] S3. Based on the recorded training trajectory, calculate the forgetting quality score FoQuS for each sample. FoQuS is obtained by fusing three complementary statistics, including the number of changes in the sample prediction accuracy, the number of rounds of continuous prediction errors, and the combination of the sample prediction accuracy and the loss value.

[0066] S4. Sort all samples in descending order according to the FoQuS and dynamically divide them into multiple selection intervals. Adaptively determine the number of samples to be selected in each selection interval according to the target sampling rate. Select samples from each selection interval to form a core set.

[0067] S5. Retrain the model using the core set and evaluate the model performance on the test set.

[0068] The specific process includes:

[0069] S1: Dataset partitioning and preprocessing. Given the original training dataset... ,in These are signal samples (usually IQ dual-channel data). The corresponding modulation category label is used. Preprocessing includes signal amplitude normalization, and the dataset is divided into training and test sets according to a preset ratio to ensure that samples of each modulation category and different signal-to-noise ratios are evenly distributed in the two sets.

[0070] S2: Full dataset training and trajectory recording. A deep learning model is trained on the complete training set for T rounds, recording information such as the predicted label, correctness, and cross-entropy loss for each sample in each training round, forming a complete dynamic training trajectory.

[0071] S3: Calculation of the comprehensive quantization value. Based on the sample prediction trajectories recorded during training, three intermediate statistics are calculated:

[0072] 1) The first statistic based on the number of changes in the accuracy of sample predictions;

[0073] 2) A second statistic based on the number of rounds of continuous prediction errors from the sample;

[0074] 3) A third statistic based on sample prediction accuracy and loss value;

[0075] After standardizing the three statistics, a weighted fusion is performed to obtain the final comprehensive quantitative value, which serves as the basis for sample selection.

[0076] S4: Dynamic Core Set Selection Based on Comprehensive Quantization Values. All samples in the training set are sorted in descending order according to their comprehensive quantization values. The sorted samples are divided into multiple selection intervals according to a preset partitioning rule. Based on a preset target sampling rate, the number of samples selected for each selection interval is adaptively determined according to a built-in sampling configuration strategy. Samples are selected from each selection interval according to the determined number, collectively forming the core set used for model training.

[0077] S5: Model training and performance validation based on the selected core set. The model is retrained using the core set dynamically selected in step S4, and its performance is evaluated on the test set. Comparative analysis verifies the effectiveness of this core set selection method, and further generalization experiments across model architectures can be conducted to examine its universality.

[0078] As one implementation method in this embodiment, in S1, the preprocessing process includes:

[0079] The IQ data of each signal sample is normalized by amplitude normalization, and standardization is achieved by calculating the mean and standard deviation of the sample.

[0080] A hierarchical random sampling strategy is adopted to divide the dataset into training and test sets to ensure that the distribution of each modulation category and each signal-to-noise ratio level in the training and test sets is consistent with that in the original dataset.

[0081] Furthermore, the data preprocessing in step S1 includes normalizing the IQ data of each signal sample, using the following normalization formula:

[0082] (4)

[0083] Where x is the original IQ data. and These are the mean and standard deviation of the sample, respectively. The dataset was split using a stratified random sampling strategy to ensure that the proportions of each modulation category and each signal-to-noise ratio level were consistent between the training and test sets.

[0084] As one implementation method in this embodiment, in S2, the process of training and recording the trajectory includes:

[0085] Perform multiple rounds of training, and record the model's predicted label for each sample in each training round;

[0086] A correctness indicator for a sample is calculated based on the predicted label and the true label. The correctness indicator is a binary variable used to indicate whether the prediction result is correct.

[0087] Record the cross-entropy loss of the sample in the current training round;

[0088] The sum of the number of correct predictions for each sample and the cross-entropy loss.

[0089] Furthermore, the training trajectory recording in step S2 specifically includes:

[0090] For each sample Record the model's predicted labels after the t-th training round. And calculate the correctness indicator for the sample:

[0091] (5)

[0092] in This is an indicator function; its value is 1 when the condition is true, and 0 otherwise. It also accumulates the cross-entropy loss and the number of correct predictions for each sample.

[0093] Furthermore, the model training parameter settings in step S2 include:

[0094] The training epochs T are set to 10-50 epochs; the optimizer is Adam or SGD, with a learning rate ranging from 0.0001 to 0.01; the batch size is set to 128-256; and the loss function is cross-entropy loss.

[0095] As one implementation method in this embodiment, in S3, the process of calculating FoQuS includes:

[0096] Calculate the first statistic, which is the number of times the sample prediction accuracy changes, counting the number of times the sample changes from a correct prediction to an incorrect prediction;

[0097] Calculate the second statistic, which is the number of consecutive incorrect predictions of the sample. The second statistic is the maximum number of consecutive incorrect predictions of the sample.

[0098] Calculate the third statistic, which is a combination of the sample prediction accuracy and the loss value, based on the cumulative number of correct predictions and the average loss of the sample.

[0099] Normalize the first, second, and third statistics respectively;

[0100] The three normalized statistics are summed to obtain the FoQuS value.

[0101] Furthermore, the calculation of the FoQuS comprehensive quantization value in step S3 includes three intermediate statistics:

[0102] 1) The first statistic based on the number of changes in the accuracy of sample predictions counts the number of times a sample changes from a correct prediction to an incorrect prediction:

[0103] (6)

[0104] 2) Based on the second statistic of the number of consecutive incorrect prediction rounds of the sample, count the number of consecutive incorrect predictions of the sample:

[0105] (7)

[0106] 3) Based on the third statistic of sample prediction accuracy and loss value, comprehensively consider the stability of correct classification and the magnitude of loss:

[0107] (8)

[0108] in, For example The cumulative number of correct predictions, Let be the cross-entropy loss in round t. This is the balance factor (default is 0.1).

[0109] Ultimately, FoQuS was obtained through comprehensive quantification:

[0110] (9)

[0111] in This indicates a normalization operation.

[0112] Furthermore, the physical meaning of the three intermediate statistics in step S3 is as follows:

[0113] The first statistic, based on the number of times the sample prediction accuracy changes, typically consists of samples located near the class decision boundary. These samples are sensitive to small perturbations in the model parameters, and including such samples helps improve the model's generalization ability and boundary stability. The second statistic, based on the number of consecutive incorrect prediction rounds, typically consists of difficult samples with low signal-to-noise ratios or complex features, which are crucial for the model's robustness in noisy environments. The third statistic, based on the sample prediction accuracy and loss value, typically consists of samples with moderate SNR and clear features, which can provide the model with clear and consistent learning signals, ensuring stable training convergence.

[0114] As one implementation method in this embodiment, in S4, the process of dynamically selecting the core set includes:

[0115] Sort all samples in descending order based on their FoQuS values;

[0116] The sorted sample sequence is divided into three selection intervals, including the first selection interval, the second selection interval, and the third selection interval;

[0117] The number of samples selected for each selection interval is adaptively determined based on the target sampling rate, wherein the sampling ratio of each selection interval is different when the target sampling rate is different;

[0118] Samples are selected from each selection interval according to a predetermined sample selection number to form the core set.

[0119] Furthermore, the dynamic core set selection strategy for the comprehensive quantized value in step S4 is as follows:

[0120] After sorting all samples in descending order according to the comprehensive quantization value (FoQuS), the sample sequence is dynamically divided into three consecutive selection intervals based on a preset division threshold:

[0121] 1) The first selection interval: the first 30% of the sorted sequence of samples;

[0122] 2) The second selection interval: the middle 40% of the sorted sequence of samples;

[0123] 3) The third selection interval: the last 30% of the sorted sequence of samples;

[0124] The division threshold and subsequent sampling ratio for each selection interval are configurable parameters preset by the method, and can be adaptively adjusted according to actual technical requirements such as the target sampling rate, forming a dynamic interval division and quota allocation mechanism based on the sorting position, which together constitute the final core set S.

[0125] Furthermore, the dynamic core set selection strategy of the comprehensive quantization value in step S4 can be dynamically adjusted according to the target sampling rate k:

[0126] When k ≤ 1%, the first selection interval accounts for 10%, the second selection interval accounts for 40%, and the third selection interval accounts for 50%;

[0127] When 1% < k ≤ 5%, the first selection interval accounts for 15%, the second selection interval accounts for 45%, and the third selection interval accounts for 40%;

[0128] When 5% < k ≤ 20%, the first selection interval accounts for 25%, the second selection interval accounts for 50%, and the third selection interval accounts for 25%;

[0129] When k > 20%, the first selection interval accounts for 35%, the second selection interval accounts for 50%, and the third selection interval accounts for 15%.

[0130] This adaptive strategy ensures that key samples are preferentially retained at very low sampling rates and sample diversity is increased at higher sampling rates.

[0131] As an implementation manner in this embodiment, in S5, the process of model training and performance verification includes:

[0132] Retraining the deep learning model using the selected core set;

[0133] Evaluating the performance of the retrained model on the test set;

[0134] Verifying the effectiveness of the core set selection method through comparative experiments.

[0135] Based on this, the present invention provides an automatic modulation recognition core set selection method based on forgetting quality scoring. Compared with the prior art, the present invention has the following advantages:

[0136] 1) Highly targeted: Specifically designed for AMR tasks, it fully considers the special properties of signal data such as time-frequency structure and signal-to-noise ratio distribution, overcoming the limitations of direct transfer from image domain methods;

[0137] 2) Comprehensive evaluation: By combining three complementary statistics, the importance of the sample is explained from multiple perspectives, such as decision boundary stability, coverage of difficult samples, and support for basic learning, thus avoiding the one-sidedness of a single statistic.

[0138] 3) Diversity assurance: A stratified sampling strategy is adopted to ensure that the core set includes both key samples and auxiliary samples, avoiding excessive bias towards a single type and ensuring a balanced distribution of selected samples under different signal-to-noise ratio conditions.

[0139] 4) Significant results: Validated on multiple public AMR datasets, it can maintain recognition accuracy close to that of the full dataset using only 1%-30% of the data. For example, on the RML2016.10a dataset, it achieves an accuracy of 54.10% using 1% of the data, which is significantly better than other benchmark methods.

[0140] 5) Strong generalization ability: It has good cross-architecture generalization ability. The selected core set maintains stable performance on various models such as CNN1D, CNN2D, and LSTM, which proves the versatility of the method.

[0141] 6) High practical value: It can significantly reduce computing and storage overhead, accelerate model development and iteration, and has important application value in resource-constrained edge computing environments, providing an effective solution for the efficient development of AMR tasks.

[0142] S1: Dataset partitioning and preprocessing:

[0143] This embodiment uses the RML2016.10a dataset, which is a widely used benchmark dataset in the field of automatic modulation recognition. The specific parameters of the dataset are as follows:

[0144] Modulation categories: Includes 11 modulation methods, namely 8PSK, BPSK, QPSK, 16QAM, 64QAM, BFSK, CPFSK, PAM4, and 3 analog modulations WB-FM, AM-SSB and AM-DSB;

[0145] Data format: Each sample is IQ dual-channel time-domain data, with 128 sampling points;

[0146] Signal-to-noise ratio range: from -20dB to 18dB, evenly distributed in 2dB intervals, for a total of 20 signal-to-noise ratio levels;

[0147] Data size: A total of 220,000 samples, with 1,000 samples per modulation category at each SNR level.

[0148] The preprocessing steps specifically include:

[0149] 1) Amplitude Normalization: Normalize the IQ data for each sample to eliminate the influence of amplitude bias. The normalization formula is as follows: .

[0150] Where x is the original IQ data. and These are the mean and standard deviation of the sample, respectively.

[0151] 2) Dataset Splitting: The dataset is randomly divided into training and test sets in a 4:1 ratio. A stratified random sampling strategy is used during the split to ensure:

[0152] The proportions of each modulation category are consistent in the training and test sets;

[0153] The samples at each signal-to-noise ratio level are uniformly distributed in the two sets;

[0154] The training set contains 176,000 samples, and the test set contains 44,000 samples.

[0155] This hierarchical partitioning strategy avoids distribution bias introduced by data partitioning, ensuring fairness in subsequent core set selection and model evaluation.

[0156] S2: Full dataset training and trajectory recording;

[0157] This step uses the CNN1D model as the basic architecture to train on the complete training set, while recording detailed training dynamic information.

[0158] (1) Model Architecture: The CNN1D model consists of four convolutional layers, each followed by batch normalization and ReLU activation functions, and finally outputs the probability distribution of 11 classes through a fully connected layer. The specific structure is as follows:

[0159] Conv1D(64,kernel_size=3)→BatchNorm→ReLU→MaxPool

[0160] Conv1D(128,kernel_size=3)→BatchNorm→ReLU→MaxPool

[0161] Conv1D(256,kernel_size=3)→BatchNorm→ReLU

[0162] Conv1D(256,kernel_size=3)→BatchNorm→ReLU→GlobalAvgPool

[0163] Dense(128)→ReLU→Dropout(0.5)

[0164] Dense(11)→Softmax

[0165] (2) Training parameter settings:

[0166] Training rounds T = 50 epochs;

[0167] Optimizer: SGD (Stochastic Gradient Descent), learning rate lr=0.01, momentum=0.9;

[0168] Batch size = 128;

[0169] Loss function: Cross-entropy loss;

[0170] Learning rate scheduling: A cosine annealing strategy is used to periodically adjust the learning rate.

[0171] (3) Trajectory recording mechanism: During the training process, for each sample Record the following information:

[0172] ① Predicted label trajectory: After the t-th epoch, record the model's predicted label for sample i:

[0173] (10)

[0174] in This represents the probability of the c-th class output by the model. The arg max operation selects the class with the highest probability as the predicted label.

[0175] ② Correctness Trajectory: Based on the predicted and true labels, calculate the correctness of the sample in round t:

[0176] (11)

[0177] ③ Loss accumulation: Record the cross-entropy loss of the sample in round t. and the total loss accumulates:

[0178] (12)

[0179] The cross-entropy loss is defined as follows:

[0180] (13)

[0181] That is, the model corresponds to the true category. The negative logarithm of the predicted probability.

[0182] ④ Cumulative number of correct predictions:

[0183] (14)

[0184] This value reflects the total number of times a sample is correctly classified throughout the entire training process.

[0185] Through a complete 50-round training, each sample generates a complete behavioral trajectory that includes the predicted trajectory, the correctness sequence, and the loss record, providing a sufficient data foundation for the subsequent calculation of the FoQuS comprehensive quantization value.

[0186] like Figure 2 As shown, the calculation of the FoQuS composite quantification value includes the calculation of three intermediate statistics and a standardized combination process.

[0187] S3: Calculation of FoQuS composite quantization value:

[0188] Based on the training trajectory recorded in step S2, this step calculates three intermediate statistics and combines them into the final FoQuS comprehensive quantization value.

[0189] (1) Calculation of the first statistic based on the number of changes in the sample prediction accuracy:

[0190] First statistic This indicator is used to capture the number of times a sample changes from a correct prediction to an incorrect prediction during training, reflecting its instability near the decision boundary. The calculation formula is Equation (6). The physical meaning of this indicator is that samples that change from correct predictions to incorrect predictions are usually located near the class decision boundary. These samples are sensitive to small perturbations in the model parameters and are easily "forgotten". Including such samples helps to improve the generalization ability and boundary stability of the model.

[0191] (2) Calculation of the second statistic based on the number of consecutive prediction errors in the sample:

[0192] Second statistic The number of consecutive incorrect predictions of a statistical sample is used to identify stubborn samples that are difficult to fit. The calculation formula is Equation (7). The physical meaning of this indicator is that samples with a high number of consecutive errors are usually difficult samples with low signal-to-noise ratio or complex features. Although these samples are difficult to learn, they are crucial to the robustness of the model in noisy environments because these samples can prevent the core set from containing only easy-to-learn high SNR samples.

[0193] (3) Calculation of the third statistic based on sample prediction accuracy and loss value:

[0194] Third statistic The overall learning value is characterized by taking into account both the stability of the samples being correctly classified and the average loss during training. The calculation formula is Equation (8).

[0195] in: This represents the cumulative number of correct predictions for the sample. The larger the value, the more stable the sample. The second term is the average loss of the sample; the larger the loss, the more difficult it is for the sample to learn. As a balance factor, this embodiment sets =0.1, third statistic The metrics ensure that the core set contains enough easy-to-learn samples to provide stable gradient information for the model. The design philosophy of the third statistic is that not all samples should be "difficult samples"; appropriately easy-to-learn samples help the model converge quickly. Samples with high third statistics are typically those with moderate SNR and clear features, providing clear and consistent learning signals for the model.

[0196] (4) FoQuS composite quantization value combination:

[0197] To eliminate the dimensional differences among the three statistics, we first perform minimization-maximization normalization on each statistic:

[0198] (15)

[0199] Then, the three normalized statistics are summed to obtain the final FoQuS comprehensive quantization value, as shown in formula (9). The advantages of this comprehensive quantization value are: the first statistic focuses on samples near the decision boundary, improving generalization ability; the second statistic focuses on difficult samples, improving noise robustness; and the third statistic focuses on stable learning, ensuring training convergence. The three complement each other, ensuring the comprehensiveness and diversity of the core set.

[0200] In this embodiment, after calculating the FoQuS comprehensive quantization value for 176,000 training samples, the distribution of the comprehensive quantization value shows an approximately normal distribution, ranging from 0.15 to 2.85, with an average value of approximately 1.5.

[0201] S4: Hierarchical Core Set Selection:

[0202] This step selects the core set using a stratified proportional sampling strategy based on the FoQuS comprehensive quantization value, ensuring the diversity and representativeness of the selected samples.

[0203] (1) Sample sorting and stratification:

[0204] All 176,000 training samples were sorted in descending order of their FoQuS quantization values ​​and then divided into three levels:

[0205] The first selection interval is the top 30% of the samples, i.e., the first 52,800 samples. The FoQuS of these samples is between 2.0 and 2.85, and the samples included typically have the following characteristics: moderate first statistic (3-6 times); a certain number of second statistic occurrences (10-25 times); a relatively high first statistic; and a relatively balanced signal-to-noise ratio distribution.

[0206] The second selection interval: the middle 40% of the samples, i.e., the middle 70,400 samples. The FoQuS of these samples are between 1.2 and 2.0, with fewer first statistics (1-3 times), moderate frequency of second statistics (5-15 times), and moderate frequency of third statistics; mainly distributed in the medium-to-high SNR range.

[0207] The third selection interval: the last 30% of samples, i.e., the last 52,800 samples. These samples have a FoQuS between 0.15 and 1.2, and are usually high SNR and easy-to-learn samples; however, they still have some value for basic model training.

[0208] (2) Stratified proportional sampling:

[0209] Set a target sampling rate k (e.g., k=0.05 means selecting 5% of the data), and draw samples from each level according to the following proportions:

[0210] Select 30% of the target quantity from the first selection interval: n_first = 0.30 × k × |D|;

[0211] Select 50% of the target quantity from the second selection interval: n_second = 0.50 × k × |D|;

[0212] Select 20% of the target quantity from the third selection interval: n_third = 0.20 × k × |D|;

[0213] Taking k=0.05 as an example, the target core set size is 176000 × 0.05 = 8800 samples, and the specific allocation is as follows:

[0214] First selection interval: 2640 samples;

[0215] Second selection interval: 4400 samples;

[0216] Third selection interval: 1760 samples;

[0217] Sampling Strategy Explanation: Instead of a simple "select the top k samples globally" strategy, a stratified proportional sampling approach is used to ensure the core set covers samples of varying importance levels. The first selection interval accounts for 30% to ensure sufficient key samples are included; the second selection interval accounts for 50%, forming the main body of the core set and balancing performance and diversity; the third selection interval accounts for 20%, retaining some simple samples to maintain training stability.

[0218] (3) Core set construction:

[0219] Within each level, a corresponding number of samples are selected sequentially from high to low based on the FoQuS comprehensive quantization value, ultimately forming the core set S. In this embodiment, the core set at a 5% sampling rate includes: coverage of all 11 modulation categories, representation of all 20 SNR levels, signal-to-noise ratio distribution that is basically consistent with the original training set, and balanced distribution of samples at different difficulty levels.

[0220] S5: Performance Verification

[0221] The CNN1D model was retrained using the selected core set, and its performance was evaluated on the test set to verify the effectiveness of the core set selection method.

[0222] (1) Core set training:

[0223] The training setup is identical to step S2, including: the same CNN1D model architecture; 50 training epochs; SGD optimizer; learning rate of 0.01; momentum of 0.9; batch size of 128; and cross-entropy loss function. The only difference is that the training dataset is reduced from the full 176,000 samples to a core set of 8,800 samples (5% sampling rate).

[0224] (2) Performance evaluation:

[0225] The model's recognition accuracy was evaluated on 44,000 test samples. Evaluation metrics included:

[0226] Accuracy of the baseline and our own method when selecting 1%, 5%, 10%, 20%, and 30% of the 176,000 samples in the training set, respectively.

[0227] (3) Comparison of experimental results:

[0228] Table 1 shows the comparison results of the method of this invention with 10 benchmark methods on the RML2016.10a dataset.

[0229] Table 1

[0230]

[0231] Ablation experiments verified the complementarity of the three statistics. The complete FoQuS comprehensive quantification value performed better than any single statistic or pairwise combination, demonstrating the importance of diversity statistics.

[0232] The automatic modulation recognition core set selection method based on FoQuS comprehensive statistics of the present invention, compared with the existing core set selection methods, takes into account the special properties of signal data, uses multiple statistics to evaluate the importance of samples, and adopts a hierarchical sampling strategy to ensure diversity. It can maintain good recognition performance while significantly reducing training data, and provides an effective solution for the efficient development of AMR tasks.

[0233] Example 2

[0234] In this embodiment, a computer terminal device is provided, including:

[0235] One or more processors;

[0236] The memory is coupled to the processor and is used to store one or more programs;

[0237] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described automatic modulation recognition core set selection method based on forgetting quality scores.

[0238] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described automatic modulation recognition core set selection method based on forgetting quality score.

[0239] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the above-described automatic modulation recognition core set selection method based on forgetting quality score.

[0240] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described automatic modulation recognition core set selection method based on forgetting quality score.

[0241] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0242] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0243] This embodiment provides such an apparatus or system. The system, referred to as an automatic modulation identification core set selection system based on forgetting quality scoring, includes:

[0244] The preprocessing module is used to preprocess the original training dataset, including signal amplitude normalization and hierarchical random partitioning, to obtain the training set and the test set;

[0245] The training trajectory recording module is used to perform multiple rounds of training on the training set using a deep learning model and record the predicted label, correctness indication, and cross-entropy loss for each sample in each training round.

[0246] The FoQuS calculation module is used to calculate the forgetting quality score FoQuS for each sample based on the recorded training trajectory. The FoQuS is obtained by fusing three complementary statistics, including the number of changes in the sample prediction accuracy, the number of rounds of continuous prediction errors, and the combination of the sample prediction accuracy and the loss value.

[0247] The core set selection module is used to sort all samples in descending order according to the FoQuS and dynamically divide them into multiple selection intervals. It adaptively determines the number of samples to be selected in each selection interval according to the target sampling rate and selects samples from each selection interval to form the core set.

[0248] The model training and validation module is used to retrain the model using the core set and evaluate the model performance on the test set.

[0249] As one implementation method in this embodiment, the preprocessing module includes:

[0250] The normalization unit is used to perform amplitude normalization processing on the IQ data of each signal sample, and achieves standardization by calculating the mean and standard deviation of the sample.

[0251] The partitioning unit is used to divide the dataset into training and test sets using a hierarchical random sampling strategy, ensuring that the distribution of each modulation category and each signal-to-noise ratio level in the training and test sets is consistent with that in the original dataset.

[0252] As one implementation method in this embodiment, the training trajectory recording module includes:

[0253] The prediction label recording unit is used for multiple rounds of training. For each sample, in each training round, it records the model's predicted label.

[0254] The correctness indicator calculation unit is used to calculate the correctness indicator of the sample based on the predicted label and the true label. The correctness indicator is a binary value indicating whether the prediction is correct.

[0255] The loss recording unit is used to record the cross-entropy loss of a sample in the current training round;

[0256] The accumulation unit is used to accumulate the number of correct predictions and the sum of cross-entropy loss for each sample.

[0257] As one implementation method in this embodiment, the FoQuS computing module includes:

[0258] The first statistic calculation unit is used to calculate the first statistic, which is the number of times the sample prediction accuracy changes, and the number of times the sample changes from a correct prediction to an incorrect prediction.

[0259] The second statistic calculation unit is used to calculate the second statistic, which is the number of consecutive incorrect predictions of the sample, and counts the maximum number of consecutive incorrect predictions of the sample.

[0260] The third statistic calculation unit is used to calculate the third statistic, which is a combination of the sample prediction accuracy and the loss value, and is calculated based on the cumulative number of correct predictions and the average loss of the sample.

[0261] The normalization processing unit is used to normalize the first statistic, the second statistic, and the third statistic respectively;

[0262] The fusion unit is used to add the three normalized statistics together to obtain the FoQuS value.

[0263] As one implementation method in this embodiment, the core set selection module includes:

[0264] The sorting unit is used to sort all samples in descending order based on the FoQuS value;

[0265] Interval partitioning unit is used to divide the sorted sample sequence into three selection intervals, including the first selection interval, the second selection interval, and the third selection interval;

[0266] The sampling quantity determination unit is used to adaptively determine the number of samples selected in each selection interval according to the target sampling rate, wherein the sampling ratio of each selection interval is different when the target sampling rate is different;

[0267] The sample selection unit is used to select samples from each selection interval according to a determined number of samples to form the core set.

[0268] As one implementation method in this embodiment, the model training and verification module includes:

[0269] Core set training unit, used to retrain deep learning models using the selected core set;

[0270] The performance evaluation unit is used to evaluate the performance of the retrained model on the test set.

[0271] The verification unit is used to verify the effectiveness of the core set selection method through comparative experiments.

[0272] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0273] The above implementation method solves the problem of selecting the core set for automatic modulation recognition based on forgetting quality score in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0274] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An automatic modulation recognition core set selection method based on forgetting quality scoring, characterized in that, It includes the following steps: S1. Preprocess the original dataset, including signal amplitude normalization and stratified random partitioning, to obtain a training set and a test set; S2. Use a deep learning model to perform multiple rounds of training on the training set, and record the predicted labels, correctness indicators, and cross-entropy losses of each sample in each training round; S3. Based on the recorded training trajectories, calculate the forgetting quality score FoQuS for each sample. The FoQuS is obtained by fusing three complementary statistics, including the number of changes in sample prediction correctness, the number of consecutive rounds of incorrect predictions for the sample, and the combination of sample prediction accuracy and loss value; In S3, the process of calculating FoQuS includes: Calculate the first statistic, which is the number of changes in sample prediction correctness, and count the number of times the sample changes from a correct prediction to an incorrect prediction; Calculate the second statistic, which is the number of consecutive rounds of incorrect predictions for the sample, and count the maximum number of consecutive rounds in which the sample is incorrectly predicted; Calculate the third statistic, which is the combination of sample prediction accuracy and loss value, and calculate it based on the cumulative number of correct predictions and average loss of the sample; Perform normalization processing on the first statistic, the second statistic, and the third statistic respectively; Add the three normalized statistics to obtain the FoQuS value; The calculation of the comprehensive quantization value of FoQuS in S3 includes three intermediate statistics: 1) The first statistic based on the number of changes in sample prediction correctness, which counts the number of times the sample changes from a correct prediction to an incorrect prediction: ； 2) The second statistic based on the number of consecutive rounds of incorrect predictions for the sample, which counts the number of consecutive incorrect predictions for the sample: ； 3) The third statistic based on sample prediction accuracy and loss value, which comprehensively considers the stability of correct classification and the magnitude of loss: ； in, For example The cumulative number of correct predictions, Let be the cross-entropy loss in round t. This is the balance factor (default is 0.1); The final FoQuS is obtained through the comprehensive quantization value: ； in This indicates a normalization operation; S4. Sort all samples in descending order according to the FoQuS, and dynamically divide them into multiple selection intervals. Adaptively determine the number of samples selected from each selection interval according to the target sampling rate, and select samples from each selection interval to form a core set; In S4, the process of dynamically selecting the core set includes: Sort all samples in descending order according to the FoQuS value; Divide the sorted sample sequence into three selection intervals, including the first selection interval, the second selection interval, and the third selection interval; Adaptively determine the number of samples selected from each selection interval according to the target sampling rate, where the sampling ratios of each selection interval are different when the target sampling rates are different; Select samples from each selection interval according to the determined number of samples selected to form a core set; The dynamic core set selection strategy of the comprehensive quantization value in S4 can be dynamically adjusted according to the target sampling rate k: When k ≤ 1%, the first selection interval accounts for 10%, the second selection interval accounts for 40%, and the third selection interval accounts for 50%; When 1% < k ≤ 5%, the first selection interval accounts for 15%, the second selection interval accounts for 45%, and the third selection interval accounts for 40%; When 5% < k ≤ 20%, the first selection interval accounts for 25%, the second selection interval accounts for 50%, and the third selection interval accounts for 25%; When k > 20%, the first choice interval accounts for 35%, the second choice interval accounts for 50%, and the third choice interval accounts for 15%. S5. Retrain the model using the core set and evaluate its performance on the test set.

2. The method according to claim 1, characterized in that, In S1, the preprocessing process includes: The IQ data of each signal sample is normalized by amplitude normalization, and standardization is achieved by calculating the mean and standard deviation of the sample. A hierarchical random sampling strategy is adopted to divide the dataset into training and test sets to ensure that the distribution of each modulation category and each signal-to-noise ratio level in the training and test sets is consistent with that in the original dataset.

3. The method according to claim 1, characterized in that, In S2, the process of training and recording trajectories includes: Perform multiple rounds of training, and record the model's predicted label for each sample in each training round; A correctness indicator for a sample is calculated based on the predicted label and the true label. The correctness indicator is a binary variable used to indicate whether the prediction result is correct. Record the cross-entropy loss of the sample in the current training round; The sum of the number of correct predictions for each sample and the cross-entropy loss.

4. The method according to claim 1, characterized in that, In S5, the model training and performance validation process includes: Retrain the deep learning model using the selected core set; Evaluate the performance of the retrained model on the test set; The effectiveness of the core set selection method was verified through comparative experiments.

5. An automatic modulation recognition core set selection system based on forgetting quality scoring, characterized in that, The system for implementing the method according to any one of claims 1-4 comprises: The preprocessing module is used to preprocess the original training dataset, including signal amplitude normalization and hierarchical random partitioning, to obtain the training set and the test set; The training trajectory recording module is used to perform multiple rounds of training on the training set using a deep learning model and record the predicted label, correctness indication, and cross-entropy loss for each sample in each training round. The FoQuS calculation module is used to calculate the forgetting quality score FoQuS for each sample based on the recorded training trajectory. The FoQuS is obtained by fusing three complementary statistics, including the number of changes in the sample prediction accuracy, the number of rounds of continuous prediction errors, and the combination of the sample prediction accuracy and the loss value. The core set selection module is used to sort all samples in descending order according to the FoQuS and dynamically divide them into multiple selection intervals. It adaptively determines the number of samples to be selected in each selection interval according to the target sampling rate and selects samples from each selection interval to form the core set. The model training and validation module is used to retrain the model using the core set and evaluate the model performance on the test set.

6. A computer terminal device, characterized in that, include: One or more processors; The memory is coupled to the processor and is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images