Satellite jamming signal edge end real-time identification method and system based on knowledge distillation

Abstract

The application discloses a satellite jamming signal edge end real-time identification method and system based on knowledge distillation, and the method comprises the following steps: S1, training a teacher network; S2, constructing a lightweight student network; S3, training the student network by using Hinton knowledge distillation loss; S4, deploying the trained student network to an edge device, and using only the frequency domain one-sided spectrum as input to form a compressed satellite signal after inference. The parameter compression ratio of the application reaches 134 times, and the storage compression ratio (INT8) reaches 510 times, far exceeding the compression limit that can be achieved by the existing structured pruning method; the Top-1 accuracy of the student model on 11 interference identification tasks reaches 84.78%, which is about 6.78 pp higher than the lightweight baseline (about 78%) trained from scratch, and about 4.78 pp higher than the general Hinton distillation baseline (about 80%); only the frequency domain one-sided spectrum needs to be calculated during inference, without calculating STFT, which saves about 1 / 2 of the edge end calculation burden compared with the teacher network.

Description

Technical Field

[0001] This invention relates to the fields of satellite communication, signal processing, deep learning model compression and edge computing, and particularly to a real-time identification method and system for satellite interference signals at the edge based on knowledge distillation. It is applicable to real-time interference identification scenarios of the COSPAS-SARSAT satellite search and rescue system on embedded edge devices such as STM32 microcontrollers, Jetson Nano, and onboard processors of satellite payloads. Background Technology

[0002] The COSPAS-SARSAT satellite search and rescue system's low-orbit satellite payload, ground terminal, and portable beacon equipment are all resource-constrained. Typical computing resources include CPUs with a main frequency of less than 1 GHz and no GPU, typical storage resources include less than 4 MB of flash memory, typical power consumption is constrained to less than 100 mW, and real-time requirements include an identification latency of less than 10 ms to match the real-time detection requirements of a channel data rate of 38.4 kbps.

[0003] One existing technology is to directly deploy large-scale deep learning models such as ResNet18 and ResNet50 on edge devices. This method typically involves millions to tens of millions of parameters, far exceeding the storage capacity of edge devices. The CPU single-sample inference latency is greater than 100 ms, and the power consumption is high, making it difficult to meet the requirements for real-time performance and battery life.

[0004] The second existing technique involves training lightweight networks such as MobileNet and ShuffleNet from scratch. However, this method struggles to learn complex features due to limited model capacity, achieving only about 78% Top-1 accuracy on 11 types of satellite interference identification tasks, and it does not utilize the dark knowledge already learned by larger models.

[0005] The third existing technology is based on the general Hinton knowledge distillation method (see the KD method proposed by Hinton et al. in 2015) to transfer knowledge from a large model to a small model. This method has been verified to be effective in the general image classification field, but its design assumes that teachers and students have the same input modality (i.e., homogeneous distillation), and it has not been optimized for heterogeneous cross-modal distillation scenarios where "teachers are bimodal fusion and students are unimodal"; nor has it designed a student network architecture for long-sequence one-dimensional signals (sampling points ≥ 8192); a simple reuse of this method for the SARSAT interference recognition task results in a Top-1 accuracy of about 80% after distillation, which does not reach the above 84% accuracy required for practical application at the edge, indicating significant room for improvement.

[0006] The fourth existing technology is a model compression method based on channel pruning, low-rank decomposition, or weight sharing. This type of method directly performs structured compression on the teacher network, which makes it difficult to maintain recognition accuracy when the compression ratio exceeds 100 times, and it does not utilize the teacher network's hidden knowledge (i.e., the inter-category similarity information contained in the soft labels) for knowledge transfer. Summary of the Invention

[0007] To address the problems existing in the prior art, the present invention aims to provide a real-time identification method and system for satellite interference signals at the edge based on knowledge distillation. This method transfers the dark knowledge of the multimodal fusion teacher model to the lightweight student network ResNet1D8_Tiny, achieving parameter compression and inference acceleration by more than 100 times, while maintaining a Top-1 accuracy of more than 84% and a low signal-to-noise ratio accuracy of more than 75%.

[0008] To achieve the above objectives, the solution of the present invention is as follows:

[0009] A real-time edge-end identification method for satellite interference signals based on knowledge distillation includes receiving satellite signals, wherein a lightweight single-modal student network is obtained through cross-modal knowledge distillation and deployed on an edge device, and the edge device performs online interference identification on the real-time acquired satellite signals, including the following steps:

[0010] S1. Train the teacher network, which is a multimodal deep learning model based on attention feature fusion (AFF). It extracts frequency domain features and time-frequency domain features through a dual-branch neural network, and adaptively fuses them using the AFF module, outputting a logits vector. ,in Number of interference categories;

[0011] S2. Construct a lightweight student network ResNet1D8_Tiny. The student network includes a two-stage large-kernel stem structure and four residual stages, with a total parameter count of less than 0.2 M. The student network takes only the frequency domain one-sided spectrum as input and outputs logits. ;

[0012] S3. Train the student network using cross-modal knowledge distillation. During training, freeze the teacher network and make it output bimodal inference using both frequency domain one-sided spectrum and time-frequency graph. The student network outputs only using frequency domain one-sided spectral inference. Backpropagation only updates the parameters of the student network, and the student network is trained using Hinton's knowledge distillation loss, which is defined as:

[0013] ;

[0014] in: This refers to the distillation temperature. For soft tag weights, For real category labels; Kullback-Leibler divergence; The logits vector output by the student network; t represents the logits vector output by the teacher network; t and s represent the teacher network and student network, respectively. Cross-entropy loss;

[0015] S4. Deploy the trained student network to an edge device, and have the edge device perform online interference identification on the real-time collected satellite signals.

[0016] The scheme further includes: each of the four residual stages of the student network contains only one TinyBasicBlock1D residual block, and the number of output channels in the four stages are 16, 32, 64, and 128 respectively; the convolution stride of the second, third, and fourth stages is... To achieve feature downsampling.

[0017] The solution further includes: in step S3, the distillation temperature... The soft label weight .

[0018] The solution further includes: In step S1, the multimodal deep learning model is a dual-branch network, which includes a frequency domain branch and a time-frequency domain branch. The frequency domain branch is based on a 1D-ResNet18 network with a dual-level large kernel stem, and the time-frequency domain branch is based on a 2D-ResNet18 network. The multimodal deep learning model with AFF outputs fused features.

[0019] ;

[0020] in:

[0021] Features of fusion; For weighting; This is element-wise multiplication; Frequency domain characteristics; These are time-frequency domain features.

[0022] The solution further includes: the edge device includes at least one of the following: STM32H7 series microcontroller, ESP32-S3 IoT chip, NVIDIA Jetson Nano edge computing module, NVIDIA Jetson TX2 edge computing module, Raspberry Pi series single-board computer and satellite payload onboard processor.

[0023] A further aspect of the solution is: the CPU single-sample inference latency of the student network on the edge device. Storage usage .

[0024] The scheme further states that the total training loss of the student network satisfies: ;

[0025] The student network is trained and quantized using INT8 before being deployed to the edge device. The quantized weights are then... satisfy:

[0026] ;

[0027] in: For quantitative scaling, This is the zero-point offset.

[0028] The scheme further includes: the teacher network takes the frequency domain single-side spectrum and time-frequency graph as dual-mode input, while the student network takes only the frequency domain single-side spectrum as single-mode input, thus forming a heterogeneous cross-modal distillation of "dual-modal teacher → single-modal student". The student network does not need to calculate the short-time Fourier transform (STFT) during the inference phase, which reduces the computational overhead at the edge compared to the teacher network.

[0029] The scheme further includes the following structural features of the student network ResNet1D8_Tiny: (a) the depth is halved from 18 layers to 8 layers compared to the standard 1D-ResNet18 network, and each of the 4 residual stages contains only 1 TinyBasicBlock1D residual block; (b) the number of output channels in the 4 residual stages is reduced from 64, 128, 256, and 512 to 16, 32, 64, and 128 compared to the standard 1D-ResNet18 network; (c) the student network does not contain any attention modules; and (d) the total number of parameters of the student network is less than 0.2 M, and the single-sample floating-point operation is less than 20 M.

[0030] A real-time edge identification system for satellite interference signals implementing the above method includes a teacher model training module, a student network module, a distillation training module, and an edge deployment module, wherein:

[0031] The teacher model training module is used to train the multimodal teacher network based on attention feature fusion (AFF).

[0032] The student network module includes a ResNet1D8_Tiny lightweight architecture;

[0033] The distillation training module is used to train the student network module according to the Hinton distillation loss formula shown below;

[0034] Hinton's distillation loss formula is:

[0035] ;

[0036] in: This refers to the distillation temperature. For soft tag weights, For real category labels; Kullback-Leibler divergence; Cross-entropy loss;

[0037] The edge deployment module is used to export and quantize the student network module to ONNX or TFLite format models.

[0038] The essential difference between this invention and existing Hinton knowledge distillation methods and existing lightweight network designs lies in:

[0039] (1) The present invention realizes heterogeneous cross-modal distillation of "bimodal teacher network (frequency domain + time-frequency graph) → unimodal student network (frequency domain single-sided spectrum only)", which is physically different from the design premise of the existing Hinton distillation that assumes that teachers and students are the same mode. The present invention enables the student network to obtain the dark knowledge of multimodal teachers under the condition of consuming only 1 / 2 of the computing resources (no STFT computing required);

[0040] (2) This invention designs a lightweight network structure ResNet1D8_Tiny for long-sequence one-dimensional satellite signals, and introduces a two-stage large-core stem structure at the front end of the frequency domain branch. To ensure a wide receptive field for long-sequence frequency domain signals, the depth of ResNet18 was halved from 18 layers to 8 layers, and the number of channels in the four stages (64, 128, 256, 512) was reduced to one-quarter (16, 32, 64, 128). The overall number of parameters was reduced from 11.18 M to 0.114 M (compression ratio of 134 times), which is a special design not disclosed in the existing ResNet1D8 series of networks.

[0041] (3) The present invention is achieved through and The optimized hyperparameter combination maintains graceful degradation characteristics under compression of more than 100 times (the difference between students and teachers in 13 signal-to-noise ratio segments is stable at 7 to 10 pp, and no "accuracy collapse" occurs in any signal-to-noise ratio segment), which is a beneficial effect that existing compression methods do not have.

[0042] (4) The present invention combines INT8 post-training quantization, which reduces the storage footprint of the student network on the STM32H7 microcontroller to about 0.12 MB and the single-sample inference latency to 1.20 ms, while meeting the resource constraints of satellite payload and portable beacon equipment.

[0043] The beneficial effects of this invention are:

[0044] (1) The parameter compression ratio reaches 134 times and the storage compression ratio (INT8) reaches 510 times, far exceeding the compression limit that existing structured pruning methods can achieve;

[0045] (2) The CPU single-sample inference latency has been reduced from about 205 ms for teachers to 1.20 ms for students, with a speedup of 170.83 times, which can meet the real-time detection requirements of 38.4 kbps data rate of SARSAT channel (identification latency ≤ 10 ms).

[0046] (3) The student model achieved a Top-1 accuracy of 84.78% on 11 types of interference recognition tasks, which is about 6.78 pp higher than the lightweight baseline trained from scratch (about 78%) and about 4.78 pp higher than the general Hinton distillation baseline (about 80%).

[0047] (4) The student model achieved an accuracy of 75.77% in the low signal-to-noise ratio range (SNR<0 dB), which is 17.32 pp higher than the traditional random forest baseline (58.73%), demonstrating significant robustness.

[0048] (5) The standard deviation of the three independent random seeds is 0.64%, which proves that the distillation process of the present invention has good reproducibility;

[0049] (6) The student model storage occupies 0.46 MB (FP32) or 0.12 MB (INT8), which meets the typical embedded constraint of less than 4 MB of flash memory;

[0050] (7) Only the frequency domain single-sided spectrum needs to be calculated during inference, without the need to calculate the STFT, which saves about 1 / 2 of the edge computing burden compared to the teacher network.

[0051] The invention will be further explained in detail below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0052] Figure 1 This is a three-stage flowchart of the overall distillation deployment of the present invention;

[0053] Figure 2 Hierarchical architecture diagram of the student network ResNet1D8_Tiny;

[0054] Figure 3 A comparison of the SNR-accuracy curves of the teacher AFF model and the student ResNet1D8_Tiny model of this invention in the range of −10 dB to +14 dB signal-to-noise ratio;

[0055] Figure 4 This is a diagram of the end-side deployment architecture of the present invention on satellite payloads or ground terminal edge devices. Detailed Implementation

[0056] This embodiment aims to solve the following technical problems in response to the background technology: (1) How to distill a single-modal student model with only frequency domain single-side spectrum as input from a multimodal fusion teacher model, that is, how to realize heterogeneous cross-modal distillation of "dual-modal teacher → single-modal student"; (2) How can the student network retain high recognition ability for long sequence one-dimensional signals under the constraint of no more than 0.2 M parameters; (3) How to maintain low signal-to-noise ratio robustness under a large model compression of more than 100 times, and avoid uncontrollable behavior of "accuracy collapse" in certain signal-to-noise ratio segments; (4) How to implement the dual constraints of single-sample inference latency ≤ 2 ms and storage occupation ≤ 1 MB on typical embedded microcontrollers such as STM32H7. To this end:

[0057] Example 1:

[0058] A real-time edge-end identification method for satellite interference signals based on knowledge distillation includes receiving satellite signals, wherein a lightweight single-modal student network is obtained through cross-modal knowledge distillation and deployed on an edge device, and the edge device performs online interference identification on the real-time acquired satellite signals, including the following steps:

[0059] S1. Train the teacher network, which is a multimodal deep learning model based on attention feature fusion (AFF). It extracts frequency domain features and time-frequency domain features through a dual-branch neural network, and adaptively fuses them using the AFF module, outputting a logits vector. ,in Number of interference categories;

[0060] The frequency domain feature expression is: ;

[0061] The time-frequency domain feature expression is: ;

[0062] Number of feature channels;

[0063] S2. Construct a lightweight student network ResNet1D8_Tiny. The student network contains a two-stage large-kernel stem structure and four residual stages, with a total parameter count of less than 0.2 M. The student network takes only the frequency domain one-sided spectrum as input and outputs logits. ;

[0064] S3. Train the student network using cross-modal knowledge distillation. During training, freeze the teacher network and make it output bimodal inference using both frequency domain one-sided spectrum and time-frequency graph. The student network outputs only using frequency domain one-sided spectral inference. Backpropagation only updates the parameters of the student network. The student network is trained using Hinton's knowledge distillation loss and based on the teacher network's output logits vector. The distillation loss is defined as...

[0065] ;

[0066] in: Distillation temperature (purely digital hyperparameter, no unit); For soft tag weights, For real category labels; Kullback-Leibler divergence; The logits vector output by the student network; t represents the logits vector output by the teacher network; t and s represent the teacher network and student network, respectively. Cross-entropy loss;

[0067] S4. Deploy the trained student network to an edge device, and have the edge device perform online interference identification on the real-time acquired satellite signals; specifically, for the satellite signals received by the antenna and down-converted to baseband, at a sampling rate... collection Point-time domain signal; for the Perform a Fast Fourier Transform on the point-time domain signal and take the one-sided amplitude spectrum to obtain the frequency domain one-sided spectrum. ;Will The student network deployed on the edge device is used for forward inference to obtain logits. ;Pick The category corresponding to the largest component is used as the identification result of the current satellite interference signal.

[0068] Wherein: each of the four residual stages of the student network contains only one TinyBasicBlock1D residual block, and the number of output channels in the four stages are 16, 32, 64 and 128 respectively; the convolution stride of the second, third and fourth stages is... To achieve feature downsampling.

[0069] In step S3, the distillation temperature The soft label weight .

[0070] In step S1, the multimodal deep learning model is a dual-branch network, which includes a frequency domain branch and a time-frequency domain branch. The frequency domain branch is based on a 1D-ResNet18 network with a dual-level large kernel stem, and the time-frequency domain branch is based on a 2D-ResNet18 network. The multimodal deep learning model with AFF outputs fused features:

[0071] ;

[0072] in:

[0073] Features of fusion; For fusion weights; This is element-wise multiplication; Frequency domain characteristics; These are time-frequency domain features.

[0074] Wherein: the edge device includes at least one of the following: STM32H7 series microcontroller, ESP32-S3 IoT chip, NVIDIA Jetson Nano edge computing module, NVIDIA Jetson TX2 edge computing module, Raspberry Pi series single-board computer and satellite payload onboard processor.

[0075] In the embodiment: CPU single-sample inference latency of the student network on the edge device Storage usage .

[0076] Wherein: the total training loss of the student network satisfies: ;

[0077] The student network is trained and quantized using INT8 before being deployed to the edge device. The quantized weights are then... satisfy:

[0078] ;

[0079] in: For quantitative scaling, This is the zero-point offset.

[0080] In this embodiment: the teacher network takes the frequency domain single-side spectrum and time-frequency graph as dual-mode input, and the student network takes only the frequency domain single-side spectrum as single-mode input, thus forming a heterogeneous cross-modal distillation of "dual-modal teacher → single-modal student". The student network does not need to calculate the short-time Fourier transform (STFT) during the inference stage, which reduces the computational overhead at the edge compared to the teacher network.

[0081] Wherein: the student network ResNet1D8_Tiny satisfies the following structural characteristics: (a) compared to the standard 1D-ResNet18 network, the depth is halved from 18 layers to 8 layers, and each of the 4 residual stages contains only 1 TinyBasicBlock1D residual block; (b) compared to the standard 1D-ResNet18 network, the number of output channels in the 4 residual stages is reduced from 64, 128, 256, 512 to 16, 32, 64, 128; (c) the student network does not contain any attention module; (d) the total number of parameters of the student network is less than 0.2 M, and the single-sample floating-point operation is less than 20 M.

[0082] The essential difference between the above method and existing Hinton knowledge distillation methods and existing lightweight network designs is that: (1) It realizes heterogeneous cross-modal distillation of "bimodal teacher network (frequency domain + time-frequency graph) → unimodal student network (only frequency domain single-side spectrum)", which is physically different from the design premise of existing Hinton distillation that assumes that teachers and students are of the same modality. This method enables the student network to obtain the hidden knowledge of multimodal teachers under the condition of consuming only 1 / 2 of the computing resources (without STFT computing); (2) This method designs a lightweight network structure ResNet1D8_Tiny for long-sequence one-dimensional satellite signals, and introduces a two-level large-core stem structure at the front end of the frequency domain branch. To ensure the receptive field for long-sequence frequency domain signals, the depth of ResNet18 is halved to 8 layers, and the number of channels in the 4 stages (64, 128, 256, 512) is reduced to one-quarter (16, 32, 64, 128). The overall number of parameters is reduced from 11.18 M to 0.114 M (compression ratio of 134 times), which is a special design not disclosed in the existing ResNet1D8 series networks; (3) This method uses and The optimal hyperparameter combination maintains graceful degradation characteristics under compression of more than 100 times (the difference between students and teachers in 13 signal-to-noise ratio segments is stable at 7 to 10 pp, and no "accuracy collapse" occurs in any signal-to-noise ratio segment), which is a beneficial effect that existing compression methods do not have; (4) The present invention combines INT8 post-training quantization, which reduces the storage occupation of the student network on the STM32H7 microcontroller to about 0.12 MB and the single-sample inference latency to 1.20 ms, while meeting the resource constraints of satellite payload and portable beacon equipment.

[0083] The following is further explanation.

[0084] The technical solution of the method described above is divided into four stages:

[0085] Phase 1: Training the teacher model. The teacher model is a multimodal network based on Attention Feature Fusion (AFF), including a frequency domain branch (a 1D-ResNet18 with a dual-level large kernel stem, approximately 3.86 M parameters), a time-frequency domain branch (2D-ResNet18, approximately 11.18 M parameters), an AFF fusion module (approximately 0.26 M parameters), and a classifier Linear(512,11). The teacher model achieves a Top-1 accuracy of 91.56% on the training set and outputs a logits vector for each training sample.

[0086] ;

[0087] For subsequent offline distillation;

[0088] Phase 2: Constructing the student network ResNet1D8_Tiny. Offline soft-object knowledge distillation is employed, meaning the teacher model's parameters are frozen after training convergence, and soft-label supervision is only provided to the student during gradient-free forward passes. The training process only updates the student parameters, thereby reducing the overhead of input acquisition and preprocessing at the terminal. The structure and parameter configuration of the student network are shown in Table 1, with a total parameter count of approximately 0.114 M.

[0089] Table 1. Structure and parameters of the student network ResNet1D8_Tiny

[0090]

[0091] The student network has the following key design features: (a) it retains the dual-level large-core stem structure to ensure high recognition capability for long-sequence one-dimensional signals; (b) each residual stage contains only one TinyBasicBlock1D residual block, with the depth halved from 18 layers in ResNet18 to 8 layers; (c) the number of channels in the four residual stages are 16, 32, 64, and 128 respectively, a reduction to one-quarter compared to the standard ResNet18's 64, 128, 256, and 512; (d) it does not use an attention module, but transmits attention information via soft tags through distillation instruction. The student network uses only the frequency domain single-sided spectrum:

[0092] For input, output logits: ;

[0093] Phase 3: Training the student network using Hinton's knowledge distillation loss; the distillation loss is defined as...

[0094] ;

[0095] in: The distillation temperature. For soft tag weights, For Kullback-Leibler divergence, For cross-entropy loss, This is a true category label. The present invention takes... , During training, the teacher's inference uses bimodal input and does not calculate gradients, while the student's inference uses only frequency domain unilateral spectral input, and backpropagation only updates the student network parameters.

[0096] Phase 4: Deploy the trained student network to edge devices. The specific steps are: export the student model to ONNX or TorchScript format, optionally followed by INT8 quantization for further compression, with the quantized weights satisfying the following requirements.

[0097]

[0098] in For quantitative scaling, It is a zero-point offset; then it runs on the edge device using an inference engine such as ONNX Runtime, TensorFlowLite or TVM; during inference, only the frequency domain single-sided spectrum needs to be calculated without calculating the STFT, further reducing the computational burden at the edge.

[0099] The technical effects of the present invention are shown in Tables 2 and 3;

[0100] Table 2 Model Compression Results

[0101]

[0102] Table 3 Recognition accuracy (graceful degradation)

[0103]

[0104] The gap between the student model and the teacher model remained stable (approximately 7 to 10 pp) under different signal-to-noise ratios, and no uncontrollable behavior of "collapse in certain signal-to-noise ratio ranges" was observed, proving that the distillation process of this method completely preserved the low signal-to-noise ratio robustness of the teacher network.

[0105] Detailed description of the accompanying drawings in the embodiments:

[0106] Figure 1 The numbers (101)–(110) are respectively: (101) Stage 1, i.e., teacher model training; (102) AFF teacher model (15.30 M parameters, Top-1 accuracy 91.56%); (103) Stage 2, i.e., student network distillation training; (104) ResNet1D8_Tiny student model (0.114 M parameters); (105) Soft label distillation loss. (Pick (106) Hard-label cross-entropy loss (107) Total Loss (Pick (108) Phase 3, namely edge deployment, (109) Model export (ONNX / TensorFlow Lite format), (110) Edge device deployment (STM32H7, ESP32-S3, Jetson Nano, Raspberry Pi, etc.).

[0107] Figure 2 The numbers (201)–(214) are respectively: (201) Input frequency domain single-sided spectrum (shape (202) First-level convolutional layer (203) First-level BatchNorm and GELU activation; (204) Second-level convolutional layer (205) Second-level BatchNorm and GELU activation; (206) Max pooling layer (Output shape) (207) First residual stage TinyBasicBlock1D ( Output shape (208) Second residual stage TinyBasicBlock1D including downsampling ( , Output shape (209) The third residual stage TinyBasicBlock1D includes downsampling ( , Output shape (210) Fourth residual stage TinyBasicBlock1D including downsampling ( , Output shape (211) Adaptive Global Average Pooling Layer AdaptiveAvgPool1d (output shape) (212) Flatten (output shape) (213) Linear classification layer (214) Output 11 types of interference recognition logits (shape) ).

[0108] Figure 4 The subscripts (401)–(409) are respectively: (401) satellite payload or ground terminal, (402) antenna receiving SARSAT signal, (403) radio frequency front end (including low noise amplifier and downconverter to downconvert 406 MHz signal to baseband), (404) analog-to-digital sampling (ADS) (405) Embedded processing chip, (406) FFT module (outputs 8192 points of single-sided spectrum), (407) ResNet1D8_Tiny inference engine (deployed on STM32H7, ESP32-S3 or Jetson platform, model size is about 0.46 MB FP32 / 0.12 MB INT8, power consumption is less than 50 mW), (408) Output of 11 types of interference identification results, (409) Subsequent action branches (interference avoidance, reporting to search and rescue center or spectrum monitoring alarm).

[0109] Here are a few examples:

[0110] Example 1, Complete Distillation Training Process:

[0111] The input consists of a pre-trained AFF teacher model and a training set of 38,500 samples (obtained from a total sample size of 55,000, split into 3D hierarchical layers in a 7:2:1 ratio). The output is the distilled-trained student model, ResNet1D8_Tiny. The core pseudocode for training is as follows:

[0112] # Distillation hyperparameters: temperature T=4, soft label weight α=0.7

[0113] T, alpha = 4.0, 0.7

[0114] optimizer = AdamW(student.parameters(),

[0115] lr=1e-3, weight_decay=1e-4),

[0116] for epoch in range(60):

[0117] for X_freq, X_tf, y in train_loader:

[0118] # Step 1: The teacher outputs soft-label logits using bimodal inference (without calculating gradients).

[0119] with no_grad():

[0120] z_T = teacher(X_freq, X_tf),

[0121] # Step 2: Students output logits using only one-sided frequency domain spectral inference.

[0122] z_S = student(X_freq),

[0123] # Step 3: Soft label KL divergence loss (scaling factor T²).

[0124] L_soft = kl_div(log_softmax(z_S / T, dim=1),

[0125] softmax(z_T / T, dim=1),

[0126] reduction='batchmean') * (T * T)

[0127] # Step 4: Hard-label cross-entropy loss,

[0128] L_hard = cross_entropy(z_S, y),

[0129] # Step 5: Calculate the weighted sum of the total loss and backpropagate to update the student parameters

[0130] L = alpha * L_soft + (1 - alpha) * L_hard,

[0131] optimizer.zero_grad(),

[0132] L.backward(),

[0133] optimizer.step().

[0134] Example 2, Edge Inference Deployment:

[0135] Deploy the trained student model to the STM32H7 microcontroller using the following steps:

[0136] First, export the student model as student.onnx using ONNX opset 11;

[0137] The second step is to use a dynamic quantization algorithm to quantize it into an INT8 model student_int8.onnx, with the quantization relation satisfying:

[0138] ;

[0139] The third step involves using the STM32 X-CUBE-AI toolchain to convert student_int8.onnx into C language inference code and integrate it into the STM32 project. The measured CPU single-sample inference latency is 1.20 ms.

[0140] Example 3, Alternative Teacher Configuration Variants:

[0141] The distillation method can be adapted to various teacher configurations, as shown in Table 4. Among them, the student model obtained by the AFF multimodal teacher has the best recognition effect.

[0142] Table 4 Student model recognition performance under different teacher configurations

[0143]

[0144] Example 4, Variant of the number of student network channels.

[0145] The number of channels in the 4-stage student network can be flexibly configured. The parameter quantity and recognition effect of different configurations are shown in Table 5. In this example, the preferred configuration is (16, 32, 64, 128).

[0146] Table 5 Variations of Student Network Channel Count

[0147]

[0148] Example 2:

[0149] A real-time edge identification system for satellite interference signals implementing the method described in Embodiment 1 includes a teacher model training module, a student network module, a distillation training module, and an edge deployment module, wherein:

[0150] The teacher model training module is used to train the multimodal teacher network based on attention feature fusion (AFF).

[0151] The student network module includes a ResNet1D8_Tiny lightweight architecture;

[0152] The distillation training module is used to train the student network module according to the Hinton distillation loss formula shown below;

[0153] Hinton's distillation loss formula is:

[0154] ;

[0155] in: This refers to the distillation temperature; For soft tag weights, For real category labels; Kullback-Leibler divergence; Cross-entropy loss;

[0156] The edge deployment module is used to export and quantize the student network module to ONNX or TFLite format models.

Claims

1. A real-time edge identification method for satellite interference signals based on knowledge distillation, comprising receiving satellite signals, characterized in that, A lightweight unimodal student network is obtained through cross-modal knowledge distillation and deployed on an edge device. The edge device then performs online interference identification on real-time acquired satellite signals, including the following steps: S1. Train the teacher network, which is a multimodal deep learning model based on attention feature fusion (AFF). It extracts frequency domain features and time-frequency domain features through a dual-branch neural network, and adaptively fuses them using the AFF module, outputting a logits vector. ,in Number of interference categories; S2. Construct a lightweight student network ResNet1D8_Tiny. The student network contains a two-stage large-kernel stem structure and four residual stages, with a total parameter count of less than 0.2 M. The student network takes only the frequency domain one-sided spectrum as input and outputs logits. ; S3. Train the student network using cross-modal knowledge distillation. During training, freeze the teacher network and make it output bimodal inference using both frequency domain one-sided spectrum and time-frequency graph. The student network outputs only using frequency domain one-sided spectral inference. Backpropagation only updates the parameters of the student network, and the student network is trained using Hinton's knowledge distillation loss, which is defined as: ； in: This refers to the distillation temperature. For soft tag weights, For real category labels; Kullback-Leibler divergence; The logits vector output by the student network; t represents the logits vector output by the teacher network; t and s represent the teacher network and student network, respectively. Cross-entropy loss; S4. Deploy the trained student network to an edge device, and have the edge device perform online interference identification on the real-time collected satellite signals.

2. The method according to claim 1, characterized in that, Each of the four residual stages of the student network contains only one TinyBasicBlock1D residual block, and the number of output channels in the four stages are 16, 32, 64, and 128, respectively; the convolution stride of stages 2, 3, and 4 is... To achieve feature downsampling.

3. The method according to claim 1, characterized in that, In step S3, the distillation temperature The soft label weight .

4. The method according to claim 1, characterized in that, In step S1, the multimodal deep learning model is a dual-branch network, which includes a frequency domain branch and a time-frequency domain branch. The frequency domain branch is based on a 1D-ResNet18 network with a dual-level large kernel stem, and the time-frequency domain branch is based on a 2D-ResNet18 network. The multimodal deep learning model with AFF outputs fused features: ； in: Features of fusion; For weighting; This is element-wise multiplication; Frequency domain characteristics; These are time-frequency domain features.

5. The method according to claim 1, characterized in that, The edge device includes at least one of the following: STM32H7 series microcontroller, ESP32-S3 IoT chip, NVIDIA Jetson Nano edge computing module, NVIDIA Jetson TX2 edge computing module, Raspberry Pi series single-board computer, and satellite payload onboard processor.

6. The method according to claim 1, characterized in that, The CPU single-sample inference latency of the student network on the edge device Storage usage .

7. The method according to claim 1, characterized in that, The total training loss of the student network satisfies: ; The student network is trained and quantized using INT8 before being deployed to the edge device. The quantized weights are then... satisfy: ； in: For quantitative scaling, This is the zero-point offset.

8. The method according to claim 1, characterized in that, The teacher network takes the bimodal input of the frequency domain single-sided spectrum and the time-frequency graph, while the student network takes only the single-modal input of the frequency domain single-sided spectrum, thus forming a heterogeneous cross-modal distillation of "bimodal teacher → single-modal student". The student network does not need to calculate the short-time Fourier transform (STFT) during the inference phase, which reduces the computational overhead at the edge compared to the teacher network.

9. The method according to claim 1, characterized in that, The student network ResNet1D8_Tiny satisfies the following structural characteristics: (a) compared to the standard 1D-ResNet18 network, the depth is halved from 18 layers to 8 layers, and each of the 4 residual stages contains only 1 TinyBasicBlock1D residual block; (b) compared to the standard 1D-ResNet18 network, the number of output channels in the 4 residual stages is reduced from 64, 128, 256, and 512 to 16, 32, 64, and 128; (c) the student network does not contain any attention modules; (d) the total number of parameters of the student network is less than 0.2 M, and the single-sample floating-point operation is less than 20 M.

10. A real-time satellite interference signal edge identification system implementing the method of any one of claims 1 to 9, characterized in that, It includes a teacher model training module, a student network module, a distillation training module, and an edge deployment module, among which: The teacher model training module is used to train the multimodal teacher network based on attention feature fusion (AFF). The student network module includes a ResNet1D8_Tiny lightweight architecture; The distillation training module is used to train the student network module according to the Hinton distillation loss formula shown below; Hinton's distillation loss formula is: ； in: This refers to the distillation temperature. For soft tag weights, For real category labels; Kullback-Leibler divergence; Cross-entropy loss; The edge deployment module is used to export and quantize the student network module to ONNX or TFLite format models.

Images