An edge-computing-oriented heterogeneous electromagnetic signal multi-modal lightweight identification method and device

Abstract

This invention discloses a lightweight multimodal recognition method and apparatus for heterogeneous electromagnetic signals for edge computing, relating to the field of multimodal recognition technology. The method includes the following steps: S1, acquiring heterogeneous electromagnetic signal data to be recognized; S2, constructing a dual-stream lightweight feature extraction network; S3, training the network model; S4, extracting features; S5, generating a fused feature vector; and S6, outputting the recognition result. This invention employs a uniform inverse residual structure and depthwise separable convolution in both the time and frequency domain feature extraction branches. Compared to traditional large convolutional networks or the serial hybrid architecture of "CNN+LSTM," this method significantly reduces the number of model parameters and floating-point operations while maintaining the feature extraction depth, and eliminates the inference latency caused by the inability of LSTM to be parallelized. This design allows the model to run smoothly on resource-constrained edge devices such as UAV onboard processors, meeting the stringent real-time requirements of battlefield reconnaissance and spectrum monitoring.

Description

Technical Field

[0001] This invention relates to the field of multimodal recognition technology, specifically to a lightweight multimodal recognition method and device for heterogeneous electromagnetic signals oriented towards edge computing. Background Technology

[0002] With the deep integration of UAV technology and electronic information technology, UAV swarms are increasingly being used in battlefield situational awareness, electromagnetic spectrum surveillance, and electronic reconnaissance. In actual operations, UAVs face extremely complex electromagnetic environments, with radar and communication signals often intertwined and coexisting. This dense distribution of heterogeneous radiation sources places extremely high demands on the perception and identification capabilities of signal reconnaissance systems.

[0003] Currently, the technology for electromagnetic signal recognition has gradually shifted from traditional manual feature extraction to deep learning. Mainstream algorithms typically employ deep models such as convolutional neural networks and residual networks to automatically extract features from the I / Q sequence or time-frequency images of the signal. Compared to traditional methods, deep learning models have achieved significant breakthroughs in feature representation capabilities and recognition accuracy, becoming the dominant research direction in the current field of signal processing.

[0004] However, the high-performance deep learning models commonly used in the industry usually have a huge number of parameters and extremely high floating-point operations, which place extremely high demands on hardware resources. Meanwhile, edge devices such as drones and handheld terminals often face strict "resource-constrained" constraints. Their onboard processors have limited computing power, insufficient storage space, and are sensitive to battery life. Directly deploying such heavy networks at edge nodes will lead to excessively high inference latency and a surge in system power consumption, making it difficult to meet the urgent real-time requirements of electromagnetic reconnaissance missions.

[0005] In addition, in order to adapt to edge computing environments, conventional lightweight methods often adopt simple model pruning or compression strategies, but this usually comes at the cost of sacrificing recognition accuracy, especially in low signal-to-noise ratio environments where performance degrades severely.

[0006] Therefore, there is an urgent need for a method to solve the technical bottleneck of the current edge signal recognition, which makes it difficult to achieve both "high accuracy" and "low computing power". Summary of the Invention

[0007] The purpose of this invention is to provide a lightweight identification method and apparatus for heterogeneous electromagnetic signals with multiple modes for edge computing, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a lightweight identification method for heterogeneous electromagnetic signals with multiple modes for edge computing, comprising the following steps:

[0009] S1. Use a broadband radio frequency receiver to acquire heterogeneous electromagnetic signal data to be identified, and construct the first mode data and the second mode data respectively;

[0010] S2. Construct a dual-stream lightweight feature extraction network that includes parallel temporal feature extraction branches and frequency domain feature extraction branches, wherein both the temporal feature extraction branches and the frequency domain feature extraction branches uniformly adopt the improved inverse residual module as the core feature extraction component;

[0011] S3. Construct a mixed standardized dataset containing samples with different signal-to-noise ratios and disturbances, and use the cross-entropy loss function and AdamW optimizer to train the current network end-to-end.

[0012] S4. Input the first modal data and the second modal data into the time domain feature extraction branch and the frequency domain feature extraction branch respectively, and output the time domain features and the frequency domain features;

[0013] S5. After performing global average pooling and concatenation on the time-domain and frequency-domain features, the results are input into the MLP adaptive fusion layer to generate a fused feature vector.

[0014] S6. Input the fused feature vector into the fully connected layer, map the features to the category space, and calculate the probability distribution of the signal to be identified belonging to the preset heterogeneous signal category through the Softmax function. Output the category with the highest confidence as the recognition result.

[0015] Preferably, the construction of the first modal data in S1 specifically includes the following steps:

[0016] S111. Perform quadrature downconversion on the acquired heterogeneous electromagnetic signal data to separate the in-phase components. and orthogonal components Synthesized complex baseband signal ;

[0017] S112. The complex baseband signal is normalized by energy, and the calculation formula is as follows:

[0018] ,

[0019] in, Indicates the average energy of the signal. Indicates a complex baseband signal. This represents the normalized complex baseband signal. Indicates the serial number;

[0020] S113. Using the sliding window method, extract a length of [length missing] from the processed complex baseband signal. The sampling sequence is used to separate the real and imaginary parts, and a sequence of dimensions is constructed. The dual-channel I / Q time series tensor is used as the input data for the time domain branch.

[0021] Preferably, the construction of the second modality data in S1 specifically includes the following steps:

[0022] S121. Perform a short-time Fourier transform on the preprocessed signal and take the magnitude of the transform result. And perform logarithmic augmentation, the calculation formula is:

[0023] ,

[0024] in, This represents a small constant to prevent the logarithm from diverging. This represents the result after logarithmic augmentation.

[0025] S122. After contrast enhancement processing, a high-contrast grayscale time-frequency distribution map is generated. The time-frequency map is then scaled to a uniform size using a bilinear interpolation algorithm. A single-channel grayscale time-frequency image is generated and used as input data for the frequency domain branch.

[0026] S123. Construct a dual-stream lightweight feature extraction network that includes parallel temporal feature extraction branches and frequency domain feature extraction branches. Both the temporal feature extraction branches and the frequency domain feature extraction branches uniformly adopt an improved inverse residual module as the core feature extraction component.

[0027] Preferably, the improved inverted residual module in S2 specifically includes the following steps:

[0028] S201, utilizing a core size of The convolutional layer will increase the number of channels of the input feature map. Expand to ,in Denotes the expansion factor, and The convolutional layer is followed by the ReLU6 activation function, which is used to map low-dimensional features to a high-dimensional space;

[0029] S202, Apply depthwise convolution kernels to the expanded... Each channel is spatially filtered independently without inter-channel information exchange. The deep convolutional layer containing the deep convolutional kernel is followed by the ReLU6 activation function.

[0030] S203. Perform global average pooling on the feature map to obtain channel descriptors, and then pass them through two fully connected layers to generate normalized descriptors. The weight coefficients of the intervals are determined, and these weight coefficients are multiplied element-wise by the feature channels to obtain the output of the linear dimension reduction projection;

[0031] S204, utilizing a core size of The convolutional layer will increase the number of channels from Compress back to output dimension Determine the number of input channels for the current module. Is it equal to the number of output channels? Furthermore, if the convolution stride is 1, then the module input and the output of the linear dimension reduction projection are added element-wise to obtain the final output.

[0032] Preferably, step S3, which uses the cross-entropy loss function and the AdamW optimizer to perform end-to-end training of the current network, specifically includes the following steps:

[0033] S301. The cross-entropy loss function is used to calculate the difference between the predicted probability distribution of the model output and the true label distribution in the dataset, and this difference is used as the objective of backpropagation optimization. The cross-entropy loss function is specifically as follows:

[0034] ,

[0035] in, This represents the total loss value of the current batch of training data. This indicates the number of samples contained in a training batch. This represents the total number of preset categories of heterogeneous signals. Indicates the first Each sample belongs to category The true label, The first output of the model represents the... Each sample belongs to category The predicted probability, Both represent serial numbers;

[0036] S302. The AdamW optimizer is used for parameter updates, with the initial learning rate set to 0.001. At the same time, a cosine annealing strategy is introduced to dynamically adjust the learning rate in order to accelerate model convergence and find the global optimum in the later stages of training.

[0037] Preferably, step S4 specifically includes the following steps:

[0038] S401. Input the first modality data into the time domain feature extraction branch. After downsampling through a one-dimensional convolutional layer, the branch stacks multiple improved inverse residual modules in series. The module uses a one-dimensional deep convolutional kernel and slides on the time axis to extract the instantaneous phase jump and amplitude evolution features of the signal.

[0039] S402. Input the second modality data into the frequency domain feature extraction branch. After downsampling through a two-dimensional convolutional layer, the branch stacks multiple improved inverse residual modules in series. The module uses a two-dimensional deep convolutional kernel to slide on the time-frequency plane to extract the energy distribution pattern and visual texture features of the signal.

[0040] Preferably, step S5 specifically includes the following steps:

[0041] S501. Perform global average pooling on the time-domain feature map output by the time-domain feature extraction branch and the frequency-domain feature map output by the frequency-domain feature extraction branch respectively, compress them into one-dimensional time-domain feature vector and frequency-domain feature vector, perform feature channel concatenation operation, and construct multimodal joint feature vector.

[0042] S502. Construct an MLP adaptive fusion layer, which includes a dimension reduction fully connected layer, a ReLU activation layer, and a dimension increase fully connected layer. After inputting the multimodal joint feature vector into the MLP adaptive fusion layer, a fused feature vector is generated.

[0043] The aforementioned lightweight identification device for heterogeneous electromagnetic signals with multimodal characteristics for edge computing includes a radio frequency front-end module, a memory, a processor, a communication interface, and a power management module.

[0044] The radio frequency front-end module is configured as the physical sensing interface of the system to receive heterogeneous electromagnetic signals in the external environment. Its internal circuitry includes, but is not limited to, a low-noise amplifier, a bandpass filter, and an analog-to-digital converter.

[0045] The memory can be volatile or non-volatile.

[0046] The processor uses a low-power embedded computing chip, including but not limited to an ARM architecture central processing unit, an edge graphics processor, a digital signal processor, a field-programmable gate array, and a neural network acceleration unit;

[0047] The communication interfaces include UART, SPI, I2C, CAN bus interfaces, and wireless transmission;

[0048] The power management module includes a voltage regulator circuit, a power conversion chip, and a power management integrated circuit.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] 1. This invention adopts an inverse residual structure and depthwise separable convolution in both the time and frequency domain feature extraction branches. Compared with traditional large convolutional networks or the serial hybrid architecture of "CNN+LSTM", this method maintains the feature extraction depth while significantly reducing the number of model parameters and floating-point operations, and eliminates the inference delay caused by the inability of LSTM to be parallelized. This design enables the model to run smoothly on resource-constrained edge devices such as UAV onboard processors, meeting the stringent real-time requirements of battlefield reconnaissance and spectrum monitoring.

[0051] 2. This invention constructs a dual-stream complementary mechanism, which uses one-dimensional inverse residuals to accurately capture microsecond-level phase transitions and temporal evolutions in I / Q sequences, and uses two-dimensional inverse residuals to efficiently extract the global energy distribution and texture structure of time-frequency images. On this basis, linear output protects the structural information of weak signals in the feature manifold from being destroyed, and adaptive channel weighting is used to actively suppress background noise. This deep complementarity of "temporal + vision" and the refined reconstruction of key features significantly improve the recognition accuracy of the system in environments with intertwined radar and communication signals and low signal-to-noise ratio.

[0052] 3. In the network structure design, this invention fully considers the memory access cost and parallel computing characteristics of edge hardware. By abandoning the computationally intensive standard convolution and adopting the efficient inverse residual operator, it significantly reduces system power consumption and extends the flight time of the UAV while ensuring high recognition accuracy. Attached Figure Description

[0053] Figure 1 An overall method flowchart is provided for embodiments of the present invention;

[0054] Figure 2 A schematic diagram of the overall architecture of the dual-stream lightweight feature extraction network is provided for embodiments of the present invention;

[0055] Figure 3 A schematic diagram of the improved internal structure of the reverse residual module is provided for embodiments of the present invention;

[0056] Figure 4 A hardware structure block diagram of a lightweight identification device for heterogeneous electromagnetic signals is provided for embodiments of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1:

[0059] Please see Figures 1 to 4 This invention provides a technical solution: a lightweight identification method for heterogeneous electromagnetic signals with multiple modes for edge computing, comprising the following steps:

[0060] S1. Use a broadband radio frequency receiver to acquire heterogeneous electromagnetic signal data to be identified, and construct the first mode data and the second mode data respectively;

[0061] S2. Construct a dual-stream lightweight feature extraction network that includes parallel temporal feature extraction branches and frequency domain feature extraction branches, wherein both the temporal feature extraction branches and the frequency domain feature extraction branches uniformly adopt the improved inverse residual module as the core feature extraction component;

[0062] S3. Construct a mixed standardized dataset containing samples with different signal-to-noise ratios and disturbances, and use the cross-entropy loss function and AdamW optimizer to train the current network end-to-end.

[0063] S4. Input the first modal data and the second modal data into the time domain feature extraction branch and the frequency domain feature extraction branch respectively, and output the time domain features and the frequency domain features;

[0064] S5. After performing global average pooling and concatenation on the time-domain and frequency-domain features, the results are input into the MLP adaptive fusion layer to generate a fused feature vector.

[0065] S6. Input the fused feature vector into the fully connected layer, map the features to the category space, and calculate the probability distribution of the signal to be identified belonging to the preset heterogeneous signal category through the Softmax function. Output the category with the highest confidence as the recognition result.

[0066] The construction of the first modal data in S1 specifically includes the following steps:

[0067] S111. Perform quadrature downconversion on the acquired heterogeneous electromagnetic signal data to separate the in-phase components. and orthogonal components Synthesized complex baseband signal ;

[0068] S112. The complex baseband signal is normalized by energy, and the calculation formula is as follows:

[0069] ,

[0070] in, Indicates the average energy of the signal. Indicates a complex baseband signal. This represents the normalized complex baseband signal. Indicates the serial number;

[0071] S113. Using the sliding window method, extract a length of [length missing] from the processed complex baseband signal. The sampling sequence is used to separate the real and imaginary parts, and a sequence of dimensions is constructed. The dual-channel I / Q time series tensor is used as the input data for the time domain branch.

[0072] The construction of the second modality data in S1 includes the following steps:

[0073] S121. Perform a short-time Fourier transform on the preprocessed signal and take the magnitude of the transform result. And perform logarithmic augmentation, the calculation formula is:

[0074] ,

[0075] in, This represents a small constant to prevent the logarithm from diverging. This represents the result after logarithmic augmentation.

[0076] S122. After contrast enhancement processing, a high-contrast grayscale time-frequency distribution map is generated. The time-frequency map is then scaled to a uniform size using a bilinear interpolation algorithm. A single-channel grayscale time-frequency image is generated and used as input data for the frequency domain branch.

[0077] S123. Construct a dual-stream lightweight feature extraction network that includes parallel temporal feature extraction branches and frequency domain feature extraction branches. Both the temporal and frequency domain feature extraction branches uniformly adopt the improved inverse residual module as the core feature extraction component.

[0078] The improved inverted residual module in S2 specifically includes the following steps:

[0079] S201, utilizing a core size of The convolutional layer will increase the number of channels of the input feature map. Expand to ,in Denotes the expansion factor, and The convolutional layer is followed by the ReLU6 activation function, which is used to map low-dimensional features to a high-dimensional space;

[0080] S202, Apply depthwise convolution kernels to the expanded... Each channel is spatially filtered independently without inter-channel information exchange. The deep convolutional layer containing the deep convolutional kernel is followed by the ReLU6 activation function.

[0081] S203. Perform global average pooling on the feature map to obtain channel descriptors, and then pass them through two fully connected layers to generate normalized descriptors. The weight coefficients of the intervals are determined, and these weight coefficients are multiplied element-wise by the feature channels to obtain the output of the linear dimension reduction projection;

[0082] S204, utilizing a core size of The convolutional layer will increase the number of channels from Compress back to output dimension Determine the number of input channels for the current module. Is it equal to the number of output channels? Furthermore, if the convolution stride is 1, then the module input and the output of the linear dimension reduction projection are added element-wise to obtain the final output.

[0083] S3 employs the cross-entropy loss function and the AdamW optimizer to perform end-to-end training on the current network, specifically including the following steps:

[0084] S301. The cross-entropy loss function is used to calculate the difference between the predicted probability distribution of the model output and the true label distribution in the dataset, and this difference is used as the objective of backpropagation optimization. The cross-entropy loss function is as follows:

[0085] ,

[0086] in, This represents the total loss value of the current batch of training data. This indicates the number of samples contained in a training batch. This represents the total number of preset categories of heterogeneous signals. Indicates the first Each sample belongs to category The true label, The first output of the model represents the... Each sample belongs to category The predicted probability, Both represent serial numbers;

[0087] S302. The AdamW optimizer is used for parameter updates, with the initial learning rate set to 0.001. At the same time, a cosine annealing strategy is introduced to dynamically adjust the learning rate in order to accelerate model convergence and find the global optimum in the later stages of training.

[0088] S4 specifically includes the following steps:

[0089] S401. Input the first modality data into the time domain feature extraction branch. After downsampling through a one-dimensional convolutional layer, the branch stacks multiple improved inverse residual modules in series. The module uses a one-dimensional deep convolutional kernel and slides on the time axis to extract the instantaneous phase jump and amplitude evolution features of the signal.

[0090] S402. Input the second modality data into the frequency domain feature extraction branch. After downsampling through a two-dimensional convolutional layer, the branch stacks multiple improved inverse residual modules in series. The module uses a two-dimensional deep convolutional kernel to slide on the time-frequency plane to extract the energy distribution pattern and visual texture features of the signal.

[0091] S5 specifically includes the following steps:

[0092] S501. Perform global average pooling on the time-domain feature map output by the time-domain feature extraction branch and the frequency-domain feature map output by the frequency-domain feature extraction branch respectively, compress them into one-dimensional time-domain feature vector and frequency-domain feature vector, perform feature channel concatenation operation, and construct multimodal joint feature vector.

[0093] S502. Construct an MLP adaptive fusion layer. The fusion layer includes a dimension reduction fully connected layer, a ReLU activation layer, and a dimension increase fully connected layer. After inputting the multimodal joint feature vector into the MLP adaptive fusion layer, a fused feature vector is generated.

[0094] Example 2:

[0095] The present invention also provides a lightweight identification device for heterogeneous electromagnetic signals with multiple modes for edge computing, including an RF front-end module, a memory, a processor, a communication interface and a power management module;

[0096] The radio frequency front-end module is configured as the physical sensing interface of the system to receive heterogeneous electromagnetic signals in the external environment. Its internal circuitry includes, but is not limited to, low-noise amplifiers, bandpass filters, and analog-to-digital converters.

[0097] Its function is to filter and amplify the weak analog radio frequency signals acquired, and convert them into digital baseband signals or digital intermediate frequency signals, which are then transmitted to the processor or stored in the memory cache via the system bus.

[0098] The memory includes both volatile and non-volatile memory. Its functions are: to store the system startup program and computer-executable instructions; to store the weight parameters of the pre-trained dual-stream lightweight feature extraction network model; to open up a data acquisition buffer area for temporary storage of the original digital sequence acquired by the radio frequency front end and intermediate data (such as the STFT spectrum matrix) during the preprocessing process.

[0099] The processor uses low-power embedded computing chips, including but not limited to ARM architecture central processing units, edge graphics processors, digital signal processors, field-programmable gate arrays, and neural network acceleration units.

[0100] When the processor executes the computer program stored in memory, the specific steps are as follows: send gain control or frequency switching instructions to the RF front-end module, acquire the digital sequence of the environmental electromagnetic signal, read data from the buffer, perform orthogonal downconversion and normalization to obtain a standardized I / Q sequence, simultaneously perform short-time Fourier transform and logarithmic enhancement to obtain a grayscale time-frequency image, call the lightweight model weights in memory, run the feature extraction network, extract the time-domain features of the I / Q sequence and the frequency-domain features of the time-frequency image, respectively, perform weighted combination of heterogeneous features through the MLP fusion layer, calculate the Softmax classification probability, and output the final signal category label (e.g., "LFM radar signal detected, confidence level 98%" or "OFDM communication signal detected, confidence level 95%").

[0101] The communication interfaces include UART, SPI, I2C, CAN bus interfaces, and wireless transmission. Their function is to transmit the identification results or category tags output by the processor back to external devices (such as UAV flight controllers or ground command stations) in real time to assist in path planning or interference avoidance decisions.

[0102] The power management module includes a voltage regulator circuit, a power conversion chip, and a power management integrated circuit. Its function is to connect to an external power supply (such as a drone battery), convert the voltage to the stable voltage required by each module (such as 3.3V, 1.8V, 1.2V), and provide power support for the RF front end, processor, memory, and communication interface.

[0103] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0104] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A lightweight multimodal identification method for heterogeneous electromagnetic signals for edge computing, characterized in that, The method includes the following steps: S1. Use a broadband radio frequency receiver to acquire heterogeneous electromagnetic signal data to be identified, and construct the first mode data and the second mode data respectively; S2. Construct a dual-stream lightweight feature extraction network that includes parallel temporal feature extraction branches and frequency domain feature extraction branches, wherein both the temporal feature extraction branches and the frequency domain feature extraction branches uniformly adopt the improved inverse residual module as the core feature extraction component; S3. Construct a mixed standardized dataset containing samples with different signal-to-noise ratios and disturbances, and use the cross-entropy loss function and AdamW optimizer to train the current network end-to-end. S4. Input the first modal data and the second modal data into the time domain feature extraction branch and the frequency domain feature extraction branch respectively, and output the time domain features and the frequency domain features; S5. After performing global average pooling and concatenation on the time-domain and frequency-domain features, the results are input into the MLP adaptive fusion layer to generate a fused feature vector. S6. Input the fused feature vector into the fully connected layer, map the features to the category space, and calculate the probability distribution of the signal to be identified belonging to the preset heterogeneous signal category through the Softmax function. Output the category with the highest confidence as the recognition result. The improved inverted residual module in S2 specifically includes the following steps: S201, utilizing a core size of The convolutional layer will increase the number of channels of the input feature map. Expand to ,in Denotes the expansion factor, and The convolutional layer is followed by the ReLU6 activation function, which is used to map low-dimensional features to a high-dimensional space; S202, Apply depthwise convolution kernels to the expanded... Each channel is spatially filtered independently without inter-channel information exchange. The deep convolutional layer containing the deep convolutional kernel is followed by the ReLU6 activation function. S203. Perform global average pooling on the feature map to obtain channel descriptors, and then pass them through two fully connected layers to generate normalized descriptors. The weight coefficients of the intervals are determined, and these weight coefficients are multiplied element-wise by the feature channels to obtain the output of the linear dimension reduction projection; S204, utilizing a core size of The convolutional layer will increase the number of channels from Compress back to output dimension Determine the number of input channels for the current module. Is it equal to the number of output channels? Furthermore, if the convolution stride is 1, then the module input and the output of the linear dimension reduction projection are added element-wise to obtain the final output.

2. The lightweight identification method for heterogeneous electromagnetic signals with multimodal characteristics oriented towards edge computing according to claim 1, characterized in that, The construction of the first modal data in S1 specifically includes the following steps: S111. Perform quadrature downconversion on the acquired heterogeneous electromagnetic signal data to separate the in-phase components. and orthogonal components Synthesized complex baseband signal ; S112. The complex baseband signal is normalized by energy, and the calculation formula is as follows: in, Indicates the average energy of the signal. Indicates a complex baseband signal. This represents the normalized complex baseband signal. Indicates the serial number; S113. Using the sliding window method, extract a length of [length missing] from the processed complex baseband signal. The sampling sequence is used to separate the real and imaginary parts, and a sequence of dimensions is constructed. The dual-channel I / Q time series tensor is used as the input data for the time domain branch.

3. The lightweight identification method for heterogeneous electromagnetic signals with multimodal characteristics according to claim 1, characterized in that, The construction of the second modality data in S1 specifically includes the following steps: S121. Perform a short-time Fourier transform on the preprocessed signal and take the magnitude of the transform result. And perform logarithmic augmentation, the calculation formula is: ， in, This represents a small constant to prevent the logarithm from diverging. This represents the result after logarithmic augmentation. S122. After contrast enhancement processing, a high-contrast grayscale time-frequency distribution map is generated. The time-frequency map is then scaled to a uniform size using a bilinear interpolation algorithm. A single-channel grayscale time-frequency image is generated and used as input data for the frequency domain branch. S123. Construct a dual-stream lightweight feature extraction network that includes parallel temporal feature extraction branches and frequency domain feature extraction branches. Both the temporal feature extraction branches and the frequency domain feature extraction branches uniformly adopt an improved inverse residual module as the core feature extraction component.

4. The lightweight identification method for heterogeneous electromagnetic signals with multimodal characteristics for edge computing as described in claim 1, characterized in that, The S3 step, which uses the cross-entropy loss function and the AdamW optimizer to perform end-to-end training on the current network, specifically includes the following steps: S301. The cross-entropy loss function is used to calculate the difference between the predicted probability distribution of the model output and the true label distribution in the dataset, and this difference is used as the objective of backpropagation optimization. The cross-entropy loss function is specifically as follows: ， in, This represents the total loss value of the current batch of training data. This indicates the number of samples contained in a training batch. This represents the total number of preset categories of heterogeneous signals. Indicates the first Each sample belongs to category The true label, The first output of the model represents the... Each sample belongs to category The predicted probability, Both represent serial numbers; S302. The AdamW optimizer is used for parameter updates, with the initial learning rate set to 0.

001. At the same time, a cosine annealing strategy is introduced to dynamically adjust the learning rate in order to accelerate model convergence and find the global optimum in the later stages of training.

5. A lightweight identification method for heterogeneous electromagnetic signals with multimodal characteristics for edge computing according to claim 1, characterized in that, S4 specifically includes the following steps: S401. Input the first modality data into the time domain feature extraction branch. After downsampling through a one-dimensional convolutional layer, the branch stacks multiple improved inverse residual modules in series. The module uses a one-dimensional deep convolutional kernel and slides on the time axis to extract the instantaneous phase jump and amplitude evolution features of the signal. S402. Input the second modality data into the frequency domain feature extraction branch. After downsampling through a two-dimensional convolutional layer, the branch stacks multiple improved inverse residual modules in series. The module uses a two-dimensional deep convolutional kernel to slide on the time-frequency plane to extract the energy distribution pattern and visual texture features of the signal.

6. A lightweight identification method for heterogeneous electromagnetic signals with multimodal characteristics for edge computing according to claim 1, characterized in that, S5 specifically includes the following steps: S501. Perform global average pooling on the time-domain feature map output by the time-domain feature extraction branch and the frequency-domain feature map output by the frequency-domain feature extraction branch respectively, compress them into one-dimensional time-domain feature vector and frequency-domain feature vector, perform feature channel concatenation operation, and construct multimodal joint feature vector. S502. Construct an MLP adaptive fusion layer, which includes a dimension reduction fully connected layer, a ReLU activation layer, and a dimension increase fully connected layer. After inputting the multimodal joint feature vector into the MLP adaptive fusion layer, a fused feature vector is generated.

7. A lightweight identification device for heterogeneous electromagnetic signals with multimodal characteristics for edge computing, characterized in that, The heterogeneous electromagnetic signal multimodal lightweight identification device is applicable to the heterogeneous electromagnetic signal multimodal lightweight identification method for edge computing as described in any one of claims 1-6, and includes an RF front-end module, a memory, a processor, a communication interface, and a power management module. The radio frequency front-end module is configured as the physical sensing interface of the system to receive heterogeneous electromagnetic signals in the external environment. Its internal circuitry includes, but is not limited to, a low-noise amplifier, a bandpass filter, and an analog-to-digital converter. The memory can be volatile or non-volatile. The processor uses a low-power embedded computing chip, including but not limited to an ARM architecture central processing unit, an edge graphics processor, a digital signal processor, a field-programmable gate array, and a neural network acceleration unit; The communication interfaces include UART, SPI, I2C, CAN bus interfaces, and wireless transmission; The power management module includes a voltage regulator circuit, a power conversion chip, and a power management integrated circuit.

Images