A method and system for fine identification of a radiation source individual based on multi-modal feature fusion

Abstract

The application relates to the technical field of electronic reconnaissance and signal processing, and discloses a radiation source individual fine identification method and system based on multi-modal feature fusion, which comprises the following steps: performing preprocessing on obtained radiation source original signals to obtain signal pulse units; extracting time domain sequences of the signal pulse units and performing multi-domain transformation to generate a first time-frequency image, a second time-frequency image and a bispectrum image; inputting the first time-frequency image, the second time-frequency image, the bispectrum image and the time domain sequence into a multi-modal fusion neural network; extracting corresponding image depth features and comprehensive sequence features through three image processing branches and a sequence processing branch; performing cross-modal fusion on the three image depth features and the comprehensive sequence features through a fusion module to output a radiation source individual identification result. The application improves the identification accuracy of radiation source individuals in a complex electromagnetic environment.

Description

Technical Field

[0001] This invention relates to the field of electronic reconnaissance and signal processing technology, specifically to a method and system for fine identification of individual radiation sources based on multimodal feature fusion. Background Technology

[0002] Radiation source identification technology aims to identify specific radiation source devices by analyzing non-ideal features in radio signals caused by physical deviations in transmitter hardware. This technology has certain application needs in fields such as radio spectrum management, electronic reconnaissance, and electromagnetic environment monitoring.

[0003] Traditional radiation source identification methods primarily rely on feature engineering, manually extracting time-domain, frequency-domain, time-frequency-domain, or higher-order statistical features of the signal, and then using classifiers such as support vector machines or random forests for identification. The feature extraction process of these methods requires a high level of prior knowledge from experts, and in low signal-to-noise ratio or complex modulation signal environments, manually designed features have limited ability to represent the underlying details of the signal. Furthermore, the extracted high-dimensional features are prone to data redundancy and multicollinearity, limiting further improvements in identification performance.

[0004] Existing deep learning-based recognition methods have reduced reliance on manual feature engineering to some extent, but they still have limitations. Some methods employ single-dimensional feature extraction, such as inputting only the time-domain sequence of the original signal or a simple time-frequency image into the network for end-to-end learning. In real electromagnetic environments, received signals are often affected by environmental noise and channel fading, and data from a single mode can hardly fully reflect the nonlinear characteristics and phase coupling relationships of the transmitter in transient and steady-state processes.

[0005] While some multimodal fusion schemes incorporate multi-domain data, they often employ direct feature concatenation or simple vector stacking, failing to effectively handle the complementary and conflicting relationships between heterogeneous data. When the quality of data in a particular modality degrades due to environmental interference, simple feature concatenation will directly transmit invalid information to subsequent classification networks.

[0006] Furthermore, existing multimodal networks typically lack effective constraints on the balance and diversity among different feature branches during training, causing the model to easily degenerate into over-reliance on a single data modality during feature extraction, failing to effectively leverage the joint representation of multidimensional transform domains. These factors collectively limit the accuracy and robustness of existing methods in identifying radiation sources in complex scenarios. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method and system for fine identification of individual radiation sources based on multimodal feature fusion. This solves the problem that existing methods for identifying individual radiation sources rely too heavily on single-modal features in complex electromagnetic environments, resulting in insufficient representation of individual differences and limited identification accuracy.

[0008] To achieve the above objectives, the present invention provides the following technical solution: A method for fine identification of individual radiation sources based on multimodal feature fusion includes acquiring the original signal of the radiation source and performing preprocessing to obtain signal pulse units; Extract the time-domain sequence of the signal pulse unit, and perform multi-domain transformation on the signal pulse unit to generate a first time-frequency image, a second time-frequency image, and a bispectral image, respectively; The first time-frequency image, the second time-frequency image, the bispectral image, and the time-domain sequence are input into a multimodal fusion neural network; The first time-frequency image, the second time-frequency image, and the bispectral image are subjected to feature extraction by the three image processing branches of the multimodal fusion neural network, respectively, and three image depth features are output. The sequence processing branch of the multimodal fusion neural network performs feature extraction on the time-domain sequence and outputs comprehensive sequence features. The fusion module of the multimodal fusion neural network performs cross-modal fusion on the three image depth features and the comprehensive sequence features, and outputs the individual identification result of the radiation source.

[0009] This invention utilizes multi-domain transformation to obtain multi-dimensional images and sequences of signals, extracts time-frequency, higher-order statistical, and sequence features of signals through a multi-branch neural network, and performs cross-modal fusion on the output features, thereby improving the identification accuracy of individual radiation sources in complex electromagnetic environments.

[0010] Furthermore, the step of acquiring the original radiation source signal and performing preprocessing to obtain a signal pulse unit includes: demodulating the original radiation source signal into a zero intermediate frequency signal; constructing a frequency domain pre-whitening filter based on the background noise power spectral density in the current frequency band; performing weighted processing on the zero intermediate frequency signal to generate a time domain whitening signal; The sliding window local energy sequence of the time-domain whitened signal is calculated, and an adaptive threshold is generated based on the average reference energy during the targetless emission silence period and the preset constant false alarm rate criterion. The sliding window local energy sequence is traversed using state machine logic with a counting tolerance mechanism, and the time index of the pulse start and pulse end is determined based on the adaptive threshold. The signal pulse unit is obtained by performing time-domain segmentation on the zero intermediate frequency signal according to the time index.

[0011] Pre-whitening process suppresses colored noise background interference, and the endpoint detection mechanism combined with constant false alarm rate and state machine logic reduces false detections caused by sudden noise, thereby improving the extraction accuracy of pulse envelope boundaries.

[0012] Furthermore, in the step of performing multi-domain transformation on the signal pulse unit to generate the first time-frequency image, the second time-frequency image, and the bispectral image, the step of generating the first time-frequency image and the second time-frequency image includes: Extract the complex signal discrete sequence of the signal pulse unit, and extract the start segment feature sequence corresponding to the rising edge of the pulse envelope and the end segment feature sequence corresponding to the falling edge of the pulse from the complex signal discrete sequence; The initial segment feature sequence and the final segment feature sequence are processed by a smooth pseudo-Wegener-Wilson distribution transform to generate a first two-dimensional time-frequency distribution matrix and a second two-dimensional time-frequency distribution matrix, respectively. Gray-scale pixel interval mapping and scale interpolation scaling are performed on the absolute amplitudes of the first two-dimensional time-frequency distribution matrix and the second two-dimensional time-frequency distribution matrix, respectively, and the first time-frequency image and the second time-frequency image are output respectively.

[0013] For the start and end phases of the pulse, a smooth pseudo-Wegener-Wilson distribution is used to suppress cross-term interference and extract transient energy distortion features generated by nonlinear components in the RF front end.

[0014] Furthermore, in the step of generating the bispectral image by performing multi-domain transformation on the signal pulse unit to generate the first time-frequency image, the second time-frequency image, and the bispectral image, the step of generating the bispectral image includes: extracting the complex signal discrete sequence of the signal pulse unit and performing zero-mean processing to obtain a zero-mean sequence; Based on a preset time delay variable, the third-order cumulant of the zero-mean sequence is calculated, and a two-dimensional third-order cumulant matrix is ​​constructed. A two-dimensional discrete Fourier transform is performed on the two-dimensional third-order cumulant matrix to generate a two-dimensional bispectral matrix. The absolute magnitude of the two-dimensional bispectral matrix is ​​extracted and logarithmic compression is performed to obtain a logarithmically compressed numerical array. The logarithmically compressed numerical array is mapped to a grayscale pixel range and spatial resampling is performed to output the bispectral image.

[0015] Gaussian white noise is suppressed by using higher-order cumulants, and non-Gaussian nonlinear features of the signal are extracted by bispectral transform to characterize the phase coupling relationship caused by internal components of the transmitter.

[0016] Furthermore, the feature extraction of the first time-frequency image, the second time-frequency image, and the bispectral image by the three image processing branches of the multimodal fusion neural network, respectively, and the output of three image depth features, includes: each of the three image processing branches includes a cascaded multi-scale depth separable convolutional module and a convolutional block attention module; Multi-scale spatial features of the input image are extracted by using depth-separable convolution kernels of different scales set in parallel in the multi-scale depth-separable convolution module, and pointwise convolution is used to perform linear combination along the channel dimension to output multi-scale fused spatial features. The input image is the first time-frequency image, the second time-frequency image, or the bispectral image. The convolutional block attention module sequentially performs channel attention weighting and spatial attention weighting on the multi-scale fused spatial features, and outputs the image depth features to obtain the three image depth features. During the channel attention extraction stage, global average pooling and global max pooling are performed in parallel on the multi-scale fusion spatial features. After mapping by a multilayer perceptron, a channel weight vector is generated. The channel weight vector is then multiplied by the multi-scale fusion spatial features channel by channel to obtain the channel weighted features. In the spatial attention extraction stage, the channel-weighted features are pooled and concatenated along the channel dimension, and a spatial weight matrix is ​​generated by filtering and mapping through a two-dimensional convolutional layer. The spatial weight matrix is ​​then multiplied element-wise with the channel-weighted features.

[0017] Multi-scale depthwise separable convolutions are used to extract multi-scale spatial features while controlling the number of network parameters, and attention mechanisms are combined to extract local features in the image.

[0018] Furthermore, the step of performing feature extraction on the time-domain sequence through the sequence processing branch of the multimodal fusion neural network and outputting comprehensive sequence features includes: the sequence processing branch includes a one-dimensional convolutional neural network module, a Transformer encoder module, and a multilayer perceptron module; The time-domain sequence is subjected to sliding filtering along the time axis by the one-dimensional convolutional kernel in the one-dimensional convolutional neural network module, and the transient distortion features of the time-domain sequence within the local time window are extracted as local features. The Transformer encoder module maps the time-domain sequence into a multi-dimensional feature vector sequence and injects positional encoding. The global long-term dependency features of the time-domain sequence are extracted as global features through matrix operations of the multi-head self-attention unit. The time-domain sequence is mapped to a high-dimensional space by the multilayer perceptron module, and aliasing features are extracted as nonlinear features; the local features, the global features and the nonlinear features are concatenated to form the comprehensive sequence features.

[0019] By combining one-dimensional convolution with a Transformer encoder, local features, global long-term dependency features, and nonlinear features of time-domain sequences are extracted in parallel, while preserving modulation distortion and temporal envelope information in the signal waveform.

[0020] Furthermore, the step of performing cross-modal fusion on the three image depth features and the comprehensive sequence features through the fusion module of the multimodal fusion neural network to output the radiation source individual identification result includes: aligning the feature dimensions of the three image depth features and the comprehensive sequence features to construct a multimodal feature tensor; The cross-modal multi-head self-attention submodule in the fusion module calculates the dot product correlation between the query matrix and the key matrix of the multimodal feature tensor, generates cross-modal attention weights, performs weighted summation on the value matrix, and outputs the cross-modal enhanced feature sequence. The gated fusion submodule in the fusion module uses fully connected layers and activation functions to map and normalize the cross-modal enhanced feature sequence, generating a fusion weight vector representing the relative contribution ratio of each modality. The fusion weight vector and the cross-modal enhanced feature sequence are weighted by element-wise scalar multiplication and then added together to generate a multimodal fusion feature. The multimodal fusion feature is then used as the individual identification result of the radiation source.

[0021] The correlation of multimodal features is calculated by a cross-modal multi-head self-attention submodule, and the fusion weights of each modality feature are assigned by a gated fusion submodule to suppress redundant interference information and complete the fusion of multimodal features.

[0022] Furthermore, the method also includes a step of training the multimodal fusion neural network, including: calculating classification cross-entropy loss, contrastive learning loss, feature consistency loss, branch balance loss, and diversity loss; and performing a weighted summation of the classification cross-entropy loss, the contrastive learning loss, the feature consistency loss, the branch balance loss, and the diversity loss to generate a comprehensive regularized loss function. The network parameters of the multimodal fusion neural network are updated based on the comprehensive regularized loss function.

[0023] Multiple loss functions are used in a weighted combination to expand the distance between different individual feature classes while maintaining the optimal balance of each feature extraction branch, thus preventing network overfitting.

[0024] A second aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the fine identification method for individual radiation sources based on multimodal feature fusion provided in the first aspect.

[0025] A third aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method for fine identification of individual radiation sources based on multimodal feature fusion provided in the first aspect.

[0026] This invention provides a method and system for fine-grained identification of individual radiation sources based on multimodal feature fusion. It has the following beneficial effects: 1. This invention generates multimodal data, including time-frequency images, bispectral images, and time-domain sequences, by performing multi-domain transformation operations on signal pulse units. This method utilizes a smoothed pseudo-Wegener-Wilson distribution and third-order cumulants to extract the time-frequency distribution features of the signal in the transient process and the nonlinear phase coupling features in the steady-state process, respectively. Compared to single-dimensional feature extraction methods, this multi-domain joint data construction method provides richer underlying signal features, offering a reliable data foundation for subsequent identification work.

[0027] 2. This invention designs dedicated feature extraction branches for different modalities of data. The image processing branch combines depthwise separable convolution with an attention mechanism, while the sequence processing branch connects a one-dimensional convolutional neural network, a Transformer encoder, and a multilayer perceptron in parallel. This structure can simultaneously extract local receptive field features, global long-term dependency features, and high-dimensional spatial mapping features of signal sequences while controlling the overall parameter scale of the model, thereby improving the network model's ability to extract and represent signal details at different scales.

[0028] 3. This invention introduces cross-modal multi-head self-attention computation and a gating network in the feature fusion stage, and trains the model using a comprehensive loss function that includes branch balancing and diversity constraints. The gating network can adaptively allocate fusion weights based on the actual quality of data from each modality, reducing the interference of damaged or invalid modal information on the final result. Combined with the regularization constraints of the comprehensive loss function, it avoids the model from degenerating into relying solely on a single modality during training, ensuring the complementarity between features from different modalities, thereby improving the accuracy and stability of radiation source individual identification. Attached Figure Description

[0029] Figure 1 The above is an overall flowchart of the radiation source identification method provided in the embodiments of the present invention.

[0030] Figure 2 This is a schematic diagram of the structure of the multimodal fusion neural network model in an embodiment of the present invention.

[0031] Figure 3 This is a flowchart of image and sequence preprocessing in an embodiment of the present invention.

[0032] Figure 4This is a schematic diagram of the time-frequency diagram preprocessing results in an embodiment of the present invention.

[0033] Figure 5 This is a schematic diagram of the dual-spectrum preprocessing results in an embodiment of the present invention.

[0034] Figure 6 This is a schematic diagram of the sequence preprocessing results in an embodiment of the present invention.

[0035] Figure 7 This is a loss curve diagram of the network training process in an embodiment of the present invention.

[0036] Figure 8 This is an accuracy curve of the network training process in an embodiment of the present invention.

[0037] Figure 9 This is a confusion matrix diagram of the discrimination results of the pre-trained network in an embodiment of the present invention.

[0038] Figure 10 This is a dimensionality reduction diagram of the features of the pre-trained network discrimination result in an embodiment of the present invention.

[0039] Figure 11 This is a confidence histogram of the pre-trained network discrimination results in an embodiment of the present invention; Figure 12 This is a comparison curve of test performance indicators in an embodiment of the present invention; Figure 13 This is a grouped bar chart of individual classification results in an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1: See attached document Figure 1 This invention provides a method for fine identification of individual radiation sources based on multimodal feature fusion, comprising the following steps: S100: Acquire the original signal pulse train of the radiation source to be identified, and demodulate the digital intermediate frequency signal to a zero intermediate frequency signal using orthogonal demodulation technology; wherein, when the acquired original signal of the radiation source to be identified is already a baseband signal or a zero intermediate frequency signal, the orthogonal demodulation step can be omitted, and the baseband signal or zero intermediate frequency signal can be directly used as the data input for subsequent preprocessing.

[0042] S200 preprocesses the zero intermediate frequency signal to obtain independent signal pulse units and their parameters; S300 performs parallel multi-domain transformation on the preprocessed independent signal pulse units to generate multimodal data. The multimodal data includes at least a first time-frequency image generated based on the signal start segment, a second time-frequency image generated based on the signal end segment, a bispectral image, and a time-domain sequence. S400 inputs multimodal data into a pre-trained multimodal fusion neural network model for feature extraction and fusion, and outputs the individual identification results of radiation sources. The multimodal fusion neural network model includes three image processing branches that process the first time-frequency image, the second time-frequency image, and the bispectral image respectively, a sequence processing branch that processes the time-domain sequence, and a fusion module that fuses the output features of each branch.

[0043] When step S100 acquires the original signal pulse train of the radiation source to be identified and demodulates the digital intermediate frequency signal to zero intermediate frequency signal using quadrature demodulation technology, the signal preprocessing module in the system specifically performs the operation of digital intermediate frequency signal reception and local carrier mixing. In some embodiments, the signal preprocessing module may also perform bandpass filtering or equivalent frequency band selection processing before or after quadrature demodulation to filter out out-of-band noise and interference components outside the target operating frequency band. This operation process specifically includes the following steps: S101, Acquire the discrete digital intermediate frequency signal sequence. Receive the raw signal from the radiation source to be identified, and use the analog-to-digital converter of the pre-amplifier to discretize the analog signal according to the set system sampling rate to acquire the discrete digital intermediate frequency signal sequence. Let the system sampling interval be... Digital intermediate frequency signal sequence The data comprises signal amplitude, carrier frequency, instantaneous phase, and additive noise components. The instantaneous phase data retains information characterizing the underlying physical features of the radiation source hardware.

[0044] S102 generates an orthogonal local carrier sequence with the same frequency as the digital intermediate frequency signal. The numerically controlled oscillator inside the signal preprocessing module operates based on the known carrier frequency. The parameters are used to synchronously generate two mutually orthogonal local carrier sequences in the digital domain: an in-phase local oscillator sequence and a quadrature local oscillator sequence. To avoid additional scaling interference on the amplitude characteristics of the original signal, both sequences are configured with normalized unit amplitude. The in-phase local oscillator sequence is a cosine signal with the same frequency as the carrier, and the quadrature local oscillator sequence is a negative sine signal with the same frequency as the carrier but out of phase by 90 degrees. The expressions for the two local carrier sequences are as follows: The in-phase local oscillator sequence is: ; The orthogonal local oscillator sequence is: ; In the formula, The reference carrier for the in-phase branch, The reference carrier for the orthogonal branch, This is a discrete-time index.

[0045] S103 performs digital mixing calculations to achieve signal spectrum shifting. The signal preprocessing module controls the internal digital multiplier to perform digital mixing on the input digital intermediate frequency signal sequence. Respectively with in-phase local oscillator sequences and orthogonal local oscillator sequences Pointwise multiplication in the time domain is performed. Time-domain multiplication is equivalent to convolution in the frequency domain. This process shifts the original spectrum of the digital intermediate frequency (IF) signal to the vicinity of the zero-frequency baseband, obtaining a low-frequency baseband signal component containing the characteristics of the target individual. Simultaneously, it generates high-frequency redundant components that are up-converted to twice the carrier frequency. After mixing, the in-phase branch mixing sequence and the quadrature branch mixing sequence are output, providing underlying data support for the subsequent construction of the zero-IF signal after removing high-frequency components.

[0046] During step S100, after completing the digital mixing operation, the signal preprocessing module in the system continues to perform low-pass filtering and downsampling processing through its internal downconversion processing module to construct a zero-IF signal. The physical mechanism of low-pass filtering is to truncate the baseband spectrum and remove high-frequency redundant components and out-of-band noise. The purpose of downsampling is to eliminate data redundancy and reduce the computational scale of subsequent multimodal neural networks. This operation process specifically includes the following steps: S104, Design and configure a digital low-pass filter. Based on the known bandwidth parameters of the original signal from the radiation source to be identified, the down-conversion processing module configures the corresponding digital low-pass filter. In specific parameter configurations, to ensure that the signal maintains linear phase characteristics after filtering and to avoid distortion of the inherent phase fingerprint characteristics within the radiation source, in some embodiments, a finite-length unit impulse response filter can be selected. Alternatively, an infinite-length unit impulse response filter or other digital filters that meet the passband fidelity requirements can be selected according to the real-time requirements of the processing platform. To filter out the second harmonic high-frequency components generated by the mixing stage and completely retain the baseband information, in some embodiments, the cutoff frequency of the filter can be set to be greater than the single-sided bandwidth of the original signal and less than the frequency of the high-frequency redundant components associated with mixing. However, its specific cutoff frequency can be adjusted according to the target signal bandwidth, sampling rate, and system anti-aliasing requirements.

[0047] S105, a low-pass filtering operation is performed to obtain two quadrature signals, I and Q. The down-conversion processing module receives the in-phase branch mixing sequence and the quadrature branch mixing sequence output from the digital multiplier. The system controls these two mixing sequences to perform discrete linear convolution operations with the unit impulse response sequence of the aforementioned digital low-pass filter. Through the convolution filtering operation, high-harmonic components in the signal band and out-of-band background noise are blocked. After the filtering process is completed, the down-conversion processing module outputs two quadrature signals, I and Q, in the baseband band, which are used as the in-phase baseband sequence and the quadrature baseband sequence, respectively.

[0048] S106: Perform downsampling processing and construct a zero-IF signal in complex signal format. After quadrature demodulation shifts the signal spectrum to zero frequency, the actual effective bandwidth of the signal narrows. To reduce data dimensionality, the downconversion processing module performs equal-interval decimation operations on the filtered in-phase baseband sequence and the quadrature baseband sequence according to a set decimation factor. In this processing logic, the specific value of the decimation factor is subject to the Nyquist sampling theorem; its value must ensure that the sampling rate of the downsampled system is greater than or equal to twice the effective bandwidth of the baseband signal, thereby avoiding spectral aliasing. After downsampling processing, the decimated in-phase sequence is obtained. with orthogonal sequences ,in This represents the processed discrete-time index. Subsequently, the signal preprocessing module uses... As real data, As the imaginary part of the data, the two real number sequences are combined and mapped to the complex space. The resulting complex signal is the zero intermediate frequency signal. Its expression is: ; In the formula, Represents the imaginary unit. The resulting zero-IF signal. It carries the amplitude and instantaneous phase fingerprint parameters of the individual radiation source, and is then input as the underlying source data to the pulse detection module for subsequent segmentation processing.

[0049] After acquiring the zero-IF signal, considering that the noise introduced by the channel and RF front-end in the actual reconnaissance environment is usually non-flat colored noise, directly performing pulse detection with a fixed threshold is prone to false alarms or missed alarms. The preprocessing module performs background noise estimation and pre-whitening filtering operations to convert the colored noise into white noise with a flat power spectrum. The specific steps of this operation are as follows: S201, Extract background noise sample sequence. During signal acquisition, the system uses the signal silence period to capture a segment of observation data emitted by a target radiation source. This observation data is stored as a background noise sample sequence. Let this discrete background noise sample sequence be... , This is a discrete-time index. This sequence reflects the colored distribution characteristics of background noise within the current frequency band.

[0050] S202, calculate the background noise power spectral density. The preprocessing module processes the extracted background noise sample sequence. Frequency domain energy distribution estimation is performed. To balance the resolution and variance of the power spectrum estimation, the preprocessing module uses the Welch method. Process it. The data is divided into multiple overlapping segments, and each segment is windowed to reduce spectral leakage. In some embodiments, a Hamming window or a Heining window can be used as the windowing function, and the overlap rate can be configured to 50%. The squared discrete Fourier transform amplitude of each windowed data segment is calculated, and the arithmetic mean of the periodograms of all data segments is taken to obtain the estimated background noise power spectral density at discrete frequency points. The calculated background noise power spectral density sequence is as follows: ,in For discrete frequency indexes.

[0051] S203, construct a frequency domain pre-whitening filter. The principle of pre-whitening is to differentially weight the spectrum of the input signal, making the weighted noise spectrum envelope tend to be flat. Based on the background noise power spectral density sequence. The preprocessing module constructs a frequency-domain pre-whitening filter. The frequency-domain transfer function of this filter... It is set to the reciprocal of the arithmetic square root of the background noise power spectral density. To avoid the problem of the denominator being zero or numerical overflow caused by the noise power being close to zero at a certain frequency point during the calculation, a small regularization constant is introduced into the denominator. Its formula is expressed as: ; In the formula, The value is set according to the system's calculation precision, such as 10. -6 The magnitude of the value can be adjusted based on the numerical accuracy of the processor and the noise power spectrum estimation results. It acts on the frequency domain components of the input signal, attenuating the frequency bands with high noise in the spectrum and compensating for the frequency bands with low noise.

[0052] S204 performs filtering operations to generate a time-domain whitened signal. The preprocessing module processes the zero-IF signal containing individual radiation sources. A discrete Fourier transform is performed to convert it to the frequency domain. Subsequently, the digital processor compares the transformed frequency-domain discrete sequence with the frequency-domain transfer function of the pre-whitening filter. A point-by-point multiplication operation is performed in the frequency domain to complete the frequency domain weighting. Finally, the preprocessing module performs an inverse discrete Fourier transform on the weighted frequency domain sequence to restore it back to the time domain, outputting the corresponding time-domain whitening signal. Let this output time-domain whitening signal be... After pre-whitening, the signal's noise floor environment possesses the stable statistical characteristics of white noise, providing a unified reference benchmark for subsequent energy-based threshold decision-making.

[0053] After acquiring the time-domain whitened signal, the preprocessing module performs sliding window energy detection and adaptive threshold decision operations. The physical mechanism of sliding window energy detection utilizes the characteristic that the energy of the emitted signal from the radiation source is higher than the background noise energy. It reduces transient random fluctuations through local smoothing, and then uses threshold comparison to extract effective signal segments. This step aims to locate the start and end times of the effective radiation source pulses, dividing the continuous signal data stream into independent pulse segments. This operation specifically includes the following steps: S205, calculates the sliding window local energy sequence. The preprocessing module is configured with a length of... A sliding data window, controlling the window in the time domain whitening signal. Slide along the top of the timeline point by point. Window length The value of needs to balance detection sensitivity and boundary resolution, and is set based on the prior information of the minimum pulse width of the known target. It is usually configured to be one-tenth to one-fifth of the number of sampling points corresponding to the expected minimum pulse width. During the sliding process, the preprocessing module calculates the arithmetic mean of the squares of the complex signal amplitudes at each discrete time point within the window, thereby obtaining the discrete local energy sequence. This sliding window averaging operation smooths out the high-frequency fluctuations within the signal, highlighting the energy abrupt changes in the pulse envelope.

[0054] S206, determine the adaptive decision threshold. After pre-whitening, the energy distribution of the signal-free background segment exhibits relatively stable statistical characteristics. The preprocessing module extracts the time-domain whitened signal. In the targetless emission silence period data segment, calculate the average reference energy of the whitening noise in this segment, denoted as . Subsequently, based on the false alarm probability requirements preset by the reconnaissance system, a decision multiplier factor was configured. The preprocessing module generates an adaptive threshold by multiplying the decision multiplier factor by the average reference energy. Its expression is: ; In the formula, The value of this threshold is determined by the constant false alarm rate criterion and is typically set within the range of 3 to 5. This threshold can be automatically adjusted based on the absolute energy level of background noise in the current frequency band, avoiding the risk of misjudgment caused by using fixed empirical values.

[0055] S207, executes fault-tolerant decision to determine the start and end points of valid pulses. To prevent the radiation source signal from being misjudged and fragmented due to multipath fading or short-term amplitude dips in the internal modulation mechanism, the preprocessing module uses state machine logic with a counting tolerance mechanism to traverse the local energy sequence. The preprocessing module internally initializes the pulse setup counter and pulse fallback counter.

[0056] When the local energy sequence Continuous Exceedance of Adaptive Threshold When the number of data points reaches the threshold, the preprocessing module identifies the discrete-time index that first crosses the threshold as the pulse start point. This threshold is set based on the number of sampling points corresponding to one-tenth of the expected shortest pulse width, used to eliminate transient noise spikes. After detecting the pulse start point, if the local energy sequence... continuously below the adaptive threshold If the number of data points reaches the fallback tolerance threshold, the discrete-time index of the first fall threshold will be identified as the endpoint of the pulse. This fallback tolerance threshold is set based on a number of sampling points greater than the maximum multipath delay spread of the current channel, to prevent the effective signal from being abnormally truncated due to channel fading.

[0057] S208, segment and output independent signal pulse units. The preprocessing module summarizes the start and end time indices of the above logical output to form a pulse boundary set. Considering that pre-whitening filtering can easily cause local phase structure distortion of the original signal, the preprocessing module uses the above-determined pulse boundary set as an index to perform time-domain segmentation on the zero-IF signal sequence containing the original phase characteristics (i.e., the complex signal without whitening processing). The segmentation operation removes the signal-free noise segments between pulses, dividing the continuous data stream into multiple independent signal pulse units. The extracted independent signal pulse units are transferred to the multi-dimensional parameter extraction stage as samples to be identified.

[0058] After segmenting the independent signal pulse units, the preprocessing module performs multi-dimensional parameter extraction. A single complex signal waveform is insufficient to fully characterize the nonlinear distortion features within the radiation source hardware. To expose physical defects in the device, the preprocessing module maps the time-domain complex sequence to multiple parameter dimensions, such as instantaneous amplitude, instantaneous phase, and instantaneous frequency, to extract fingerprint features. This process specifically includes the following steps: S209, Calculate the instantaneous amplitude characteristic sequence. Let the sequence of independent signal pulse units currently being processed be... , This is a local discrete-time index within the pulse. This independent signal pulse unit sequence contains in-phase branch data and quadrature branch data. The preprocessing module controls the arithmetic unit to calculate the magnitude of the complex signal point-by-point. Its calculation logic involves finding the arithmetic square root of the sum of the squares of the in-phase branch data and the squares of the quadrature branch data at the corresponding discrete-time points. After the calculation is complete, the instantaneous amplitude envelope sequence of the pulse is extracted and denoted as... This instantaneous amplitude envelope sequence reflects the nonlinear amplitude modulation effect of the radiation source power amplifier.

[0059] S210, extract the instantaneous phase sequence and perform phase unwinding processing. The preprocessing module calculates the ratio of orthogonal branch data to in-phase branch data and calls the arctangent function to obtain the wrapped phase sequence distributed in the principal value range of negative to positive pi.

[0060] Because the phase of a physical signal accumulates continuously over time, crossing the principal value boundary of the arctangent function can trigger spurious phase jumps. The preprocessing module performs an unwinding operation on the wrapped phase sequence: the processor traverses the sequence, and when it detects that the absolute value of the phase jump between two adjacent points exceeds the constant pi, it recovers the continuously changing true absolute phase sequence by accumulating or subtracting the phase compensation amount of two pi constants, denoted as . This absolute phase sequence contains the phase noise characteristics of the voltage-controlled oscillator inside the radiation source.

[0061] S211, calculate the instantaneous frequency characteristic sequence. The instantaneous frequency is physically equivalent to the derivative of the instantaneous phase with respect to time. In the discrete digital processing domain, the preprocessing module estimates the instantaneous frequency using the phase difference method. The preprocessing module calculates the absolute phase sequence. The phase difference is calculated by dividing the current time index value by the value of the adjacent previous time index. This phase difference is then divided by the system sampling interval time constant to generate a discrete instantaneous frequency sequence, denoted as [missing information]. This sequence demonstrates the drift of the radiation source carrier frequency and the transient spurious characteristics of the transmitter.

[0062] S212 performs length alignment, normalization, and concatenation of the multidimensional feature sequence. Given the varying durations of the individual pulses intercepted by the reconnaissance, the preprocessing module performs length alignment on the extracted three-dimensional feature sequence to meet the stringent requirements of the subsequent neural network for a fixed input tensor dimension.

[0063] In some embodiments, the three-dimensional feature sequence includes an instantaneous amplitude feature sequence, an absolute phase sequence, and an instantaneous frequency feature sequence, used to characterize parameters in the energy domain, phase domain, and frequency domain, respectively. In embodiments with multi-antenna arrays or multi-channel reception capabilities, the preprocessing module can further extract spatial domain parameters such as angle of arrival, channel phase difference, channel amplitude difference, or array covariance features, and input them as supplementary parameters into the subsequent model. The standard sequence length is set to... For lengths greater than The sequence is truncated at the end; for sequences shorter than 1000, the truncated end is trunc The sequence is padded with zeros at the end.

[0064] Subsequently, the preprocessing module performs maximum-minimum normalization operations on the aligned instantaneous amplitude feature sequence, absolute phase sequence, and instantaneous frequency sequence, respectively. To avoid the denominator zero anomaly caused by constant sequence values, the processor introduces a regularization protection constant into the denominator term of the normalization calculation. This normalization operation uniformly scales and maps the numerical range of various physical parameters to a range of 0 to 1.

[0065] Finally, the preprocessing module concatenates the normalized 3D sequence into a matrix according to the channel dimension, constructing a three-row matrix. The multidimensional feature matrix consists of several components. The instantaneous amplitude feature sequence characterizes the signal's energy domain or envelope characteristics, the absolute phase sequence characterizes the signal's phase domain characteristics, and the instantaneous frequency feature sequence characterizes the signal's frequency domain variation characteristics. This multidimensional feature matrix serves as the parameter output for independent signal pulse units, characterizing the physical state of the pulse units. Subsequent parallel multi-domain transformations, based on these independent signal pulse units, generate a first time-frequency image, a second time-frequency image, a bispectral image, and a time-domain sequence. In some embodiments, the multidimensional feature matrix can also be concatenated or mapped with the time-domain sequence and used as supplementary input data for the sequence processing branch.

[0066] See attached document Figure 3 After receiving the preprocessed independent signal pulse units, the multi-domain transformation module performs parallel multi-domain transformation processing. This operation aims to extract the transient characteristics of the radiation source device at its underlying level and generate multimodal data. For ease of explanation, the calculation paths in the multi-domain transformation module are described in the order of steps; in actual execution, the first time-frequency image generation path, the second time-frequency image generation path, the bispectral image generation path, and the time-domain sequence generation path can be executed in parallel, or they can be executed in a preset order according to processor resources. In some embodiments of the present invention, the operation of the multi-domain transformation module to generate the first time-frequency image and the second time-frequency image specifically includes the following steps: S301, acquire the start and end intervals of fingerprint features. The multi-domain transform module extracts the start and end segments of the signal waveform containing fingerprint features. At the instants of the radiation source transmitter's power amplifier and voltage-controlled oscillator, the transient setup and fallback responses of these analog devices exhibit physically nonlinear characteristics. The preprocessed complex signal discrete sequence is set as... , This is a discrete-time index.

[0067] The multi-domain transform module, based on the set observation window length, extracts a fixed number of data points from the beginning of the discrete sequence of the complex signal to construct a starting segment feature sequence, denoted as . The length of the observation window is set based on the typical transient response time of the target radiation source device. It needs to cover the rising edge transition region of the pulse envelope to ensure complete capture of transient characteristic information and avoid dilution by long-term steady-state signals.

[0068] Similarly, the multi-domain transform module extracts an equal number of data points from the end of the discrete sequence of the complex signal to construct the final segment feature sequence, denoted as . The truncation operation ensures that subsequent two-dimensional transformations focus on the transient region where physical defects in the device are concentrated.

[0069] S302, executes smooth pseudo-Wegener-Wigner distribution transform. The multi-domain transform module uses smooth pseudo-Wegener-Wigner distribution transform to process the initial and final feature sequences separately. This transform utilizes the signal's own local conjugate multiplication operation to obtain the instantaneous autocorrelation function, and achieves high time-frequency concentration through a dual weighted filtering mechanism in both the time and frequency domains.

[0070] Taking the initial segment feature sequence as an example, the computation unit is configured with independent time-domain smoothing window functions and frequency-domain smoothing window functions. The computation unit uses the time-domain smoothing window to smooth the local conjugate multiplication sequence in the time dimension, and then uses the frequency-domain smoothing window to weight and attenuate the delay time variable dimension. The processor performs a discrete Fourier transform on the weighted discrete sequence, extracts the absolute magnitude of the transform result, and generates a two-dimensional time-frequency distribution matrix distributed at discrete time points and discrete frequency points. The multi-domain transform module uses the same logic to process the final segment feature sequence and obtain the corresponding two-dimensional time-frequency distribution matrix.

[0071] S303, Image Mapping and Scale Normalization. After obtaining the two two-dimensional time-frequency distribution matrices, the multi-domain transformation module performs scale normalization and image transformation operations. The computation unit uses a maximum-minimum projection mechanism to linearly map the absolute amplitude of each element of the two-dimensional time-frequency distribution matrix to a grayscale pixel range of 0 to 255. The transformed numerical matrix corresponds to a two-dimensional pixel array.

[0072] To meet the requirement of fixed input tensor dimension for multimodal fusion neural network models, the multi-domain transformation module uses bilinear interpolation algorithm to resample the two-dimensional pixel array, uniformly scaling the image space size to a resolution of 224×224.

[0073] Reference Appendix Figure 4 S304 outputs local time-frequency image features. After interpolation and scaling, the multi-domain transform module outputs the first time-frequency image based on the initial segment feature sequence, denoted as... The second time-frequency image is output based on the feature sequence of the ending segment, denoted as... The first and second time-frequency images, presented in the form of a two-dimensional matrix, show the high-frequency energy distribution characteristics at the start and end of the radiation source. These images, as spatial visual components in the multimodal data, are then transferred to a multimodal fusion neural network model for feature extraction.

[0074] After generating the time-frequency image, the multi-domain transform module continues to perform high-order spectral analysis on the independent signal pulse units to generate bispectral image components in the multimodal data. This process specifically includes the following steps: S305 calculates higher-order cumulants and bispectral matrices. The multi-domain transform module receives the preprocessed discrete sequence of complex signals. In an embodiment of the present invention, the arithmetic unit performs bispectral analysis on the discrete sequence. To suppress the interference of the DC component in the signal on subsequent cumulative quantity calculations, the arithmetic unit may first calculate the complex signal discrete sequence. The statistical mean of the sampled data is calculated, and the value of each sampled point is subtracted from the statistical mean to obtain a zero-mean sequence.

[0075] Subsequently, the computation unit calculates the third-order cumulant based on the zero-mean sequence. Specifically, the computation unit introduces two independent time delay variables. Within the time index interval of effective signal overlap, the conjugate sequence of the zero-mean sequence is multiplied point-by-point with the two delay sequences corresponding to the two time delay variables, and the product results are accumulated and averaged to obtain the third-order cumulant elements under the corresponding delay combination. By traversing all time delay combinations within the preset delay range, a two-dimensional third-order cumulant matrix is ​​constructed.

[0076] Next, the computational unit performs a two-dimensional discrete Fourier transform on the two-dimensional third-order cumulant matrix to generate a two-dimensional bispectral matrix containing complex elements, denoted as . In the formula, and These are two independent discrete frequency variables.

[0077] When the power devices inside the radiation source transmitter are operating in a nonlinear state, secondary phase coupling between different frequency components is easily induced. Bispectral analysis can quantitatively characterize the phase coupling features in this frequency dimension, and at the same time, by utilizing the blinding effect of the third-order cumulant on Gaussian background noise, it can suppress low-order statistical features affected by Gaussian noise and extract the inherent nonlinear fingerprint information of the device.

[0078] S306, Amplitude Logarithmic Compression and Image Mapping. The dynamic range of amplitude values ​​of elements within a two-dimensional bispectral matrix is ​​relatively large. To highlight the fingerprint characteristics in the low-amplitude region, the multi-domain transformation module first extracts the absolute amplitude values ​​of each element in the two-dimensional bispectral matrix, forming a bispectral amplitude matrix.

[0079] To avoid triggering a system exception due to undefined logarithmic operations when the amplitude is zero, the computation unit adds a preset positive bias constant to each absolute amplitude element of the matrix as a computational protection measure before performing the logarithmic calculation. Subsequently, the computation unit performs a logarithmic compression operation on the superimposed biased bispectral amplitude matrix to obtain a compressed bispectral amplitude array.

[0080] Reference Appendix Figure 5 The computational unit maps the logarithmically compressed numerical array to a grayscale pixel range of 0 to 255 using a linear scaling method, forming an initial bispectral grayscale numerical array. To adapt to the unified input space standard of the image processing branch in the multimodal fusion neural network model, the multi-domain transformation module resamples the spatial dimension of this array to a resolution of 224×224 using a bilinear interpolation algorithm, outputting the final bispectral image, denoted as... .

[0081] In an embodiment of the present invention, the multi-domain transformation module is configured with a time-domain sequence processing unit. This unit is used to extract and standardize the one-dimensional time-domain features of the signal to construct the sequence components in the multimodal data. The time-domain sequence processing unit performs data format standardization specifically including the following steps: S307, Extract the time-domain amplitude sequence. The multi-domain transform module receives the preprocessed discrete sequence of the full-range complex signal, denoted as... The computational unit calculates the absolute magnitude of the discrete sequence of the complex signal point by point. Specifically, this involves calculating the positive square root of the sum of the squares of the real and imaginary parts of the complex sample at each discrete time point. This operation removes the absolute phase information, obtaining a one-dimensional time-domain amplitude sequence that characterizes the fluctuations of the signal envelope, denoted as... .

[0082] S308, Sequence Length Evaluation. Due to the differences in the duration of the captured pulse signals, the number of data points in the extracted one-dimensional time-domain amplitude sequence varies each time. To meet the input data dimensionality requirements of the multimodal fusion neural network model, the multi-domain transformation module sets a target standard length. In this embodiment of the invention, the target standard length is set to 1024 points. The calculation unit counts the actual number of data points in the current one-dimensional time-domain amplitude sequence and compares this actual number of data points with the target standard length.

[0083] S309, Perform length standardization. Based on the comparison results, the calculation unit formats the one-dimensional time-domain amplitude sequence. If the actual number of data points equals the target standard length, the calculation unit retains the original sequence; if the actual number of data points is greater than the target standard length, the calculation unit retains valid data within the target standard length consecutively, using the sequence start time index as a reference, and truncates the data exceeding this window; if the actual number of data points is less than the target standard length, the calculation unit retains all existing valid data and continuously pads the end of the sequence with zero-value data for alignment. Through the above truncation, retention, or zero-padding mechanisms, the calculation unit ensures that the dimension of each batch of input data is consistent.

[0084] Reference Appendix Figure 6 S310, output time-domain sequence features. After standardization, the multi-domain transformation module generates a one-dimensional time-domain amplitude sequence with a fixed length equal to the target standard length. This sequence preserves the time-domain energy envelope features of the radiation source and serves as a one-dimensional modal component in the multimodal data. It is synchronously aggregated with the first time-frequency image, the second time-frequency image, and the bispectral image, and input into the multimodal fusion neural network model for feature extraction. In other embodiments, the time-domain sequence can also be a phase sequence, an instantaneous frequency sequence, a complex signal sequence, or a combination sequence consisting of at least two of amplitude, phase, and frequency.

[0085] After the parallel multi-domain transformation processing in steps S301 to S310, the multi-domain transformation module generates multimodal data including a first time-frequency image, a second time-frequency image, a bispectral image, and a time-domain sequence. This multimodal data is then input into a multimodal fusion neural network model.

[0086] See attached document Figure 2 In embodiments of the present invention, the multimodal fusion neural network model includes three image processing branches that process a first time-frequency image, a second time-frequency image, and a bispectral image, respectively. Those skilled in the art will understand that these three branches maintain a consistent physical structure at the network level, but their internal network weight parameters are updated independently.

[0087] The following section uses the processing of a single image input branch as an example to illustrate in detail the operation process of extracting multi-scale spatial features in image processing branches. This process specifically includes the following steps: S401, performs multi-scale depthwise convolution. The image processing branch receives the preprocessed two-dimensional image components as the input image tensor. The image processing branch internally cascades a multi-scale depthwise separable convolution module and a convolutional block attention module. The multi-scale depthwise separable convolution module is used to extract multi-scale spatial features of the input image using at least two different scales of depthwise separable convolution kernels set in parallel. Specifically, this module internally sets up three convolutional computation paths with different receptive field scales in parallel, and the kernel sizes corresponding to each path are preset to 3×3, 5×5, and 7×7, respectively. For each parallel path, the computation unit independently performs depthwise convolution operations, using a single-channel two-dimensional convolution kernel to slide filter across each independent channel of the input image tensor to extract the basic structure of the image spatial domain.

[0088] S402, performing pointwise convolution to complete cross-channel fusion. After completing depthwise convolution at each scale, the computation unit uses a 1×1 convolution kernel to perform pointwise convolution on the output of each independent path. This pointwise convolution linearly combines the output features of the depthwise convolution along the channel dimension, achieving cross-channel data information fusion. Decoupling standard convolution into two steps of depthwise convolution and pointwise convolution reduces the parameter load within the network while extracting multi-scale spatial features. After the above operations, the computation unit generates local feature tensors with three different receptive field dimensions.

[0089] S403, Feature Tensor Concatenation and Dimensionality Reduction. The multi-scale deep separable convolutional module performs a cascaded concatenation operation along the channel dimension on the three generated local feature tensors to construct an aggregated feature tensor. To control the expansion of the network's channel dimension, the computational unit uses a 1×1 fusion convolutional layer to perform dimensionality reduction and linear compression operations on this aggregated feature tensor, outputting multi-scale spatial features. These multi-scale spatial features characterize local time-frequency fluctuations and macroscopic spectral energy distribution at the pixel level.

[0090] S404, Perform channel attention weighting calculation. The convolutional block attention module receives the multi-scale spatial features and sequentially performs channel attention weighting and spatial attention weighting on the multi-scale spatial features. In the channel attention extraction stage, the computation unit performs global average pooling and global max pooling on the multi-scale spatial features along the entire spatial plane, generating two one-dimensional vectors containing channel statistical information. The computation unit feeds these two one-dimensional vectors into a weight-sharing multilayer perceptron network for parameter mapping, adds the mapping outputs element-wise, and generates channel weight vectors with values ​​ranging from 0 to 1 using a sigmoid activation function. The computation unit multiplies this channel weight vector element-wise with each channel corresponding to the multi-scale spatial features, outputting the channel-weighted features. This operation is used to improve the activation value response of feature channels that have discriminative power for identifying individual radiation sources.

[0091] S405, Performs Spatial Attention Weighted Calculation. During the spatial attention extraction stage, the computation unit performs average pooling and max pooling operations on the channel-weighted features along the channel dimension, compressing them into two single-channel two-dimensional spatial state matrices. The computation unit concatenates these two two-dimensional spatial state matrices along the channel dimension, filters the concatenation result using a 7×7 two-dimensional convolutional layer, and transforms it using a sigmoid activation function to generate a two-dimensional spatial weight matrix. The computation unit then multiplies this spatial weight matrix element-wise with each channel plane of the channel-weighted features, outputting the recalibrated spatial weighted features. This operation is used to numerically enhance pixel regions in the image that possess discriminative physical characteristics.

[0092] S406, Network Post-processing and Feature Output Shaping. The image processing branch sequentially performs batch normalization and Gaussian error linear unit activation mapping on the spatially weighted features. Batch normalization adjusts the numerical distribution of internal features to the standard normal range to accelerate network backpropagation convergence; activation mapping introduces nonlinear relationships to improve the model's fitting ability.

[0093] After processing, the computation unit adds the activation result to the input features of the corresponding convolutional or attention module in the image processing branch at the element-wise level, constructing a residual connection. To ensure the alignment consistency of the matrix dimensions during the addition operation, the computation unit can use a size-matching projection layer composed of 1×1 two-dimensional convolutional layers to align the channel dimensions of the input features to be added to match the activation result before performing the addition. This residual connection structure ensures that the gradient path is unobstructed during the update of deep network parameters.

[0094] Finally, the three independent image processing branches, for their respective inputs—the first time-frequency image, the second time-frequency image, and the bispectral image—output image depth features with abstract semantic representations, denoted as follows: , and .

[0095] As a further detailed explanation of the convolutional block attention module in S404 to S406 above, in the embodiments of the present invention, the image processing branch in the multimodal fusion neural network model is configured with a convolutional block attention module. This module is used to recalibrate the multi-scale spatial features output by the multi-scale depth-separable convolutional module to highlight important features and suppress irrelevant information. Those skilled in the art should understand that the convolutional block attention module internally cascades a channel attention submodule and a spatial attention submodule, and its weighted processing operation specifically includes the following steps: S411, Channel Feature Compression and Mapping. The channel attention submodule receives multi-scale spatial features as input feature tensors. Combined with the preprocessing results of the radiation source signal, this input feature tensor is represented as a three-dimensional data block containing the number of channels, height, and width. The computation unit performs global average pooling and global max pooling operations in parallel on the spatial dimensions of this input feature tensor.

[0096] Global average pooling extracts background statistics for each channel, while global max pooling extracts the local response extrema for each channel, generating a one-dimensional channel description vector for each. The computational unit then feeds these two channel description vectors into a multilayer perceptron with shared network weights. This multilayer perceptron contains two linear transformation layers; the first layer performs dimensionality reduction, decreasing the channel dimension according to a preset scaling ratio. The second layer performs a dimensionality increase operation, restoring the dimensionality to the original number of input channels. This structure is designed to enable the model to learn the cross-channel interaction relationships between the frequency components of each channel and to control the number of network parameters.

[0097] S412, Perform channel feature weighting. The computation unit adds the two intermediate channel feature vectors output by the multilayer perceptron element-wise. The result of the addition is input into the sigmoid activation function, which maps and generates channel attention weight vectors with values ​​ranging from 0 to 1.

[0098] After generating the weights, the computation unit performs element-wise multiplication of the channel attention weight vector with the original input feature tensor along the channel dimension, and outputs the channel-weighted feature tensor.

[0099] This operation is used to enhance the numerical response of the channel characterizing the transient energy distribution of the radiation source, and correspondingly suppress the expression of the channel corresponding to the ambient background noise.

[0100] S413, Spatial Feature Pooling and Concatenation. The spatial attention submodule receives the channel-weighted feature tensor as input. The computation unit independently performs average pooling and max pooling operations along the channel dimension of this tensor. The pooling operation here is used to aggregate pixel information from all channels at the same spatial physical location, generating two single-channel two-dimensional spatial state maps. Subsequently, the computation unit concatenates and concatenates these two two-dimensional spatial state maps along the channel direction to form a dual-channel aggregated feature map.

[0101] S414 performs spatial feature weighting. The computation unit uses a 7×7 two-dimensional convolutional layer to filter the aggregated feature map of the two channels to obtain a sufficient local spatial receptive field. The result of the filtering calculation is mapped by a sigmoid activation function to generate a two-dimensional spatial attention weight matrix with values ​​ranging from 0 to 1.

[0102] Subsequently, the computational unit performs element-wise multiplication of the two-dimensional spatial attention weight matrix with the channel-weighted feature tensor using a tensor broadcasting mechanism, outputting the final attention feature tensor after spatial recalibration. This operation is used to extract and numerically enhance the physical spatial pixel regions in the first time-frequency image, the second time-frequency image, or the bispectral image where the energy peak distribution of the actual radiation source signal exists.

[0103] S415, Feature Post-processing and Residual Output. The image processing branch performs batch normalization on the final attention feature tensor, adjusting the internal data distribution to a standard normal distribution range to accelerate gradient backpropagation during model training. After normalization, the data stream enters the Gaussian error linear unit for nonlinear mapping. The computation unit performs matrix addition at the pixel level between the activated features and the input features of the convolutional block attention module, constructing residual connections. When the number of channels or spatial dimensions of the two are inconsistent, the computation unit can perform dimension matching through a 1×1 convolutional projection layer. This residual connection structure helps maintain the stability of the gradient transmission path during deep network parameter updates, avoiding information loss in deep networks. Finally, the three independent image processing branches output image features with abstract semantic representations, denoted as... , and It is then transferred to subsequent modules to participate in joint cross-modal inference.

[0104] In embodiments of the present invention, the multimodal fusion neural network model includes a sequence processing branch for processing time-domain sequences. This branch employs a parallel architecture, comprising a one-dimensional convolutional neural network module, a Transformer encoder module, and a multilayer perceptron module. It extracts sequence features from different levels and perspectives through parallel processing and performs joint mapping.

[0105] Those skilled in the art will understand that the parallel design of these three modules is intended to accommodate the local transient envelope, global periodic correlation, and high-dimensional abstract representation of the radiation source signal sequence. The following details the operation process of extracting the time-domain features of the signal from the sequence processing branch, which specifically includes the following steps: S421, Distribute the time-domain input sequence. The sequence processing branch receives the signal amplitude sequence, which has been truncated and normalized in the preprocessing stage, as the input feature. Combined with the signal preprocessing logic, this input feature is a one-dimensional time-domain sequence normalized to 1024 sampling points. The computing unit simultaneously distributes this time-domain sequence to three parallel feature extraction modules to initiate the parallel feature extraction computation process.

[0106] S422 utilizes a one-dimensional convolutional neural network (CNN) module to extract local features. After receiving the time-domain sequence, the CNN module performs sliding filtering calculations along the time axis using a one-dimensional convolutional kernel. Internally, this module consists of multiple alternating stacked one-dimensional convolutional and pooling layers. The computation unit sets the local receptive field size and sliding stride to perform a weighted summation of the convolutional kernel with adjacent local sampling points in the sequence. This operation captures the amplitude fluctuations and transient distortion features of the radiation source signal within local time windows such as the rising or falling edges of the pulse. After processing, the data stream undergoes batch normalization and a nonlinear activation function, outputting a two-dimensional feature map corresponding to the local pattern features.

[0107] S423, the Transformer encoder module extracts global dependency features. To obtain the ultra-long sequence dependencies of the radiation source signal, the Transformer encoder module receives the time-domain sequence in parallel. To meet the tensor dimension requirements of the self-attention mechanism, the computation unit uses a linear projection layer to map the single-dimensional sampling point sequence into a multi-dimensional feature vector sequence. The computation unit pre-injects positional encoding into the sequence to preserve the absolute positional information of the sampling points on the time axis. Subsequently, the data stream enters the multi-head self-attention unit to perform matrix operations.

[0108] The computation unit uses different network weight matrices to linearly map the input sequence into a query matrix, a key matrix, and a value matrix, respectively. The computation unit obtains the correlation score between any two sampling times in the sequence by calculating the dot product of the query matrix and the key matrix, thereby capturing the global temporal correlation and long-term dependency of the signal. The computation unit adds the self-attention output after normalized exponential function mapping to the original sequence features, performs residual calculation, and then performs layer normalization to output the global long-term dependency feature matrix.

[0109] S424 utilizes a multilayer perceptron module to extract high-dimensional spatial features. The multilayer perceptron module also receives the initial time-domain sequence. This module consists of multiple fully connected networks and alternately stacked nonlinear activation units. The computational unit treats the one-dimensional time-domain sequence containing 1024 sampling points as a complete feature vector, and uses the weight parameter matrix in the fully connected layers to perform spatial transformation and dimensional expansion, mapping the signal data points to a higher-dimensional abstract space. This operation is used to extract nonlinear aliasing features caused by hardware defects in the radio frequency front-end of the radiation source, generating a high-dimensional spatial feature vector.

[0110] S425 performs multi-source sequence feature concatenation and mapping. The computing unit receives the two-dimensional feature map output by the one-dimensional convolutional neural network module and the global long-term dependency feature matrix output by the Transformer encoder module, and uses a flattening operation to convert both into one-dimensional feature vectors to eliminate the shape differences in the spatial and temporal dimensions of the outputs of different modules.

[0111] Subsequently, the computational unit concatenates the two flattened one-dimensional feature vectors with the high-dimensional feature vector output by the multilayer perceptron module along the feature dimension, constructing a joint sequence feature tensor. The computational unit then feeds this joint sequence feature tensor into the multilayer perceptron for mapping. The multilayer perceptron performs feature dimension reorganization and compression, removes redundant information generated during parallel extraction, and performs linear projection to obtain the final comprehensive sequence features, denoted as . This feature encompasses the multi-level representation of signals in local, global, and high-dimensional spaces, and flows to subsequent fusion modules to participate in cross-modal data interaction.

[0112] In embodiments of the present invention, the top layer of the multimodal fusion neural network model is configured with a cross-modal interaction and gated adaptive weighted fusion module. This module is used to process the multi-source heterogeneous features output from the image processing branch and the sequence processing branch, dynamically evaluate the confidence of each modal data stream using a gated network, perform weighted aggregation, and output multimodal fusion features. Those skilled in the art will understand that this module configuration is used to improve the problem of feature failure in single-modal environments under complex electromagnetic interference. The feature fusion and recognition operation process performed by this module specifically includes the following steps: S431 receives multi-source features and performs dimension alignment. The fusion module receives the first time-frequency image features, denoted as [feature name missing], which are independently output by the image processing branch. The second time-frequency image features are denoted as The bispectral image features are denoted as The comprehensive sequence features output from the sequence processing branch are denoted as... .

[0113] Since the feature tensors extracted by each branch differ in the number of channels or spatial scale, the computation unit uses multiple structurally independent linear projection layers or fully connected layers to perform linear transformations on the above four heterogeneous features respectively, and uniformly map them into a fixed-dimensional feature space to obtain the dimension-aligned first time-frequency image features, second time-frequency image features, bispectral image features and comprehensive sequence features, so as to eliminate the computational barrier between image features and one-dimensional sequence features.

[0114] S432 performs cross-modal feature organization. After feature dimension alignment, the computational unit organizes these four features according to modal dimensions, constructing a multimodal feature sequence containing four modal nodes and generating a multimodal feature tensor. This feature organization operation integrates the frequency domain distribution, bispectral coupling characteristics, and temporal envelope variation information of the radiation source signal into a unified data structure, establishing a joint representation of the radiation source state from different observation perspectives.

[0115] In step S433, cross-modal multi-head self-attention computation is performed. The computation unit inputs the multimodal feature tensors into the query mapping layer, key mapping layer, and value mapping layer, respectively, to obtain the query matrix, key matrix, and value matrix. Subsequently, the computation unit calculates the dot product correlation between different modal features within multiple attention heads and generates cross-modal attention weights using a normalized exponential function. The computation unit then uses the cross-modal attention weights to perform a weighted summation of the value matrix to obtain the cross-modal interaction features corresponding to each attention head.

[0116] The cross-modal interaction features output by each attention head are concatenated and linearly mapped to form a cross-modal enhanced feature sequence that still contains four modal nodes. Each modal node corresponds to the first time-frequency image feature, the second time-frequency image feature, the bispectral image feature, and the time-domain sequence feature, respectively. This step is used to explicitly model the correlation between the first time-frequency image feature, the second time-frequency image feature, the bispectral image feature, and the time-domain sequence feature.

[0117] S434 generates gated adaptive weights. The computation unit inputs cross-modal enhancement features into the gated network. This gated network consists of fully connected layers and a sigmoid activation function, or a fully connected layer, a normalization layer, and a sigmoid activation function.

[0118] The computational unit performs dimensionality reduction projection calculations on the cross-modal enhanced feature sequence using the network weight matrix and bias terms of the fully connected layer, and inputs the output into the Sigmoid activation function to generate an initial weight vector with values ​​ranging from 0 to 1. Subsequently, the computational unit normalizes the initial weight vector so that the sum of the four weight scalars is 1, thus forming a fused weight vector representing the relative contribution ratio of each modality. This fused weight vector contains four independent weight scalars, corresponding to the first time-frequency image feature, the second time-frequency image feature, the bispectral image feature, and the time-domain sequence feature, respectively.

[0119] Reference Appendix Figure 10 S435 performs adaptive weighting and feature fusion. The computation unit extracts the weight scalars assigned to each modality based on the feature fusion weight vector, and performs element-wise scalar multiplication with the four modal features enhanced across modalities using a tensor broadcast mechanism. After weighting, the computation unit performs element-wise addition of the four recalibrated feature matrices, outputting the final multimodal fusion features. The business logic of this operation is that when the data of a certain modality is distorted due to environmental noise, the gating network can adaptively reduce the weight scalar corresponding to that modality, while relatively increasing the weights of the undisturbed modalities, thereby maintaining the overall stability of the system's recognition.

[0120] Reference Appendix Figure 11S436 performs individual radiation source classification and identification. The computing unit feeds the multimodal fusion features into the final classification network. The classification network consists of a linear projection layer and a Softmax classifier. The linear projection layer flattens and compresses the multidimensional multimodal fusion features, mapping them into a logistic value vector that matches the preset total number of radiation source categories. The Softmax classifier uses a normalized exponential algorithm to convert this logistic value vector into a relative probability distribution of each category label. The computing unit reads this probability distribution and selects the category label corresponding to the data dimension with the highest predicted probability value as the final identification result of the currently input radiation source individual. This category label uniquely corresponds to a specific radiation source transmitter entity.

[0121] S437, Perform model training and parameter updates. During the network model training phase, the computing unit pre-acquires a sample dataset containing the identity identifiers of real radiation sources and uses these real identity identifiers to construct real label vectors in one-hot encoded format.

[0122] After forward propagation to compute and output the relative probability distribution, the computational unit at least uses classification cross-entropy loss to calculate the classification error between the probability distribution and the true label vector, and can further combine contrastive learning loss, feature consistency loss, branch balance loss and diversity loss to construct a comprehensive regularization loss function.

[0123] Based on the comprehensive regularization loss function, the computational unit uses the backpropagation algorithm and gradient descent optimizer to iteratively update the network weight parameters of each branch and fusion module in the multimodal fusion neural network model. This training process continues until the preset training completion conditions are met.

[0124] In embodiments of the present invention, during the construction and parameter optimization phases of the multimodal fusion neural network model, a comprehensive regularization loss function is constructed and calculated to guide the iterative update of network parameters. The comprehensive regularization loss function improves the model's generalization performance and robustness in complex electromagnetic interference environments by jointly constraining classification accuracy, cross-modal feature interaction, branch representation balance, and feature diversity across multiple dimensions. The construction and calculation process of this comprehensive regularization loss function specifically includes the following steps: S511 configures the overall architecture and weight coefficients of the comprehensive regularization loss function. The computation unit weights and combines the classification cross-entropy loss, contrastive learning loss, feature consistency loss, branch balance loss, and diversity loss to construct the overall comprehensive regularization loss function, expressed as follows: ; In the formula, The overall comprehensive regularization loss value represents the multimodal fusion neural network model; This represents the cross-entropy loss value. This represents the contrastive learning loss value; The value represents the feature consistency loss. This represents the branch balance loss value; Represents the diversity loss value; , , , and These represent the balance weight parameters corresponding to each loss term. In a specific embodiment of the present invention, the pre-set weight parameter values ​​may be specifically: , , , 0.05 and To ensure the convergence stability of the network model in the early stages of training, the balance weights are allowed to be fine-tuned in actual engineering debugging, and the values ​​of each balance weight are limited to the real number range of 0 to 2.

[0125] S512, Calculate the classification cross-entropy loss. After the classification module outputs the probability distribution of the preset categories, the calculation unit calculates the classification cross-entropy loss in combination with the true label vectors of the training samples. This loss is used to constrain the model's classification head to accurately predict the identity of individuals from radiation sources.

[0126] Specifically, the computation unit extracts the true label vector in the one-hot encoding format corresponding to the current training sample, retrieves the index position of the effective category, and obtains the predicted probability value corresponding to that index position output by the classification module. Subsequently, the computation unit calculates the natural logarithm of this predicted probability value and takes its negative value, using the resulting scalar value as the classification cross-entropy loss value. This loss value is used to force the classification head to improve the accuracy of identifying the correct radiation source entity.

[0127] S513, Calculate the contrastive learning loss. To narrow the feature distance between samples of the same type of radiation source and widen the feature distance between samples of different types of radiation sources at the data feature level, the computation unit introduces a contrastive learning loss in the multimodal feature space. Within a single training batch containing a preset number of pulse samples, the computation unit constructs positive and negative sample pairs by retrieving the real radiation source hardware identity identifiers associated with each pulse.

[0128] In some embodiments, a training batch may contain 64 pulse samples, and may be stratified according to radiation source identification, ensuring that each radiation source category participating in contrastive learning within the same training batch contains at least two pulse samples. The computation unit classifies different pulse signal features originating from the same transmitter entity into positive sample pairs and pulse signal features originating from different transmitter entities into negative sample pairs. When a certain anchor sample does not have a corresponding positive sample in the current training batch, the computation unit may skip the calculation of the contrastive learning loss corresponding to that anchor sample, or select a corresponding sample from the sample cache queue as a positive sample.

[0129] The calculation unit uses the cosine similarity operator to calculate the first similarity between the current anchor sample feature and the positive sample feature, and multiple second similarities between the anchor sample feature and all negative sample features in the current batch. Next, the exponential values ​​of the first similarity and each of the second similarities are calculated with the natural constant as the base. The exponential value of the first similarity is divided by the sum of the exponential values ​​of the first similarity and all the exponential values ​​of the second similarity. Then, the negative natural logarithm of the ratio after division is obtained to obtain the contrastive learning loss value.

[0130] This loss term is used to enable the model to capture subtle inter-class fingerprint differences caused by inherent defects in the radiation source hardware.

[0131] S514, Calculate the feature consistency loss. In order to achieve alignment of different feature extraction branches with the same sample representation space, and to ensure that each branch can mine the common fingerprint features of the radiation source signal, the computing unit constructs the feature consistency loss by calculating the differences between the output features of each branch.

[0132] In specific operation, the computing unit extracts four feature vectors after dimensional alignment by the parameter mapping layer. These four feature vectors include the first time-frequency image feature vector. Second time-frequency image feature vector Bispectral image feature vector and the comprehensive sequence feature vector .

[0133] The computing unit pairs these four feature vectors together, calculates the absolute difference between each feature vector pair, and then calculates the squared Euclidean distance between the difference vectors. Finally, the six alignment distance values ​​are summed to output the final feature consistency loss value. This loss term, from a physical and statistical perspective, constrains the heterogeneous feature branches to maintain consistency at the identity discrimination semantic level, prompting different branches to focus on the inherent hardware emission defects of the radiation source, thereby reducing the representation bias between heterogeneous spaces.

[0134] S515, Calculate the branch balance loss. To encourage each feature extraction branch to contribute equally to the fusion decision and to prevent the gated fusion submodule from over-relying on a single modality while ignoring complementary features of other modalities, the computation unit constructs the branch balance loss based on the fusion weight vector generated by the gated fusion submodule.

[0135] Specifically, the computation unit extracts four weight scalars dynamically generated by the gated fusion submodule and assigned to the corresponding four feature branches. These four weight scalars are denoted as the first weight scalar. Second weight scalar Third weight scalar With the fourth weight scalar Subsequently, the computing unit calculates the algebraic differences between each of the four weight scalars and the theoretical mean value of 0.25, and then squares each algebraic difference. Finally, the results of the four squared terms are summed to obtain the branch balance loss value. By minimizing the mean square deviation of each weight scalar from the theoretical mean, the gated fusion network is constrained to maintain the cooperative participation of each feature extraction branch, avoiding the phenomenon of single-modal overload causing other branches to be idle.

[0136] S516, Calculate the diversity loss. To encourage each branch to learn differentiated and complementary features and reduce feature information redundancy among multimodal data, the computation unit utilizes the correlation between feature vectors of different branches to construct the diversity loss. Specifically, the computation unit calculates the diversity loss based on the aforementioned first time-frequency image feature vector. Second time-frequency image feature vector Bispectral image feature vector and the comprehensive sequence feature vector Perform pairwise dot product calculations and divide by the L2 norm product of the pairwise vectors to obtain the absolute cosine similarity values ​​between the six specific modal pairs. Next, the computational unit directly weights and sums these six absolute cosine similarity values ​​to output the final diversity loss value. This loss term applies differential constraints to the four feature vectors at the underlying computation level, reducing the non-discriminatory redundancy correlation between features in each branch. This guides the image processing branch and the sequence processing branch to mine heterogeneous features from different physical dimensions such as the time-frequency domain, higher-order spectral domain, and the original time domain, thus avoiding the feature redundancy problem caused by parameter sharing.

[0137] In embodiments of the present invention, the multimodal fusion neural network model optimizes the scheduling strategy and anti-overfitting mechanism by configuring specific parameters during the training phase, so as to help the model achieve stable convergence on the image and sequence sample set obtained by complex signal processing, and improve its generalization discrimination ability on the individual fingerprint features of radiation source hardware.

[0138] In terms of overall operation, the parameter optimization scheduling strategy balances the exploratory ability in the early stages of training with the precise tuning ability in the later stages by dynamically controlling the step size of the learning rate. Meanwhile, the anti-overfitting mechanism suppresses the model's excessive reliance on noisy data in the training samples by introducing random perturbations and residual paths into the network structure. Together, these two mechanisms ensure the model's robustness in unknown electromagnetic testing environments. This process specifically includes the following steps: S521, configured with an adaptive weight decay optimizer. The computation unit selects an adaptive moment estimation optimizer with decoupled weight decay as the network weight update operator. The optimizer is used to update the trainable parameters in the multimodal fusion neural network model, including the network weight parameters in the three image processing branches, the sequence processing branch, the fusion module, and the classification module. The three image processing branches process the first time-frequency image, the second time-frequency image, and the bispectral image, each with a size of 224×224, respectively. The sequence processing branch processes a time-domain sequence of length 1024 points. During training iterations, the computation unit uses this optimizer to update the parameters of each branch and the fusion module in the multimodal fusion neural network model. The computation unit can preset the initial learning rate to 0.1% and the batch size to 64.

[0139] To prevent the weight values ​​from becoming too large in the later stages of model training, the computation unit is configured with a weight decay coefficient of 0.01. By decoupling the weight decay term from the gradient-based step size update process, the current weight matrix is ​​directly decayed proportionally to constrain the numerical range of the network weights. This works in conjunction with the comprehensive regularization loss function to perform parameter optimization.

[0140] S522, Perform cosine annealing scheduling of the learning rate. To adjust the convergence rate of the model parameters at different training stages, the computation unit uses a cosine annealing strategy to dynamically adjust the learning rate. Specifically, the computation unit calculates the cosine function value based on the ratio of the current number of iterations to the preset maximum total number of iterations, and multiplies the difference between the initial maximum learning rate and the preset lower limit of the learning rate by the cosine function value, so that the current learning rate decreases smoothly in a cosine curve as the number of iterations increases.

[0141] In this embodiment, the lower limit of the learning rate is set to 0, and the maximum total number of iterations is set to 100. Under different electromagnetic signal sample sets, the maximum total number of iterations is limited to between 50 and 200. This dynamic adjustment avoids the parameter oscillation problem caused by a fixed learning rate in the later stages of training, allowing the parameters to converge smoothly to the target error convergence point.

[0142] S523, Implement temporary backoff and random depth regularization. As an optional implementation, the temporary backoff layer is specifically cascaded after the convolutional block attention modules within the three image processing branches, and within the multilayer perceptron module within the sequence processing branch. During each forward propagation computation, the computation unit randomly sets the activation output of some neurons to 0 according to a preset 50% dropout probability, in order to reduce the over-reliance of neurons on specific feature combinations.

[0143] Meanwhile, for the multi-scale depth separable convolution module, the computation unit adopts a random depth mechanism. During the training process, it randomly skips the nonlinear transformation calculation of some convolutional layers with a preset survival probability and establishes a direct identity channel across layers. The survival probability can be set to 0.8, and its adjustable value range can be limited to 0.6 to 0.95. This structured regularization method enhances the robustness of the model to different electromagnetic environment noise.

[0144] S524 deploys a residual connection structure. Between the cascaded levels of the image processing and sequence processing branches, computational units construct residual connection paths. Specifically, this involves directly adding the input feature vector to the output tensor of the nonlinear transformation layer element-wise across the nonlinear transformation layer.

[0145] This mechanism establishes an identity mapping channel to ensure that when calculating gradients during backpropagation, the error signal can bypass the nonlinear activation function of the deep network and be directly transmitted back to the shallow network. This is used to alleviate the gradient vanishing problem that is prone to occur in deep neural networks during training, thereby protecting the integrity of multi-scale spatial features and long-range dependency features in multi-layer transmission and supporting the model to deeply mine the fine fingerprint features of radiation sources.

[0146] S525, Perform sample augmentation. To expand the coverage of training samples and improve the model's resistance to signal distortion, the computation unit performs multi-dimensional sample augmentation operations at the data input layer. The objects of sample augmentation include independent signal pulse units obtained by orthogonal demodulation and preprocessing, as well as the resulting 224×224 resolution time-frequency images and bispectral images.

[0147] The enhancement methods specifically include superimposing additive white Gaussian noise with different signal-to-noise ratios (SNRs) into the signal sequence, performing random temporal phase shifts on the signal pulses, and randomly masking the time-frequency images. To simulate real-world electronic reconnaissance scenarios, the SNR of the additive white Gaussian noise is limited to between 4 and 10 dB, and the masking area is limited to between 5% and 20%. By simulating signal distortion under different electromagnetic environments, the model is forced to learn the inherent hardware bias characteristics of individual radiation sources, rather than simply memorizing signal waveforms at specific SNRs.

[0148] Reference Appendix Figure 7 -Appendix Figure 9 S526, Training Completion Determination. The computational unit monitors the changes in classification loss values ​​on the training and validation sets in real time during training, and combines this with... Figure 7 The loss curve shown, attached Figure 8 The accuracy curve shown and the attached Figure 9 The confusion matrix shown evaluates the training convergence state and recognition performance of the model. When the comprehensive regularization loss function does not reach the condition for further decrease within a preset number of consecutive iterations (e.g., no further decrease after ten consecutive iterations), or reaches the preset maximum number of iterations, the computation unit determines that the model training is complete. At this point, the computation unit saves the current optimal weight parameters of each network layer, obtaining a fine-grained recognition model that uniquely corresponds to a specific radiation source transmitter entity.

[0149] Example 2: This embodiment provides a fine identification system for individual radiation sources based on multimodal feature fusion. The system includes a signal preprocessing module, a multi-domain transformation module, a multimodal fusion neural network module, and an identification output module.

[0150] The signal preprocessing module receives the raw signal from the radiation source to be identified and performs quadrature demodulation, down-conversion, filtering, pre-whitening, sliding window energy detection, adaptive threshold decision, pulse segmentation, and parameter extraction on the raw signal to obtain independent signal pulse units and their parameters. When the input raw radiation source signal is already a baseband signal or a zero-IF signal, the quadrature demodulation process can be omitted from the signal preprocessing module. The filtering process may include one or more of low-pass filtering, band-pass filtering, or equivalent band selection processing.

[0151] The multi-domain transformation module receives independent signal pulse units output by the signal preprocessing module and performs parallel multi-domain transformation processing on these units to generate multimodal data. The multimodal data includes at least a first time-frequency image generated based on the signal start segment, a second time-frequency image generated based on the signal end segment, a bispectral image generated based on the entire signal pulse segment, and a time-domain sequence.

[0152] The multimodal fusion neural network module receives multimodal data output from the multi-domain transformation module and performs feature extraction, cross-modal interaction, gated adaptive weighted fusion, and classification processing on the multimodal data. The multimodal fusion neural network module includes three image processing branches, one sequence processing branch, a fusion module, and a classification module.

[0153] The three image processing branches process the first time-frequency image, the second time-frequency image, and the bispectral image, respectively. Each image processing branch includes a multi-scale depth-separable convolutional module and a convolutional block attention module. The multi-scale depth-separable convolutional module is used to extract multi-scale spatial features from the corresponding image, and the convolutional block attention module is used to sequentially perform channel attention weighting and spatial attention weighting on the multi-scale spatial features.

[0154] The sequence processing branch is used to process time-domain sequences. This branch includes a parallel arrangement of a one-dimensional convolutional neural network module, a Transformer encoder module, and a multilayer perceptron module. The one-dimensional convolutional neural network module extracts local features of the time-domain sequence, the Transformer encoder module extracts global long-term dependency features, and the multilayer perceptron module extracts abstract features of the time-domain sequence in high-dimensional space.

[0155] The fusion module is used to fuse the features output from three image processing branches and one sequence processing branch. The fusion module includes a cross-modal multi-head self-attention submodule and a gated fusion submodule. The cross-modal multi-head self-attention submodule performs cross-modal interaction on the projected and aligned features of each branch, calculating the correlation between features of different modalities. The gated fusion submodule dynamically generates the fusion weights for each branch based on the cross-modal interaction results and performs a weighted summation of the features from each branch to obtain the multimodal fused features.

[0156] The classification module receives multimodal fusion features and outputs the probability distributions for each preset radiation source category through a fully connected network and a Softmax classifier. The identification output module reads the probability distributions and selects the category label corresponding to the highest probability value as the individual radiation source identifier for the current input signal.

[0157] In this embodiment, a fine-grained identification system for radiation sources based on multimodal feature fusion can be deployed in an electronic device. The electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the steps of the fine-grained identification method for radiation sources based on multimodal feature fusion described in Embodiment 1.

[0158] The present invention also provides a computer device, which may be an embedded signal processing device, an edge computing device, a server, a vehicle-mounted electronic reconnaissance device, a shipborne electronic reconnaissance device, or a fixed spectrum monitoring device.

[0159] It includes a processor and a memory. The memory stores computer programs that the processor can execute. When the computer program is executed by the processor, it performs the methods described above.

[0160] The present invention also provides a storage medium storing a computer program, which is executed by a processor to perform the method described above.

[0161] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0162] Specific application examples: In a specific electronic reconnaissance application embodiment, the system of the present invention receives the intercepted raw digital intermediate frequency signal, performs orthogonal demodulation and adaptive pulse segmentation, and extracts standardized multimodal data, including start and end time-frequency images, bispectral images, and one-dimensional time-domain sequences. A pre-trained multimodal fusion neural network model performs parallel feature extraction and gated adaptive weighted fusion on the heterogeneous data, outputting the probability distribution of each preset radiation source category. The system determines the hardware identity of the target radiation source individual based on the highest probability value, achieving high-accuracy individual identification under complex background noise.

[0163] Taking the interception of raw airborne radar signals with a center frequency of 2.4 GHz and a system sampling rate of 100 MHz as an example, after the signal preprocessing module removes the second harmonic redundant components, it constructs a complex sequence of zero intermediate frequency signals by extraction. The complex signal expression is as follows: In the pulse detection phase, the average reference energy during the targetless silence period in the current frequency band is calculated to be 1.5mV. 2 Based on the preset false alarm probability setting and a multiplier factor of 4, the adaptive threshold for sliding window detection is calculated to be 6.0mV. 2 Data points that continuously exceed this threshold are considered valid pulses once the established tolerance is reached. The extracted pulses are fed in parallel into the multi-domain transformation module to generate a 224×224 pixel grayscale time-frequency image, a bispectral image, and a 1024-point time-domain amplitude sequence.

[0164] In cross-modal interactive computing, if the reconnaissance system experiences sudden narrowband interference causing local distortion of the second time-frequency image features, the gating network detects the decrease in modality confidence through a multi-head self-attention mechanism. In its dynamically output feature fusion weight vector, the weight scalar assigned to the second time-frequency image adaptively decreases to 0.1, while the weights of other modalities remain around 0.3, effectively suppressing the negative propagation of distorted features through feature weighting. During model training, parameter updates depend on the overall comprehensive regularization loss function, with the core constraint formula being... In the application, classification loss weights are set. With a weight of 1.0 and the remaining auxiliary weights set to 0.1, this joint loss function drives the network optimizer to smoothly converge the overall error to a preset limit of 0.05 within 100 iterations.

[0165] To verify the recognition performance of the scheme in a real channel environment, this embodiment constructs an experimental comparison and verification set. The system collects the transmitted signals of 10 communication radios from the same batch with the same internal hardware model, and superimposes 4dB to 20dB of additive white Gaussian noise and Rayleigh multipath fading through a channel simulator to form a test set containing 50,000 pulse samples.

[0166] See attached document Figure 12 This figure illustrates the accuracy trends under different signal-to-noise ratio (SNR) conditions. The horizontal axis represents the SNR of the test samples (in dB), and the vertical axis represents the recognition accuracy (in %). The figure uses contrasting black and white lines: a thick black solid line with solid dots represents the multimodal fusion scheme of this invention, while a thin black dashed line with hollow triangles represents the single-modal baseline scheme with only time-frequency images as input. Experimental data shows that in the high SNR range of 15dB to 20dB, both schemes achieve an accuracy exceeding 95%; however, in the low SNR environment of 4dB to 8dB, the baseline scheme curve drops to 72%, while the scheme of this invention still maintains a recognition success rate of over 89%.

[0167] See attached document Figure 13 This figure visually illustrates the distribution of the model's recognition accuracy for 10 specific radiation source individuals at a signal-to-noise ratio of 10 dB. The horizontal axis represents the category numbers (1 to 10) of the 10 radiation source individuals, and the vertical axis represents the percentage of recognition accuracy for a single category. For each category number, two bars are plotted side-by-side for comparison, distinguished by a high-resolution grayscale fill pattern: the bar with a black diagonal shading represents the present invention, and the pure white bar with a black border represents the single-modal baseline solution. Quantitative data shows that all bars with black diagonal shading have a height that is at or above the 0.96 mark, ranging from 0.96 to 0.99; while the adjacent pure white bars have a more volatile height, with some categories even below 0.85.

[0168] The bar chart comparison results objectively confirm that the multimodal fusion features extracted by the method of the present invention not only have better overall performance, but also exhibit highly balanced discrimination stability for different devices with slight hardware deviations, overcoming the feature failure problem that is prone to occur in the identification of specific individuals in single-modal schemes.

Claims

1. A method for fine-grained identification of individual radiation sources based on multimodal feature fusion, characterized in that, Includes the following steps: The raw signal from the radiation source is acquired and preprocessed to obtain signal pulse units; Extract the time-domain sequence of the signal pulse unit, and perform multi-domain transformation on the signal pulse unit to generate a first time-frequency image, a second time-frequency image, and a bispectral image, respectively; The first time-frequency image, the second time-frequency image, the bispectral image, and the time-domain sequence are input into a multimodal fusion neural network; The first time-frequency image, the second time-frequency image, and the bispectral image are subjected to feature extraction by the three image processing branches of the multimodal fusion neural network, respectively, and three image depth features are output. The sequence processing branch of the multimodal fusion neural network performs feature extraction on the time-domain sequence and outputs comprehensive sequence features. The fusion module of the multimodal fusion neural network performs cross-modal fusion on the three image depth features and the comprehensive sequence features, and outputs the individual identification result of the radiation source.

2. The method for fine identification of individual radiation sources based on multimodal feature fusion according to claim 1, characterized in that, The step of acquiring the original signal from the radiation source and performing preprocessing to obtain the signal pulse unit specifically includes: The original signal from the radiation source is demodulated into a zero intermediate frequency signal; A frequency domain pre-whitening filter is constructed based on the background noise power spectral density in the current frequency band, and the zero intermediate frequency signal is weighted to generate a time domain whitening signal. Calculate the sliding window local energy sequence of the time-domain whitened signal, and generate an adaptive threshold based on the average reference energy during the targetless emission silence period and a preset constant false alarm rate criterion; The state machine logic with a counting tolerance mechanism is used to traverse the local energy sequence of the sliding window, and the time index of the pulse start and pulse end is determined based on the adaptive threshold. The signal pulse unit is obtained by performing time-domain segmentation on the zero intermediate frequency signal according to the time index.

3. The method for fine identification of individual radiation sources based on multimodal feature fusion according to claim 1, characterized in that, The steps of performing multi-domain transformation on the signal pulse unit to generate the first time-frequency image, the second time-frequency image, and the bispectral image respectively specifically include: When generating the first time-frequency image and the second time-frequency image, the complex signal discrete sequence of the signal pulse unit is extracted, and the start segment feature sequence corresponding to the rising edge of the pulse envelope and the end segment feature sequence corresponding to the falling edge of the pulse are respectively extracted from the complex signal discrete sequence. The initial segment feature sequence and the final segment feature sequence are processed by a smooth pseudo-Wegener-Wilson distribution transform to generate a first two-dimensional time-frequency distribution matrix and a second two-dimensional time-frequency distribution matrix, respectively. Perform grayscale pixel interval mapping and scale interpolation scaling on the absolute amplitudes of the first two-dimensional time-frequency distribution matrix and the second two-dimensional time-frequency distribution matrix respectively, and output the first time-frequency image and the second time-frequency image respectively; When generating the bispectral image, the complex signal discrete sequence of the signal pulse unit is extracted and zero-mean processing is performed to obtain the zero-mean sequence. Based on the preset time delay variable, the third-order cumulant of the zero-mean sequence is calculated to construct a two-dimensional third-order cumulant matrix. A two-dimensional discrete Fourier transform is performed on the two-dimensional third-order cumulant matrix to generate a two-dimensional bispectral matrix. The absolute magnitude of the two-dimensional bispectral matrix is ​​extracted and logarithmic compression is performed to obtain a logarithmically compressed numerical array. The logarithmically compressed numerical array is mapped to a grayscale pixel range and spatial resampling is performed to output the bispectral image.

4. The method for fine identification of individual radiation sources based on multimodal feature fusion according to claim 1, characterized in that, The steps of performing feature extraction on the first time-frequency image, the second time-frequency image, and the bispectral image through the three image processing branches of the multimodal fusion neural network, and outputting the three image depth features, specifically include: The three image processing branches all include cascaded multi-scale depth-separable convolutional modules and convolutional block attention modules. Multi-scale spatial features of the input image are extracted by using depth-separable convolution kernels of different scales set in parallel in the multi-scale depth-separable convolution module, and pointwise convolution is used to perform linear combination along the channel dimension to output multi-scale spatial features. The input image is the first time-frequency image, the second time-frequency image, or the bispectral image. The convolutional block attention module sequentially performs channel attention weighting and spatial attention weighting on the multi-scale spatial features, and outputs the image depth features to obtain the three image depth features. During the channel attention extraction stage, global average pooling and global max pooling are performed in parallel on the multi-scale spatial features. After mapping by a multilayer perceptron, a channel weight vector is generated. The channel weight vector is then multiplied by the multi-scale spatial features channel by channel to obtain the channel weighted features. In the spatial attention extraction stage, the channel-weighted features are pooled and concatenated along the channel dimension, and a spatial weight matrix is ​​generated by filtering and mapping through a two-dimensional convolutional layer. The spatial weight matrix is ​​then multiplied element-wise with the channel-weighted features.

5. The method for fine identification of individual radiation sources based on multimodal feature fusion according to claim 1, characterized in that, The steps of performing feature extraction on the time-domain sequence through the sequence processing branch of the multimodal fusion neural network and outputting comprehensive sequence features specifically include: The sequence processing branch includes a one-dimensional convolutional neural network module, a Transformer encoder module, and a multilayer perceptron module; The time-domain sequence is subjected to sliding filtering along the time axis by the one-dimensional convolutional kernel in the one-dimensional convolutional neural network module, and the transient distortion features of the time-domain sequence within the local time window are extracted as a two-dimensional feature map. The Transformer encoder module maps the time-domain sequence into a multi-dimensional feature vector sequence and injects positional encoding. The global long-term dependency features of the time-domain sequence are extracted as a global long-term dependency feature matrix through matrix operations of the multi-head self-attention unit. The time-domain sequence is mapped to a high-dimensional space through the multilayer perceptron module, and the aliasing features are extracted as high-dimensional space feature vectors. The two-dimensional feature map, the global long-term dependency feature matrix, and the high-dimensional space feature vector are concatenated to form the comprehensive sequence feature.

6. The method for fine identification of individual radiation sources based on multimodal feature fusion according to claim 1, characterized in that, The steps for performing cross-modal fusion of the three image depth features and the comprehensive sequence features through the fusion module of the multimodal fusion neural network to output the individual identification result of the radiation source specifically include: Align the feature dimensions of the three image depth features with the comprehensive sequence features to construct a multimodal feature tensor; The cross-modal multi-head self-attention submodule in the fusion module calculates the dot product correlation between the query matrix and the key matrix of the multimodal feature tensor, generates cross-modal attention weights, performs weighted summation on the value matrix, and outputs the cross-modal enhanced feature sequence. The gated fusion submodule in the fusion module uses fully connected layers and activation functions to map and normalize the cross-modal enhanced feature sequence, generating a fusion weight vector representing the relative contribution ratio of each modality. The fusion weight vector and the cross-modal enhanced feature sequence are weighted by element-wise scalar multiplication and then added together to generate a multimodal fusion feature. The multimodal fusion feature is then used as the individual identification result of the radiation source.

7. The method for fine identification of individual radiation sources based on multimodal feature fusion according to claim 1, characterized in that, It also includes the step of training the multimodal fusion neural network, including: Calculate the classification cross-entropy loss, contrastive learning loss, feature consistency loss, branch balance loss, and diversity loss; The classification cross-entropy loss, the contrastive learning loss, the feature consistency loss, the branch balance loss, and the diversity loss are weighted and summed to generate a comprehensive regularized loss function; The network parameters of the multimodal fusion neural network are updated based on the comprehensive regularized loss function.

8. A fine-grained identification system for radiation sources based on multimodal feature fusion, characterized in that, Applied to the fine-grained identification method for individual radiation sources based on multimodal feature fusion as described in any one of claims 1-7: The signal preprocessing module is used to acquire the original signal from the radiation source and perform preprocessing to obtain signal pulse units; A multi-domain transformation module is used to extract the time-domain sequence of the signal pulse unit and perform multi-domain transformation on the signal pulse unit to generate a first time-frequency image, a second time-frequency image, and a bispectral image, respectively. A multimodal fusion neural network module, comprising three image processing branches, a sequence processing branch, and a fusion module; The three image processing branches are respectively used to perform feature extraction on the first time-frequency image, the second time-frequency image and the bispectral image, and output three image depth features; The sequence processing branch is used to perform feature extraction on the time-domain sequence and output comprehensive sequence features; The fusion module is used to perform cross-modal fusion on the three image depth features and the integrated sequence features, and output the individual identification result of the radiation source.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, when the processor executes the computer program, it implements the steps of the method for fine identification of individual radiation sources based on multimodal feature fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the method for fine identification of individual radiation sources based on multimodal feature fusion as described in any one of claims 1 to 7.

Images