Multi-modal authentication method for unmanned aerial vehicles based on physical layer, acoustics and form features

Abstract

This invention relates to a multimodal authentication method for unmanned aerial vehicles (UAVs) based on physical layer, acoustic, and shape features. The method includes: simultaneously acquiring communication signal data, raw acoustic signals, and raw shape images of the UAV; preprocessing the communication signal data to obtain processed IQ data; preprocessing the raw acoustic signals to extract acoustic features; preprocessing the raw shape images and obtaining deep shape features through a neural network; and inputting the processed IQ data, acoustic features, and deep shape features into a trained multimodal fusion model for multimodal fusion and optimization to obtain the authentication result. The multimodal authentication method for UAVs provided by this invention balances the requirements of security, stability, and lightweight design. It constructs a lightweight and robust multimodal fusion authentication method combining acoustic authentication, physical layer authentication, and shape authentication, providing more reliable identity security for UAVs.

Description

Technical Field

[0001] This invention relates to the core technology of information security, and also to cross-domain integration of communication technology, acoustic technology, signal processing, computer vision technology and artificial intelligence. Specifically, it relates to a multimodal authentication method for unmanned aerial vehicles (UAVs) based on physical layer, acoustic and shape features, which is used to realize the trusted verification of the identity of UAVs and ensure the flight safety and communication security of UAVs. Background Technology

[0002] As a core terminal of the low-altitude Internet of Things (IoT), drones have been widely used in various fields such as inspection, transportation, and operations. Their communication security is a crucial foundation for the high-quality development of the drone industry and an important component of the low-altitude safety and control system. As a vital carrier supporting the construction of the intelligent IoT, the legitimacy of drones and their communication security are directly related to the operational order of low-altitude airspace and the smooth execution of various tasks.

[0003] However, drones currently face severe security challenges and are vulnerable to various attacks such as counterfeiting, signal replay, and man-in-the-middle attacks. Intrusion by illegal drones not only disrupts legitimate operations but also causes security issues such as communication link leakage and data tampering. End-to-end communication security for drones is difficult to guarantee effectively. Therefore, drone authentication technology has become a core requirement for low-altitude security protection. At the same time, drone onboard equipment is limited by hardware design, generally having low computing power, limited storage resources, and strict power consumption constraints. Traditional cryptographic authentication systems rely on complex key calculations and management, resulting in high computing power consumption, which is difficult to directly adapt to the lightweight hardware characteristics of drones.

[0004] To adapt to the hardware constraints of drones, lightweight acoustic authentication, physical layer authentication, and shape authentication have become important methods for drone identity verification: Acoustic authentication relies on the drone's own acoustic characteristics to achieve identity verification without the need for additional hardware modules, adapting to the lightweight requirements of drones, and offering a simple and real-time authentication process; Physical layer authentication is based on the inherent physical layer attributes of the drone's wireless transmitter, requiring no key management, resulting in low communication overhead and enabling seamless real-time authentication, meeting the needs of drone wireless communication scenarios; Shape authentication extracts unique shape features by collecting images of the drone's appearance, offering strong intuitiveness and difficulty in replication, and assisting in identity determination at the physical form level.

[0005] Each of the three authentication methods has significant drawbacks when used individually, making it difficult to meet the high security and robustness requirements of drones: acoustic authentication is susceptible to environmental noise and external interference, resulting in poor authentication stability and weak resistance to replay attacks; physical layer authentication is greatly affected by drone movement and attitude changes, lacking robustness and having weak open-set recognition capabilities; and shape feature authentication is easily affected by factors such as shooting angle, lighting conditions, and occlusion, limiting the accuracy of feature extraction. The limitations of a single authentication path prevent it from effectively addressing the diverse security threats in the complex operating environment of drones, making it difficult to simultaneously meet the requirements of security, stability, and lightweight design.

[0006] Therefore, there is an urgent need to construct a lightweight and robust multimodal fusion authentication method that combines acoustic authentication, physical layer authentication, and shape authentication to provide more reliable identity security for drones. Summary of the Invention

[0007] To address the aforementioned problems in the existing technology, this invention provides a multimodal authentication method for unmanned aerial vehicles (UAVs) based on physical layer, acoustic, and shape characteristics. The technical problem to be solved by this invention is achieved through the following technical solution: This invention provides a multimodal authentication method for unmanned aerial vehicles (UAVs) based on physical layer, acoustic, and shape features, comprising: Simultaneously acquire the drone's communication signal data, raw acoustic signals, and raw shape images; The communication signal data is subjected to preprocessing operations of sampling truncation, downsampling, and sampling normalization in sequence to obtain the processed IQ data; The original acoustic signal is subjected to wavelet transform, amplitude normalization, and time-frequency alignment operations in sequence. Then, acoustic features that are strongly correlated with the UAV identity and are difficult to replicate are extracted from the preprocessed acoustic signal. The original shape image is sequentially subjected to image denoising, size normalization, and illumination correction to obtain the processed shape image. Deep shape features that are strongly correlated with the UAV's identity and are difficult to replicate are extracted from the processed shape image. The processed IQ data, the acoustic features, and the deep shape features are input into the trained multimodal fusion model for multimodal fusion and optimization to obtain the authentication result.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a multimodal authentication method for drones based on physical layer, acoustic, and shape features. It simultaneously acquires raw acoustic signals, communication signal data, and raw shape images, performs feature extraction and targeted preprocessing on each, and then employs a multimodal fusion model based on a three-stream encoder + attention fusion architecture to achieve multimodal fusion and dynamic optimization. This invention introduces a three-modal complementary mechanism of acoustic features, physical layer IQ data, and shape features, effectively overcoming the shortcomings of single authentication such as environmental sensitivity, channel dependence, and illumination angle influence. Through attention mechanism and multi-loss function optimization, it dynamically balances the contributions of the three modalities, improving authentication accuracy and robustness. The extracted acoustic and shape features possess physical properties that are difficult to replicate, and combined with multi-dimensional fusion effect evaluation, it strengthens the system's anti-attack capability. Furthermore, it uses a lightweight, low-cost mobile terminal as the acquisition device, eliminating the need for dedicated and expensive hardware, adapting to the actual application needs of drone authentication, and providing more comprehensive and reliable security for drone identity authentication.

[0009] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a multimodal authentication method for unmanned aerial vehicles (UAVs) based on physical layer, acoustic, and shape features provided by the present invention. Figure 2 This is a flowchart illustrating another UAV multimodal authentication method based on physical layer, acoustic and shape features provided by the present invention. Detailed Implementation

[0011] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0012] Please see Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating a multimodal authentication method for unmanned aerial vehicles (UAVs) based on acoustic, physical layer, and shape features provided by the present invention. Figure 2 This is a flowchart illustrating another UAV multimodal authentication method based on acoustic, physical layer, and shape features provided by the present invention. The present invention provides a UAV multimodal authentication method based on physical layer, acoustic, and shape features, which includes: Step 1: Synchronously acquire the communication signal data, raw acoustic signals, and raw shape images of the UAV.

[0013] Specifically, the raw acoustic signals of the drone are acquired through a mobile terminal, the raw shape image of the drone is acquired through an image acquisition device, and the communication signal data of the drone is acquired through software-defined radio.

[0014] The authentication subject in this embodiment is a drone, and the application scenario is any scenario in which a drone is used. Factors such as environmental noise, communication distance, device mobility, lighting conditions, and shooting angle need to be considered.

[0015] Based on the application scenarios described above, lightweight and low-cost acquisition devices should be selected. Priority should be given to mobile terminals with audio acquisition modules, such as smartphones and tablets. Software-defined radios capable of acquiring communication signal data and image acquisition devices with image capture capabilities should also be selected, such as mobile terminals with cameras or high-definition cameras. This ensures that the audio acquisition module of the mobile terminal can effectively acquire acoustic signals, the software-defined radio can accurately receive the communication signal data of the drone, and the image acquisition device can clearly capture images of the drone's shape. All three types of devices should meet the time accuracy requirements for synchronous acquisition.

[0016] The mobile terminal's audio acquisition module, software-defined radio, and image acquisition equipment are activated to simultaneously acquire the drone's raw acoustic signals, communication signal data, and raw shape images. During the acquisition process, relevant information such as acquisition time, acquisition equipment, drone's relative position, shooting angle, and lighting conditions are recorded to provide a reference for subsequent signal processing and authentication. The acquired raw acoustic signals, communication signal data, and raw shape images are stored in a preset format, ensuring data integrity and readability.

[0017] In an optional embodiment, acquiring the UAV's communication signal data via software-defined radio includes: S1. Receives radio wave signals transmitted by the drone via radio, and obtains baseband analog signals through down-conversion by the radio frequency front-end.

[0018] S2. The continuous time signal segment corresponding to the preamble sequence extracted from the baseband analog signal. Perform orthogonal demodulation to demodulate into in-phase components in the analog domain. and orthogonal components The preamble sequence is a predefined known signal segment (i.e., a signal segment) in the UAV communication protocol. (e.g., preambles for WiFi, DJI OcuSync).

[0019] In this embodiment, the in-phase component and orthogonal components They are represented as follows:

[0020]

[0021] in, The carrier frequency of the preamble sequence, The time factor is 2, which is used to compensate for the amplitude attenuation of the demodulated signal; the negative sign is used to ensure the phase consistency of the quadrature components.

[0022] S3, respectively for cutoff frequencies of In-phase components and orthogonal components Perform low-pass filtering to obtain the in-phase component after low-pass filtering. and the quadrature components after low-pass filtering .

[0023] S4. Using an analog-to-digital converter (ADC) at a sampling frequency The in-phase components after low-pass filtering and the quadrature components after low-pass filtering Sampling is performed to obtain the in-phase component sequence in the discrete domain. orthogonal component sequences in the discrete domain ,in, and Satisfying the Nyquist sampling theorem, i.e. ≥2 They are represented as follows:

[0024]

[0025] in, =0,1,2,..., 1, The number of sampling points in the leading sequence. = / , The total duration of the leading sequence. The sampling interval is... =1 / .

[0026] S5. The discrete-domain in-phase component sequence orthogonal component sequences in the discrete domain Communication signal data combined into a complex matrix form.

[0027] Specifically, the extracted communication signal data is stored in the form of a complex matrix for subsequent preprocessing and feature fusion. The communication signal data is represented as follows:

[0028] Where, IQ∈ , It is a complex field, and the matrix dimension is the number of sampling points in the preceding sequence. ×1, where j is the imaginary unit.

[0029] Step 2: Perform preprocessing operations such as sampling truncation, downsampling, and sampling normalization on the communication signal data in sequence to obtain the processed IQ data.

[0030] In one specific embodiment, step 2 may include: Step 2.1: Sample and truncate the communication signal data, remove redundant data, retain the valid data segment, and use the retained valid data segment as the truncated IQ data, represented as:

[0031] in, The truncated IQ data, , Select the number of sampling points to be used for drone certification.

[0032] Step 2.2: Following a preset downsampling rate, downsample the truncated IQ data while preserving core physical layer features to obtain downsampled IQ data. This reduces the data volume and improves subsequent processing efficiency. Represented as:

[0033] in, k =0,1,..., , The downsampling rate is... It is a positive integer.

[0034] Step 2.3: Normalize the downsampled IQ data to map it to […]. The IQ data is obtained by tracing the data within the interval [1,1] to eliminate the impact of differences in data magnitude. , represented as:

[0035] in, For the k-th sampling point of the processed IQ data, , These represent the operations of finding the maximum and minimum values ​​of a vector, respectively.

[0036] Step 3: Perform wavelet transform, amplitude normalization, and time-frequency alignment operations on the original acoustic signal in sequence, and then extract acoustic features that are strongly correlated with the identity of the UAV and are difficult to replicate from the preprocessed acoustic signal.

[0037] In one specific embodiment, step 3 may include: Step 3.1: Utilize the low-pass filter coefficients of the db8 wavelet basis. and high-pass filter coefficients The original acoustic signal is decomposed into 5 layers, with the decomposition process proceeding from layer 1 to layer 5, resulting in the low-frequency coefficients and high-frequency coefficients after decomposition from layer 1 to layer 5.

[0038] Specifically, this embodiment uses the db8 wavelet basis function to perform 5-level wavelet decomposition and reconstruction on the original acoustic signal to remove high-frequency noise. Let the original acoustic signal before preprocessing be a continuous-time signal x(t), and its discrete sampling sequence be x[ ], Discrete indices for coefficients at each level, initial conditions =x[ ].

[0039] Based on the dual-scale equation of scaling function φ(t) and wavelet function ψ(t) of the db8 wavelet basis, the low-frequency coefficients after each level of decomposition are calculated using a recursive formula. and the high frequency coefficients after decomposition j ranges from 1 to 5. The scaling function φ(t) and the wavelet function ψ(t) are expressed as follows:

[0040]

[0041] in, These are the low-pass filter coefficients of the db8 wavelet basis; These are the high-pass filter coefficients of the db8 wavelet basis, used to separate high-frequency noise; This is a normalization factor to ensure the conservation of signal energy before and after decomposition.

[0042] Wavelet decomposition achieves multi-scale decomposition of a signal through downsampling and filtering. Let the low-frequency coefficients after the j-th level decomposition be... High-frequency coefficients are The decomposition process proceeds recursively from level 1 to level 5. Therefore, the low-frequency coefficients after decomposition at level j... and the high-frequency coefficients after the j-th layer decomposition They are represented as follows:

[0043]

[0044] Where j=1,2,3,4,5 corresponds to 5 levels of decomposition, and the signal length is halved after each level of decomposition. The value range is dynamically adjusted according to the hierarchy. The initial condition is when j=0. =x[ ], which corresponds to the discrete sampling sequence of the original acoustic signal; These are the low-frequency coefficients after the 5th layer decomposition. , , , , These are the high-frequency coefficients after decomposition into 1-5 layers, containing noise of different scales.

[0045] Step 3.2, based on adaptive threshold For the high-frequency coefficients after decomposition of the j-th layer Soft thresholding is performed to suppress high-frequency noise, resulting in the high-frequency coefficients of the j-th layer after thresholding. .

[0046] Specifically, the high-frequency coefficients after decomposition of layers 1-5 Soft thresholding is performed to suppress noise and preserve effective signal details. The high-frequency coefficients after thresholding at the j-th layer... Represented as: ,

[0047] ,

[0048] in, The threshold is adaptive, and N is the number of sampling points for the original acoustic signal. This is an estimate of the noise standard deviation of the high-frequency coefficients in the j-th layer. It is a median function. For symbolic functions, Corresponding high frequency coefficients The indexes are kept consistent to ensure accurate execution of point-by-point thresholding.

[0049] This embodiment avoids signal abrupt changes caused by hard thresholding by using soft thresholding, thus ensuring the smoothness of the signal after denoising.

[0050] Step 3.3: Based on the low-frequency coefficients after decomposition at level 5 and the high-frequency coefficients after thresholding at levels 1 to 5, the denoised acoustic signal is recovered through 5-level wavelet reconstruction.

[0051] Specifically, based on the processed low-frequency coefficients and the high-frequency coefficients after thresholding from layer 1 to layer 5 The denoised acoustic signal x′(t) is recovered by five-layer wavelet reconstruction.

[0052] Here, the reconstruction formula is expressed as:

[0053] The reconstruction process proceeds from layer 5 back to layer 0. This is a floor operation, corresponding to the upsampling in the decomposition process, i.e., zero-padding interpolation; the final reconstruction result is... [ ]=x′[ That is, the denoised discrete acoustic signal is converted into a continuous acoustic signal x′(t) after digital-to-analog conversion.

[0054] Step 3.4: Perform amplitude normalization processing on the denoised acoustic signal to obtain the normalized acoustic signal.

[0055] Specifically, the amplitude of the denoised acoustic signal x′(t) is normalized to unify the amplitude range of the denoised acoustic signal x′(t) to a preset interval (i.e., [...]). 1,1]) to eliminate the influence of different signal strengths.

[0056] Step 3.5: Perform time and frequency alignment on the normalized acoustic signal to align it with the communication signal data and original shape image from the same UAV at the same time reference and frequency resolution, thus obtaining the preprocessed acoustic signal.

[0057] Specifically, based on the timestamps of the collected acoustic signals, the frequency parameters preset by the UAV communication protocol, and the synchronization clock information of the acquisition equipment, the normalized acoustic signals are time-calibrated and frequency-unified to ensure that the acoustic signals collected at different times and under different frequency conditions are consistent, and at the same time, they are matched with the spatiotemporal dimensions of the communication signal data and shape image data collected at the same time.

[0058] Step 3.6: Extract acoustic features from the preprocessed acoustic signal that are strongly correlated with the UAV's identity and difficult to replicate using Mel-frequency cepstral coefficients (MFCC). Acoustic characteristics are the audio signal characteristics generated by the rotation of the propeller during drone flight or the audio signal characteristics generated by the operation of the motor during drone flight.

[0059] Step 4: Perform image denoising, size normalization, and illumination correction operations on the original shape image in sequence to obtain the processed shape image, and extract deep shape features that are strongly correlated with the UAV's identity and are difficult to replicate from the processed shape image.

[0060] In one specific embodiment, step 4 may include: Step 4.1: Use Gaussian filtering to denoise the original shape image to obtain the denoised shape image.

[0061] Specifically, Gaussian filtering is used to denoise the acquired original shape image. The size of the Gaussian filter kernel is adaptively adjusted according to the image noise intensity, and the formula is as follows:

[0062] in, σ represents the coordinates of the pixels within the filter kernel relative to the kernel center, and σ is the standard deviation of the Gaussian kernel, which is dynamically set according to the image noise level. The filtering process achieves noise suppression by convolving the Gaussian kernel with the local region of the image.

[0063] Step 4.2: Use bilinear interpolation to normalize the size of the denoised shape image to obtain the normalized shape image.

[0064] Specifically, bilinear interpolation is used to scale the denoised shape image to a preset uniform size to ensure the clarity of the scaled image and to ensure that the feature extraction dimensions of different images are consistent.

[0065] Let the pixel coordinates of the denoised shape image be... The dimensions are The pixel coordinates of the normalized shape image are The dimensions are ,but:

[0066]

[0067] , , ,

[0068] ,

[0069]

[0070] in, This is a round-down operation. This is a rounding up operation. , For interpolation weights, This represents the pixel grayscale value.

[0071] Step 4.3: Perform illumination correction on the normalized shape image through histogram equalization to enhance image contrast, and obtain the processed shape image.

[0072] Specifically, histogram equalization is used to perform illumination correction on the normalized shape image in order to enhance image contrast and eliminate the effects of uneven illumination.

[0073] Let the gray level range of the normalized shape image be [0, L]. 1], grayscale kThe number of pixels is The total number of pixels in the image is Then gray level k The cumulative distribution function is:

[0074] The mapping relationship of gray level k′ after equalization is as follows:

[0075] in, This is a rounding operation. This mapping is used to uniformize the grayscale distribution of the normalized shape image, thereby achieving illumination correction.

[0076] Step 4.4: Input the processed shape image into the MobileNet network. The MobileNet network's convolutional layers, depthwise separable convolutional layers, and pooling layers progressively extract local and global features from the processed shape image. The output of the last fully connected layer of the MobileNet network is selected as the deep shape feature. .

[0077] Specifically, the lightweight convolutional neural network MobileNet is used to extract deep shape features from the image. Specifically, the processed shape image is input into the MobileNet network, and local and global features of the image are extracted step by step through convolutional layers, depthwise separable convolutional layers, and pooling layers; the output of the last fully connected layer of the MobileNet network is selected as the deep shape feature. The feature dimensions can be adjusted according to actual needs, but it is necessary to ensure that the features have good discriminative power and stability.

[0078] Furthermore, the extracted deep morphological features This includes at least one of the following: the fuselage outline features, wing shape features, tail structure features, and marking pattern features of the drone.

[0079] Step 5: Input the processed IQ data, acoustic features, and deep shape features into the trained multimodal fusion model for multimodal fusion and optimization to obtain the authentication result.

[0080] In this embodiment, the multimodal fusion model includes an input layer, a preliminary modal feature fusion layer, a mapping and dynamic fusion layer, and a task head. The task head includes a parallel acoustic classification head, a physical layer regression head, and a shape classification head.

[0081] In one specific embodiment, step 5 may include: Step 5.1: Input the processed IQ data, acoustic features, and deep shape features into the trained multimodal fusion model through the input layer.

[0082] Step 5.2: Using a modal feature preliminary fusion layer, perform initial weighted fusion of the processed IQ data, acoustic features, and deep shape features to obtain preliminary joint features. .

[0083] Step 5.21: In the preliminary modal feature fusion layer, based on the importance scores of acoustic modality, physical layer modality, and shape modality in the authentication scenario. and Calculate the acoustic modal weighting coefficients respectively. and appearance 3, respectively represented as: , ,

[0084] Where, 0 < <1、0< <1、0< 3<1 and + + 3=1, and The score range is 0-10. and The requirements of the scenario are quantitatively assessed and then set accordingly.

[0085] Step 5.22: Perform joint processing on the processed IQ data, acoustic features, and deep morphological features to obtain preliminary joint features. , represented as:

[0086] in, Acoustic characteristics, For the processed IQ data, These are deep-seated external features. This represents element-wise multiplication of a scalar and a vector. This refers to the concatenation operation of the feature dimensions of the vectors, i.e., the weighted vectors. and By concatenating the columns, a complete joint feature vector is formed.

[0087] Step 5.3: Initially combine features Input mapping and dynamic fusion layer, based on preliminary joint features Obtain the mapped acoustic features Mapped IQ data and mapped shape features And through an attention mechanism, the mapped acoustic features Mapped IQ data and mapped shape features Dynamic weighted fusion is performed to obtain optimized joint features. .

[0088] Step 5.31: Map the acoustic features, processed IQ data, and deep shape features to the same dimensional space through linear transformation to obtain the mapped acoustic features. Mapped IQ data and the mapped morphological features .

[0089] Specifically, in this embodiment, the acoustic features are: , D 1 represents the acoustic feature dimension, and the processed IQ data is... , D 2 represents the physical layer feature dimension, and the extracted deep shape features are: ∈ , D 3 represents the shape feature dimension. This embodiment uses linear transformation to map the three modal features to the same dimensional space, ensuring consistency in weight calculation and resulting acoustic features. Mapped IQ data and mapped shape features They are represented as follows: , ,

[0090] in, , , It is a learnable linear projection matrix. , , , D To unify mapping dimensions, , , For bias vectors, , , , It represents the multiplication operation between matrices and vectors, achieving a unified feature dimension.

[0091] Step 5.32: Based on the mapped acoustic features Mapped IQ data and mapped shape features Attention weight coefficients for acoustic modalities are calculated using an MLP (Multi-Layer Perceptron with a single hidden layer) and a Softmax function. Attention weight coefficients of physical layer modes Attention weight coefficients for shape modality The effective contribution of each mode is quantified and expressed as:

[0092]

[0093] in, This represents a vector concatenation operation that merges the mapped features of the three modalities into one. Dimensional vector, σ( ) is the ReLU activation function, i.e., σ(x) = max(0,x), which enhances the nonlinear expressive power. ( () is the normalization function, ensuring that the weights satisfy 0 < <1、0< <1、0< <1, and + + =1, , Here is the learnable weight matrix for the attention network. , , K For the hidden layer dimension, , Let be the bias vector of the attention network. , .

[0094] This embodiment calculates the attention weight coefficients of three modal features through an attention mechanism, dynamically enhancing the contribution of effective features and suppressing the interference of ineffective features.

[0095] Step 5.33, Attention weight coefficients based on acoustic modality Attention weight coefficients of physical layer modes Attention weight coefficients for shape modality The mapped acoustic features Mapped IQ data and mapped shape features Dynamic weighted fusion is performed to enhance the contribution of effective features and suppress invalid interference, resulting in optimized joint features, denoted as:

[0096] in, For the optimized joint features, and The core logic is that when a certain modality is disturbed by factors such as environmental noise, channel fluctuations, and light occlusion, its attention weight will automatically decrease, while the weights of the other two modalities will increase accordingly, thus achieving dynamic complementarity and synergistic enhancement among the three modalities.

[0097] Step 5.4: Combine the optimized joint features Enter the task header to obtain the authentication result.

[0098] Specifically, the optimized joint features The trained task header is input, and a scalar probability value is output through a final fully connected layer of the task header. The scalar probability value is compared with a preset threshold. If the scalar probability value is greater than or equal to the preset threshold, the authentication result is determined to be valid; otherwise, the authentication result is determined to be invalid.

[0099] In an optional embodiment, the cross-entropy loss function for each acoustic mode is set separately. Mean square error loss function of physical layer modes and the cross-entropy loss function of shape mode. The total loss function is formed by combining the weighted sums. This is used to balance the training accuracy of the three modalities.

[0100] For the cross-entropy loss function of acoustic modes Optimized joint features The acoustic-related part (in) It will be sent to the acoustic sorting head and output Then, the cross-entropy loss function of the acoustic modes is calculated by combining the real labels. The classification training requirement for matching the extracted acoustic features with the identity of the authentication subject is represented as follows:

[0101] in, The number of training samples for acoustic modalities. No. e The acoustic modal ground truth labels for each acoustic modal training sample, for legitimate entities, For illegal entities, the value is 1. =0, For the first e The probability of a valid prediction of acoustic features for a training sample of acoustic modalities. The value range is [0,1]. It is the natural logarithm, used to quantify the deviation between the predicted probability and the true label.

[0102] For the mean squared error (MSE) loss function of physical layer modes Optimized joint features The physical layer related parts ( This will be sent to the physical layer regression header, and the output will be... Then, the mean square error loss function of the physical layer modes is calculated by combining the actual physical characteristics. The regression training requirements for adapting the processed IQ data to the real physical layer features are expressed as follows:

[0103] in, Where is the number of training samples for the physical layer modalities, and D is the dimension of the preprocessed IQ data. For the first f The true physical layer feature values ​​of the d-th dimension of each physical layer modality training sample. For the first f The prediction validity probability of the d-th physical layer feature of each physical layer modality training sample. The average loss is obtained by normalizing the sample number and feature dimension.

[0104] For the cross-entropy loss function of shape mode Optimized joint features The shape-related parts in the middle ( It will be sent to the shape sorting head and output Then, the cross-entropy loss function of the shape modality is calculated by combining the real labels. The classification training requirement for matching the extracted physical features with the identity of the authentication subject is represented as follows:

[0105] in, The number of training samples for the shape modality. For the first g The true labels for the shape modalities of training samples, where 1 represents a legitimate subject and 0 represents an illegitimate subject. For the first g The probability of a valid prediction of the shape features of a training sample of shape modality. The value range is [0,1].

[0106] Therefore, the total loss function Represented as:

[0107] in, for The weighting coefficients, for The weighting coefficients, for The weighting coefficients, and 0 < <1、0< <1、 + <1, the weighting coefficients are used to balance the training accuracy of the three modalities and can be dynamically adjusted according to the needs of the certification scenario. For example, in scenarios with complex environmental noise, the weighting coefficients can be increased. To enhance acoustic modal training; in scenarios with significant channel fluctuations, it can increase It focuses on physical layer modal training; in well-lit, unobstructed scenes, it can... Increase to enhance shape modality training.

[0108] Therefore, when training a multimodal fusion model, the total loss function is minimized using the backpropagation algorithm. This drives the convergence of the multimodal fusion model, thereby obtaining a well-trained multimodal fusion model.

[0109] Step 6: Regularly evaluate the fusion results and dynamically adjust the weighting coefficients used for fusion based on the actual application scenario.

[0110] Specifically, this embodiment can also dynamically adjust the acoustic modal weighting coefficients by periodically evaluating the fusion effect. and appearance To improve authentication accuracy. This embodiment focuses on authentication accuracy. ,robustness Resistance to attack The core indicator is used to calculate the overall performance score after normalization. ,like If the preset performance threshold is not met, the weights will be shifted towards the dominant modes corresponding to the weakest performance indicators, and the performance will be dynamically updated. and 3.

[0111] In one specific embodiment, step 6 may include: Step 6.1: Calculate the authentication accuracy rate Robustness indicators Anti-attack capability index .

[0112] Here, authentication accuracy Represented as:

[0113] in, The number of samples in which legitimate entities are correctly identified as legitimate. This represents the number of samples in which illegal entities were correctly identified as illegal. The number of samples where illegal entities were incorrectly identified as legitimate. This represents the number of samples where legitimate entities were incorrectly identified as illegitimate.

[0114] Here, robustness index Accuracy fluctuation coefficients under different environments are used for quantification to reflect the stability of the fusion model in complex scenarios, such as environmental noise, channel fluctuations, and illumination obstruction, and are expressed as follows:

[0115] in, The number of test environments is specified, and these environments can be, for example, quiet environments, noisy environments, long-distance environments, bright light environments, and occluded environments. ( k 1=1,2,…, ) is the first k Authentication accuracy in one environment This represents the average authentication accuracy across all environments. Robustness metrics. The value range is [0,1]. The closer it is to 1, the stronger the robustness, that is, the smaller the accuracy fluctuation.

[0116] Here, the attack resistance index To quantify the model's resistance to attacks in typical attack scenarios such as spoofing and man-in-the-middle attacks, the following expression is used:

[0117] in, The number of attack types, For the first m The number of samples in which attackers were incorrectly identified as legitimate entities under certain attack scenarios. For the first m Total number of attack samples across various attack scenarios. Anti-attack capability indicators. The value range is [0,1]. The closer it is to 1, the stronger the resistance to attack, that is, the lower the attack false positive rate.

[0118] Step 6.2: Assess the accuracy of the authentication. Robustness indicators Anti-attack capability index Normalization is performed separately to obtain the normalized authentication accuracy. Normalized robustness index Normalized attack resistance index , respectively represented as: , ,

[0119] in, , These represent the minimum and maximum accuracy values ​​from historical certifications. , These represent the minimum and maximum robustness values ​​in historical authentication. , These represent the minimum and maximum values ​​of resistance to attacks in historical authentication.

[0120] Step 6.3: Based on the authentication accuracy rate Robustness indicators and resistance to attack indicators Receive overall performance score , represented as:

[0121] in, , , All are importance weights, satisfying + + =1, , , It can be preset according to the needs of the scenario.

[0122] Step 6.4: Determine the overall performance score and preset performance thresholds The relationship, if S < Then, the weights are tilted towards the dominant modes corresponding to the weakest indicators, according to acoustic principles. Weighting coefficient Determine the updated acoustic modal weighting coefficients Updated and the updated appearance .

[0123] Specifically, based on the overall performance score Adjusting modal weights, the core logic is that when the overall performance does not meet expectations, i.e., S < When performance is satisfactory, the weights are shifted towards the advantageous modes corresponding to the weakest indicators; when performance meets the standards, the weights remain stable.

[0124] Here, the updated acoustic modal weighting coefficients Updated and the updated appearance They are represented as follows:

[0125]

[0126]

[0127] in, The step size for adjusting the weights needs to be preset to a small number, such as 0.05, to avoid excessive fluctuations in the weights. The contribution of acoustic modes to the improvement of the bottleneck index. This represents the contribution of physical layer modes to the improvement of the bottleneck performance index. The contribution of shape modality to the improvement of the weakest link index. , They are represented as follows:

[0128]

[0129]

[0130] in, , , The scores for the three metrics are based on the use of only acoustic modalities. , , The scores for the three metrics are based on the use of only the physical layer modes. , , The scores for the three metrics are based on the shape modality alone. , , These are the preset thresholds for each indicator.

[0131] Step 6.5: Update the acoustic modal weighting coefficients. Updated And the updated Feedback is sent to step 5.21 to complete one optimization.

[0132] This invention proposes a multimodal authentication method for unmanned aerial vehicles (UAVs) based on acoustic, physical layer, and shape features. It primarily addresses the problems of insufficient authentication accuracy, poor robustness, and low attack threshold inherent in single authentication methods. Acoustic authentication suffers from environmental sensitivity; minute changes in the physical environment can cause unpredictable distortions in acoustic features, compromising the stability and uniqueness of authentication characteristics. Physical layer authentication is channel-dependent, relying on real-time wireless channel characteristics, which dynamically change with spatial location, propagation path, and other factors. Furthermore, legitimate and illegitimate entities can share the channel feature space, causing authentication features to lose their unique identifier attribute. Shape feature authentication is susceptible to factors such as shooting angle, lighting conditions, and occlusion. The implementation scheme of this invention is as follows: The authentication object is selected as a drone, and the authentication scenario is any scenario in which the drone is used; the original acoustic signal, communication signal data, and original shape image of the drone are collected simultaneously; high-frequency noise is removed based on wavelet transform, and then the acoustic signal data is preprocessed by operations such as amplitude normalization and time-frequency alignment; finally, core acoustic features are extracted, extracting acoustic features that are strongly correlated with the identity of the authentication subject and are difficult to replicate from the preprocessed signal; communication signal data is processed by sampling truncation, downsampling, and sampling normalization; denoising, size normalization, and illumination correction preprocessing are performed on the shape image data to extract deep shape features such as fuselage outline and wing shape; a three-stream encoder + attention fusion architecture is adopted for the three modal features, and their feature dimensions are weighted and fused to form a joint feature representation, and the modal weights are dynamically adjusted based on the attention mechanism; during training, multiple loss functions are used to balance the learning of each modality; finally, the fusion effect is evaluated periodically, and the fusion weights are adjusted according to the actual scenario.

[0133] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0134] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, disclosure, and appended claims in carrying out the claimed invention. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0135] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, any modifications made without departing from the inventive concept should be considered within the scope of protection of the present invention.

Claims

1. A physical layer, acoustic and profile feature based multi-modal authentication method for drones, characterized in that, include: Simultaneously acquire the drone's communication signal data, raw acoustic signals, and raw shape images; The communication signal data is subjected to preprocessing operations of sampling truncation, downsampling, and sampling normalization in sequence to obtain the processed IQ data; The original acoustic signal is subjected to wavelet transform, amplitude normalization, and time-frequency alignment operations in sequence. Then, acoustic features that are strongly correlated with the UAV identity and are difficult to replicate are extracted from the preprocessed acoustic signal. The original shape image is sequentially subjected to image denoising, size normalization, and illumination correction to obtain the processed shape image. Deep shape features that are strongly correlated with the UAV's identity and are difficult to replicate are extracted from the processed shape image. The processed IQ data, the acoustic features, and the deep shape features are input into the trained multimodal fusion model for multimodal fusion and optimization to obtain the authentication result.

2. The method of claim 1, wherein, Simultaneously acquire the UAV's communication signal data, raw acoustic signals, and raw shape images, including: The drone's original acoustic signals are acquired through a mobile terminal, its original shape image is acquired through an image acquisition device, and its communication signal data is acquired through software-defined radio. The acquisition of UAV communication signal data via software-defined radio includes: The radio wave signals transmitted by the drone are received by radio and then down-converted by the radio frequency front-end to obtain the baseband analog signal. a continuous-time signal segment corresponding to a preamble sequence intercepted in the baseband analog signal performing quadrature demodulation to demodulate into analog-domain in-phase and quadrature components and quadrature components respectively wherein, a carrier frequency that is a preamble sequence, is time; For the in-phase components respectively and the orthogonal components Perform low-pass filtering to obtain the in-phase component after low-pass filtering. and the quadrature components after low-pass filtering ; via analog-to-digital converter at sampling frequency The low-pass filtered in-phase components are respectively and the quadrature components after low-pass filtering Sampling is performed to obtain the in-phase component sequence in the discrete domain. orthogonal component sequences in the discrete domain , respectively represented as: in, =0,1,2,..., 1, The number of sampling points in the leading sequence. = / , The total duration of the leading sequence. The sampling interval is... =1 / , ≥2 ; The discrete-domain in-phase component sequence and the discrete domain orthogonal component sequence Communication signal data combined into a complex matrix form is represented as follows: Where, IQ∈ , It is a complex field, and the matrix dimension is the number of sampling points in the preceding sequence. ×1, where j is the imaginary unit.

3. The UAV multimodal authentication method according to claim 1, characterized in that, The communication signal data is preprocessed by sequentially performing sampling truncation, downsampling, and sampling normalization to obtain processed IQ data, including: The communication signal data is sampled and truncated so that the remaining valid data segments are used as the truncated IQ data; The truncated IQ data is downsampled according to a preset downsampling rate to obtain downsampled IQ data. The downsampled IQ data is normalized to map the downsampled IQ data to [ The processed IQ data is obtained by analyzing the interval [1, 1].

4. The UAV multimodal authentication method according to claim 1, characterized in that, The original acoustic signal is subjected to wavelet transform, amplitude normalization, and time-frequency alignment operations in sequence. Then, acoustic features that are strongly correlated with the UAV's identity and difficult to replicate are extracted from the preprocessed acoustic signal, including: Low-pass filter coefficients using the db8 wavelet basis and high-pass filter coefficients The original acoustic signal is decomposed into 5 layers, with the decomposition process proceeding recursively from layer 1 to layer 5, yielding the low-frequency coefficients and high-frequency coefficients after decomposition from layer 1 to layer 5, and the low-frequency coefficients after decomposition at layer j. and the high-frequency coefficients after the j-th layer decomposition They are represented as follows: Where j = 1, 2, 3, 4, 5, the initial condition is when j = 0. =x[ ], For the discrete index of coefficients at each level, x[ [ ] represents the discrete sampling sequence of the original acoustic signal. ; Based on adaptive threshold The high-frequency coefficients after decomposition of the j-th layer Perform soft thresholding to obtain the high-frequency coefficients of the j-th layer after thresholding. , , This is the noise standard deviation estimate for the high-frequency coefficients of the j-th layer, where N is the number of sampling points of the original acoustic signal; Based on the low-frequency coefficients after the 5th layer decomposition and the high-frequency coefficients after thresholding from the 1st to the 5th layers, the denoised acoustic signal is recovered through 5-layer wavelet reconstruction. The amplitude of the denoised acoustic signal is normalized to obtain the normalized acoustic signal. The normalized acoustic signal is time- and frequency-aligned to align with the communication signal data and original shape image from the same UAV at the same time reference and frequency resolution, thus obtaining the pre-processed acoustic signal. The acoustic features that are strongly correlated with the UAV's identity and difficult to replicate are extracted from the preprocessed acoustic signals. These acoustic features are either the audio signal features generated by the rotation of the propeller during UAV flight or the audio signal features generated by the operation of the motor during UAV flight.

5. The UAV multimodal authentication method according to claim 1, characterized in that, The original shape image is sequentially subjected to image denoising, size normalization, and illumination correction to obtain a processed shape image. Deep shape features strongly correlated with the UAV's identity and difficult to replicate are extracted from the processed shape image, including: The original shape image is denoised using Gaussian filtering to obtain a denoised shape image. The size of the denoised shape image is normalized by bilinear interpolation to obtain a normalized shape image. The normalized shape image is subjected to illumination correction by histogram equalization to enhance image contrast, resulting in the processed shape image. The processed shape image is input into the MobileNet network. Local and global features of the processed shape image are extracted step by step through the convolutional layer, depthwise separable convolutional layer, and pooling layer of the MobileNet network. The output of the last fully connected layer of the MobileNet network is selected as the deep shape feature.

6. The UAV multimodal authentication method according to claim 1, characterized in that, The multimodal fusion model includes an input layer, a preliminary modal feature fusion layer, a mapping and dynamic fusion layer, and a task head. The task head includes a parallel acoustic classification head, a physical layer regression head, and a shape classification head. The processed IQ data, the acoustic features, and the deep shape features are input into a trained multimodal fusion model for multimodal fusion and optimization to obtain authentication results, including: The processed IQ data, the acoustic features, and the deep shape features are input into the trained multimodal fusion model through the input layer; The modal feature preliminary fusion layer performs initial weighted fusion of the processed IQ data, the acoustic features, and the deep shape features to obtain preliminary joint features. ; The preliminary joint features Input the mapping and dynamic fusion layer, based on the preliminary joint features Obtain the mapped acoustic features Mapped IQ data and mapped shape features And through an attention mechanism, the mapped acoustic features Mapped IQ data and mapped shape features Dynamic weighted fusion is performed to obtain optimized joint features. ; The optimized joint features Input the task header to obtain the authentication result.

7. The UAV multimodal authentication method according to claim 6, characterized in that, The modal feature preliminary fusion layer performs initial weighted fusion of the processed IQ data, the acoustic features, and the deep shape features to obtain preliminary joint features. ,include: In the preliminary modal feature fusion layer, the importance scores of acoustic modality, physical layer modality, and shape modality in the authentication scenario are used. and Calculate the acoustic modal weighting coefficients and appearance 3, respectively represented as: ， ， Where, 0 < <1、0< <1、0< 3<1 and + + 3=1, and The score range is 0 to 10; The processed IQ data, the acoustic features, and the deep morphological features are jointly processed to obtain preliminary joint features. , is represented as: in, Acoustic characteristics, For the processed IQ data, These are deep-seated external features. This represents element-wise multiplication of a scalar and a vector. This is an operation that concatenates the feature dimensions of a vector.

8. The UAV multimodal authentication method according to claim 6, characterized in that, Based on the aforementioned preliminary joint features Obtain the mapped acoustic features Mapped IQ data and mapped shape features And through an attention mechanism, the mapped acoustic features Mapped IQ data and mapped shape features Dynamic weighted fusion is performed to obtain optimized joint features. ,include: The acoustic features, the processed IQ data, and the shape features are mapped to the same dimensional space through linear transformation to obtain the mapped acoustic features. Mapped IQ data and mapped shape features , respectively represented as: ， ， in, , , It is a learnable linear projection matrix. , , , D To unify mapping dimensions, D 1 represents the acoustic feature dimension. D 2 represents the physical layer feature dimension. D 3 represents the external shape feature dimension. , , For bias vectors, , , ; Based on mapped acoustic features Mapped IQ data and mapped shape features Attention weight coefficients of acoustic modes are calculated using MLP and Softmax functions. Attention weight coefficients of physical layer modes Attention weight coefficients for shape modality , is represented as: Where, σ( ) is the ReLU activation function. ( ) is the normalization function. , Here is the learnable weight matrix of the attention network. , , K For the hidden layer dimension, , Let be the bias vector of the attention network. , ; Attention weight coefficients based on acoustic modalities Attention weight coefficients of physical layer modes Attention weight coefficients for shape modality The mapped acoustic features Mapped IQ data and mapped shape features Dynamic weighted fusion is performed to obtain optimized joint features. .

9. The UAV multimodal authentication method according to claim 6, characterized in that, The total loss function for training the multimodal fusion model is: in, For the total loss function, Let cross-entropy loss function be the acoustic modal. Let be the mean square error loss function of the physical layer modes. Let the cross-entropy loss function be the shape mode. and The weighting coefficient is 0 < <1、0< <1、 + <1, The number of training samples for acoustic modalities. No. e The true acoustic modal labels of each acoustic modal training sample. For the first e The probability of a valid prediction of acoustic features for a training sample of acoustic modalities. Where is the number of training samples for the physical layer modalities, and D is the dimension of the preprocessed IQ data. For the first f The true physical layer feature values ​​of the d-th dimension of each physical layer modality training sample. For the first f The predicted value of the d-th physical layer feature of each physical layer modality training sample. The number of training samples for the shape modality. For the first g True labels for the shape modalities of training samples. For the first g The probability of predicting the legitimacy of the shape features of a training sample of shape modality.

10. The UAV multimodal authentication method according to claim 6, characterized in that, Also includes: Calculate authentication accuracy Robustness indicators Anti-attack capability index ; For authentication accuracy Robustness indicators Anti-attack capability index Normalization is performed separately to obtain the normalized authentication accuracy. Normalized robustness index Normalized attack resistance index ; Based on authentication accuracy Robustness indicators and resistance to attack indicators Receive overall performance score , is represented as: in, , , All are importance weights, satisfying + + =1; Determine the overall performance score and preset performance thresholds The relationship, if S < Then, the weights are tilted towards the advantageous modes corresponding to the weakest indicators, based on the acoustic mode weighting coefficients. Determine the updated acoustic modal weighting coefficients Updated and the updated appearance , respectively represented as: in, Adjust the step size for weights. The contribution of acoustic modes to the improvement of the bottleneck index. This represents the contribution of physical layer modes to the improvement of the bottleneck performance index. This represents the contribution of shape modality to the improvement of the weakest link index.

Images