Unmanned aerial vehicle image transmission signal modulation mode identification system and method based on multi-dimensional feature cooperation

Abstract

The invention discloses an unmanned aerial vehicle image transmission signal modulation mode identification system and method based on multi-dimensional feature cooperation. The method comprises the following steps: (S100) collecting and preprocessing unmanned aerial vehicle communication signal data; (S200) the one-dimensional features are extracted and remodeled into a two-dimensional feature map, and normalization processing is carried out; estimating a two-dimensional time-frequency diagram of the image transmission signal; (S300) feature combination: combining the obtained remolded graph and the time-frequency graph as four channels to form a multi-channel feature graph; (S400) generating a training set and a test set; (S500) constructing a VGGR16ChannelAttention deep learning network, and carrying out the training of the VGGR16ChannelAttention deep learning network; and (S600) identifying the image transmission signal modulation mode of the unmanned aerial vehicle to be identified. According to the method, the time-frequency analysis technology and the deep learning technology are fused, the method surpasses a traditional method through time-frequency synchronous optimization transformation, and complex signals can be accurately analyzed.

Description

Technical Field

[0001] The present invention relates to a method for identifying a modulation mode of an image transmission signal of an unmanned aerial vehicle (UAV), and in particular to a system and method for identifying a modulation mode of an image transmission signal of an unmanned aerial vehicle (UAV) based on multi-dimensional feature collaboration. Background Art

[0002] With the rapid development of drone technology, drones are increasingly used in many fields such as aerial photography, environmental monitoring, agricultural plant protection, and traffic monitoring. When performing tasks, drones transmit key information such as images and videos in real time through image transmission systems, providing important decision-making basis for ground control stations. However, the modulation method of drone image transmission signals is directly related to the efficiency and security of data transmission. Therefore, accurately identifying the modulation method of drone image transmission signals has become an important research topic.

[0003] Traditional methods for identifying modulation modes of UAV image transmission signals mainly rely on signal processing technologies, such as spectrum analysis, feature extraction, and pattern matching. Although these methods can identify the modulation mode of signals to a certain extent, they often seem powerless when faced with complex and changing signal environments and an increasing number of modulation types. In addition, traditional methods usually require manual design of feature extractors, which not only requires a large workload, but also has difficulty in dealing with the nonlinearity and time-varying nature of signals.

[0004] As an important branch of artificial intelligence, deep learning has achieved remarkable results in image recognition, speech recognition, natural language processing and other fields with its powerful automatic feature extraction and learning capabilities. In recent years, deep learning technology has also been gradually introduced into the field of communication signal processing, especially in modulation mode recognition, showing great potential. By building a deep neural network model, deep learning can automatically extract features from signal data and accurately identify different modulation modes through large-scale data training.

[0005] The method for identifying the modulation mode of UAV image transmission signals based on deep learning not only has broad application prospects and important research value, but also has significant necessity in improving recognition accuracy, enhancing robustness, supporting multi-modulation type recognition, and realizing automation and intelligence. By making full use of the advantages of deep learning, accurate and efficient recognition of the modulation mode of UAV image transmission signals can be achieved, providing strong support for the development and application of UAV technology. In the future, with the continuous development and improvement of deep learning technology, it is believed that the method for identifying the modulation mode of UAV image transmission signals based on deep learning will play an important role in more fields. Summary of the invention

[0006] The purpose of the present invention is to provide a system and method for identifying the modulation mode of UAV image transmission signals based on multi-dimensional feature collaboration. The method integrates time-frequency analysis and deep learning technology, surpasses traditional methods through time-frequency synchronization optimization transformation, and can accurately analyze complex signals.

[0007] In order to achieve the above object, the present invention provides a method for identifying the modulation mode of a UAV image transmission signal based on multi-dimensional feature collaboration, characterized in that the method comprises:

[0008] (S100) UAV communication signal data collection and preprocessing

[0009] The analog / data device inside the receiver performs analog-to-digital conversion on the received analog signal, and converts the received signal into two digital baseband signals, the in-phase component si(t) and the orthogonal component si(t) through orthogonal demodulation and digital down-conversion, and then converts the s i The two signals s(t) and sq(t) are combined to obtain the time domain signal s(t) = s i (t)+j·s q (t), j represents the imaginary unit;

[0010] (S200) extracting and reshaping one-dimensional features into a two-dimensional feature map, and performing normalization processing; estimating a two-dimensional time-frequency map of the image transmission signal; wherein the one-dimensional features include: instantaneous amplitude, instantaneous phase and instantaneous frequency;

[0011] The estimated two-dimensional time-frequency diagram of the image transmission signal includes:

[0012] S221) Perform time-frequency synchronization optimization transformation on the combined time domain signal to obtain T tfso Time-frequency distribution: The combined time domain signal is analyzed with time as the horizontal axis and frequency as the vertical axis. The collected data is processed by dynamic harmonic analysis and transformation, and each output can obtain T tfso Time spectrum, and then calculate the T of each set of data along the time axis tfso The maximum value of the time spectrum is obtained by splicing the maximum values ​​of each group of data to obtain the T of all sample data. tfso Time-frequency spectrum maximum diagram;

[0013] The expression of the time-frequency synchronization optimization transformation is:

[0014]

[0015] In formula (1), tfr1(t,f)=FFT(s(t)·h(t)), tfr2(t,f)=FFT(s(t)·h'(t)); s(t) is the time domain signal; h(t) is the window function; h'(t) is the derivative of the window function h(t); FFT is Fourier transform, j is the imaginary unit; the adjustment factor N is the number of points in FFT, h len is the window length; Im means taking the imaginary part;

[0016] S222) Extraction of UAV image transmission signal based on robust principal component analysis RPCA: According to the difference in the correlation between the remote control signal and the image transmission signal in the time-frequency domain, the remote control signal and the image transmission signal are respectively allocated to the sparse matrix and the low-rank matrix using the low-rank matrix recovery method; RPCA simultaneously recovers the low-rank matrix A and the sparse matrix E by minimizing the L2 norm of the reconstruction error to separate the UAV image transmission signal information; the minimized reconstruction error is the time-frequency spectrum matrix T tfso The difference between (t, f) and the sum of the low-rank matrix A and the sparse matrix E;

[0017] (S300) Feature merging: merging the instantaneous amplitude reshaping map, the instantaneous phase reshaping map, the instantaneous frequency reshaping map and the time-frequency map obtained in step (S200) as four channels into a multi-channel feature map;

[0018] (S400) Generate a training set and a test set: Each drone signal sample corresponds to four features: an instantaneous amplitude reshaping map, an instantaneous phase reshaping map, an instantaneous frequency reshaping map, and a time-frequency map, which are used to generate a training set and a test set;

[0019] (S500) constructing a VGGR16_Channel Attention deep learning network and training it: embedding a Channel Attention module in the VGG16 deep learning network, inserting the Channel Attention module after each convolutional layer block and before the pooling layer, and coordinating the multi-dimensional features of the input layer, adjusting the number of input channels of the first convolutional layer that accepts 4-channel input to 4 so that it can accept feature maps of 4 channels, obtaining a VGGR16_Channel Attention deep learning network, and training it;

[0020] (S600) Identify the modulation mode of the drone image transmission signal to be identified: input the sample image of the drone signal to be identified into the trained VGGR16_Channel Attention deep learning network, and output the recognition result of the drone image transmission signal.

[0021] Preferably, in step (S200), extracting the one-dimensional features and reshaping them into a two-dimensional feature map, and performing normalization processing, comprises:

[0022] S211) Feature extraction: extracting the instantaneous amplitude, instantaneous phase and instantaneous frequency of the UAV image transmission signal from the collected time domain signal s(t);

[0023] For the time domain signal s(t) = s i(t)+j·s q (t),

[0024] The instantaneous amplitude is:

[0025] The instantaneous phase is:

[0026] The instantaneous frequency is:

[0027] In the formula, arctan() means to find the inverse tangent. represents the derivative;

[0028] S212) Feature reshaping: reshaping each one-dimensional feature to obtain a two-dimensional feature map: an instantaneous amplitude reshaping map, an instantaneous phase reshaping map, and an instantaneous frequency reshaping map;

[0029] S213) Feature normalization: Each two-dimensional feature map is normalized to ensure that all feature maps are in the same numerical range, and the values ​​in the two-dimensional array are normalized to the range of [-1,1].

[0030] Preferably, in step S221), h(t) is

[0031] Preferably, in step S222), the RPCA problem is transformed into a convex optimization problem, the goal of which is to minimize the sum of the nuclear norm of the low-rank matrix A and the L1 norm of the sparse matrix E, while ensuring that D = A + E. The optimization problem is expressed as:

[0032]

[0033] in, It means finding the minimum value for the low-rank matrix A and the sparse matrix E, and st represents the constraint condition; |||| * represents the nuclear norm of the matrix, ||||1 represents the L1 norm of the matrix, and λ is the balance parameter between low-rank terms and sparse terms;

[0034] Solving equation (2), we can obtain the low-rank matrix A, which is also the separated UAV image transmission signal information.

[0035] Preferably, said λ=0.1.

[0036] Preferably, in step (S400), the training set and the test set are combined in a ratio of 7:3.

[0037] Preferably, in step (S500), the structure of the VGGR16_Channel Attention deep learning network includes: an input layer, a convolutional layer block one, a channel attention module one, a pooling layer one, a convolutional layer block two, a channel attention module two, a pooling layer two, a convolutional layer block three, a channel attention module three, a pooling layer three, a convolutional layer block four, a channel attention module four, a pooling layer four, a convolutional layer block five, a channel attention module five, a pooling layer five, a fully connected layer and a Softmax classifier connected in sequence.

[0038] Preferably, in step (S500), the training of the VGGR16_Channel Attention deep learning network includes: using a random sampling algorithm to disrupt the order of sample images in the training set to obtain a training set after the disorder; setting the number of training cycles Epoch, after all samples are sent to the neural network, after multiple iterations of training to ensure that all data are in the convergence stage at the end; selecting the batch size BatchSize, setting the learning rate LearningRate, and modifying the loss function to a weighted loss function based on the mean square error MSE to tal ; Input the shuffled drone sample image training set and validation set into the VGGR16_Channel Attention deep learning network, and continuously iterate and update the parameters until the weighted loss function converges to obtain the trained VGGR16_Channel Attention deep learning network.

[0039] Preferably, w1, w2, w3, and w4 are set to represent the weights of the input feature instantaneous amplitude reshaping map, instantaneous phase reshaping map, instantaneous frequency reshaping map, and time-frequency map, respectively. The weighted loss function based on the mean square error is calculated for each feature separately, and then the weighted sum is calculated:

[0040] For a certain sample, the model's predicted values ​​for the four features are The true values ​​are y1, y2, y3, and y4 respectively. Then, the loss function based on the mean square error is calculated as:

[0041] MSE total =w1×MSE1+w2×MSE2+w3×MSE3+w4×MSE4

[0042] Among them, the mean square error loss MSE of each feature i as follows:

[0043]

[0044] Where n is the number of elements in the sample, and ikare the predicted value and true value of the kth element of the ith feature, respectively.

[0045] Another object of the present invention is to provide a drone image transmission signal modulation mode recognition system based on multi-dimensional feature collaboration, which adopts the drone image transmission signal modulation mode recognition method based on multi-dimensional feature collaboration to identify the drone image transmission signal modulation mode.

[0046] The system and method for identifying modulation mode of UAV image transmission signal based on multi-dimensional feature collaboration of the present invention have the following advantages:

[0047] (1) The method of the present invention combines time-frequency analysis with deep learning technology, surpasses traditional methods through time-frequency synchronization optimization transformation, and can accurately analyze complex signals. It synergistically utilizes one-dimensional features (including instantaneous amplitude, instantaneous phase, and instantaneous frequency) and two-dimensional image features (i.e., frequency spectrum) to achieve sophisticated multi-feature representation. At the same time, the present invention uses a VGGR16 network enhanced by a channel attention mechanism to improve the performance of visual tasks. This method maintains the lightweight and efficient model and enhances the feature learning capability, thereby pushing the drone image transmission signal modulation mode recognition technology to a new level;

[0048] (2) The method of the present invention adopts the deep convolutional neural network VGGR16_Channel Attention, which effectively extracts high-level features of images or signals by stacking multiple convolutional layers, especially the subtle and complex modulation features in the drone image transmission signal. Its small convolution kernel design not only reduces the parameters, but also enhances the nonlinearity of the network and enriches the feature representation;

[0049] (3) The method of the present invention adopts the deep convolutional neural network VGGR16_Channel Attention, introduces an efficient channel attention mechanism, dynamically adjusts the importance of feature channels, enables the model to focus on key features, improves recognition accuracy and efficiency, reduces redundant information, and reduces computational costs;

[0050] (4) The method of the present invention also exhibits wide application adaptability, can identify multiple modulation modes, and has strong robustness to channel noise, interference, etc. At the same time, with the help of pre-trained weights and modern deep learning frameworks, the model is easy to train and optimize, converges quickly, and has strong generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of the method for identifying the modulation mode of UAV image transmission signals based on multi-dimensional feature collaboration of the present invention.

[0052] Figure 2 It is a schematic diagram of the modulation mode identification system of the UAV image transmission signal based on multi-dimensional feature collaboration of the present invention.

[0053] Figure 3 It is a time-frequency spectrum diagram of the time-frequency synchronization optimization transformation of the UAV image transmission signal of the present invention.

[0054] Figure 4 This is a block diagram of the VGGR16_Channel Attention deep learning network structure of the present invention.

[0055] Figure 5 This is the confusion matrix diagram of the VGGR16_Channel Attention deep learning network of the present invention. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be noted that if the specific conditions are not specified in the examples, the experiments were carried out according to conventional conditions or the conditions recommended by the manufacturer. If the manufacturers of the reagents or instruments are not specified, they are all conventional products that can be purchased from the market.

[0058] Example 1

[0059] A method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration, such as Figure 2 As shown, it is a schematic diagram of the UAV image transmission signal modulation mode identification system based on multi-dimensional feature collaboration of the present invention, wherein the UAV transmits the image transmission signal to the ground remote control station, the remote control station sends the remote control signal to the UAV, and the monitoring device collects the remote control signal and the image transmission signal.

[0060] like Figure 1 As shown, it is a flow chart of the method for identifying the modulation mode of the UAV image transmission signal based on multi-dimensional feature collaboration of the present invention, and the identification method comprises:

[0061] (S100) UAV communication signal data collection and preprocessing.

[0062] The data acquisition system based on the AD9361 zero-IF receiver architecture is used to collect signals from a certain type of drone from DJI. The analog / digital device inside the receiver performs analog-to-digital conversion on the received analog signal, and converts the received signal into two digital baseband signals, the in-phase component si(t) and the orthogonal component sq(t) through orthogonal demodulation and digital down-conversion. i (t), s q (t) The two signals are combined to obtain the time domain signal s(t) = si (t)+j·s q (t), j represents the imaginary unit, usually

[0063] (S200) One-dimensional features (instantaneous amplitude, instantaneous phase and instantaneous frequency) are extracted and reshaped, and a two-dimensional time-frequency diagram of the image transmission signal is estimated.

[0064] S210) One-dimensional feature extraction and reshaping, including:

[0065] S211) Feature extraction: Extract the instantaneous amplitude, instantaneous phase and instantaneous frequency of the drone image transmission signal from the collected time domain signal s(t)

[0066] For the time domain signal s(t) = s i (t)+j·s q (t)

[0067] Instantaneous amplitude:

[0068] Instantaneous phase:

[0069] Instantaneous frequency:

[0070] In the formula, arctan() means to find the inverse tangent. It means to find the derivative.

[0071] S212) Feature reshaping: reshape each one-dimensional feature sequence (instantaneous amplitude, instantaneous phase and instantaneous frequency) to obtain three 224×224 two-dimensional feature maps (instantaneous amplitude reshaping map, instantaneous phase reshaping map and instantaneous frequency reshaping map).

[0072] Considering that the VGG16 model was originally designed to process two-dimensional image data, and the input includes one-dimensional features (instantaneous amplitude, instantaneous phase, and instantaneous frequency), it is necessary to reshape the one-dimensional feature sequence into a two-dimensional feature map.

[0073] Assume that the one-dimensional instantaneous amplitude A1(t) (1)

[0074] =(a1, a2, ..., a 1024 , a 1+1*1024 , a 2+1*1024 , …, a 1024+1*1024 , …, a 1+48*1024 , a 2+48*1024 , …, a 1024+48*1024 )

[0075] It is 1024×49=50176 points of data. Among them, a1, a2, ... are the original values ​​of the instantaneous amplitude A1(t).

[0076] Reshape to obtain the two-dimensional instantaneous amplitude A2(t), which is a 224×224 matrix:

[0077]

[0078] Similarly, the two-dimensional instantaneous phase P2(t) is obtained, which is a 224×224 matrix; the two-dimensional instantaneous frequency F2(t) is obtained, which is a 224×224 matrix.

[0079] It is important to note that the reshaping operation itself does not change the nature of the data, it only changes the arrangement of the data (the reshaping process does not involve operations such as padding or truncation that may have a certain impact on the data). The conversion from one dimension to two dimensions is a change in format and should not affect the information content carried by the data.

[0080] S213) Feature normalization

[0081] Each 2D feature map (or reshaped map) is normalized to ensure that all feature maps are in the same numerical range.

[0082] Normalization helps the model converge and improve performance by normalizing the values ​​in the two-dimensional array to the range of [-1, 1].

[0083] S220) estimating a two-dimensional time-frequency diagram of the image transmission signal, comprising:

[0084] S221) Perform time-frequency synchronization optimization transformation on the combined time domain signal to obtain T tfso Time-frequency distribution.

[0085] Since the drone signal is a non-stationary signal, the time-frequency synchronization optimization transformation is used to analyze the merged time domain signal with time as the horizontal axis and frequency as the vertical axis. By processing the collected 8000×2048 data points, each group of data is 2048 points, and a total of 8000 analyses are performed, each time representing a time point. The 2048 points correspond to the signal frequency within the 40MHz bandwidth, and the frequency of each point is 0.019532625MHz.

[0086] The above has been processed 8000 times by dynamic harmonic analysis and transformation, and each output can get T tfso Time spectrum; then calculate the T of each set of data along the time axis tfso The maximum value of the time spectrum is obtained by splicing the maximum values ​​of each group of data to obtain the T of all sample data. tfso Time-frequency spectrum maximum plot.

[0087] The time-frequency synchronization optimization transformation expression is:

[0088]

[0089] And tfr1(t, f)=FFT(s(t)·h(t)), tfr2(t,f)=FFT(s(t)·h'(t)).

[0090] Among them, s(t) is the time domain signal, h(t) is the window function, here we take h′(t) is the derivative of the window function h(t), where FFT is Fourier transform, j is the imaginary unit; adjustment factor N is the number of points in FFT, h len is the window length; Im represents the imaginary part.

[0091] The time-frequency synchronous optimization transform improves the resolution of time-frequency analysis and can capture the time-frequency characteristics of the signal more finely; it enhances the accuracy of signal component extraction and reduces cross-term interference in the time-frequency domain; it better adapts to the characteristics of non-stationary signals and maintains stable time-frequency analysis capabilities; it improves the efficiency of signal processing overall, especially in scenarios requiring high-precision time-frequency analysis; at the same time, the time-frequency synchronous optimization transform supports a wide range of application fields, providing strong support for research and applications in related fields.

[0092] S222) Extraction of UAV image transmission signal based on Robust Principal Component Analysis (RPCA).

[0093] According to the difference in the correlation between the remote control signal and the image transmission signal in the time-frequency domain, the method of low-rank matrix recovery is used to allocate the parts with strong correlation to the low-rank matrix, while the parts with weak correlation are stored in the sparse matrix. The image transmission signal exists continuously within the bandwidth range, while the remote control signal is a stepped oblique (or broken) line with a frequency changing over time in the time-frequency domain. Therefore, when separating the two, most of the image transmission signal is divided into the low-rank matrix, and the remote control signal is divided into the sparse matrix, thereby achieving the purpose of separating the remote control signal from the image transmission signal.

[0094] RPCA minimizes the reconstruction error (the instant spectrum matrix T tfso The L2 norm of the difference between (t, f) and the sum of the low-rank matrix A and the sparse matrix E) is used to simultaneously recover the low-rank matrix A and the sparse matrix E. RPCA shows strong robustness when processing data containing noise and outliers.

[0095] The RPCA problem is transformed into a convex optimization problem, the goal is to minimize the sum of the nuclear norm of the low-rank matrix A and the L1 norm of the sparse matrix E, while ensuring that D = A + E. The optimization problem can be expressed as:

[0096]

[0097] in, It means finding the minimum value for the low-rank matrix A and the sparse matrix E, and st represents the constraint condition; |||| * represents the nuclear norm of the matrix, ||||1 represents the L1 norm of the matrix, and λ is the balance parameter between the low-rank term and the sparse term, where λ=0.1.

[0098] Solving equation (2), we get the low-rank matrix A, which is the separated UAV image transmission signal information. In practice, the CVX module in Matlab tool is used to solve convex optimization problems and get the low-rank matrix A.

[0099] like Figure 3 As shown in the figure, this is a time-frequency spectrum diagram of the time-frequency synchronization optimization transformation of the UAV image transmission signal, where the modulation modes of the UAV image transmission signal are the time-frequency spectrum diagrams corresponding to six types, namely 2PSK, 8PSK, 4QAM, 16QAM, 64QAM, and 256QAM.

[0100] (S300) Feature merging: merging the instantaneous amplitude reshaping map, the instantaneous phase reshaping map, the instantaneous frequency reshaping map and the time-frequency map as four RGB channels into a multi-channel feature map.

[0101] Feature merging is the process of combining multiple two-dimensional feature maps into a multi-channel feature map. In this example, there are four two-dimensional feature maps, representing the instantaneous amplitude reshaping map, the instantaneous phase reshaping map, the instantaneous frequency reshaping map, and the time-frequency map. These four feature maps will be merged into a multi-channel feature map as four RGB channels (actually this is an analogy because the feature map is not a real color channel).

[0102] The specific steps are as follows:

[0103] 1) Prepare feature map:

[0104] Make sure that each feature map has the same size, for example, 224×224.

[0105] 2) Merge feature maps:

[0106] Create a new empty feature map with the same dimensions as the single feature map but with 4 channels.

[0107] Copy the first feature map (the instantaneous amplitude reshape map) to the first channel of the new feature map.

[0108] Copy the second feature map (the instantaneous phase reshaping map) to the second channel of the new feature map.

[0109] Copy the third feature map (the instantaneous frequency reshaped map) to the third channel of the new feature map.

[0110] Copy the fourth feature map (time-frequency map) to the fourth channel of the new feature map.

[0111] 3) Process the merged feature map:

[0112] The merged feature map is now a 224×224×4 multi-channel feature map.

[0113] This multi-channel feature map is used as input to the deep learning model.

[0114] (S400) Generate training set and test set

[0115] Referring to the data format of a certain model of DJI drone, the modulation modes of the image transmission signals generated by simulation are 2PSK, 8PSK, 4QAM, 16QAM, 64QAM, 256QAM, etc. For each modulation signal, the signal-to-noise ratio ranges from -10dB to 15dB, with an interval of 5dB. At each signal-to-noise ratio, 1000 drone signal samples are obtained, and each drone signal sample corresponds to four features (instantaneous amplitude reshaping map, instantaneous phase reshaping map, instantaneous frequency reshaping map and time-frequency map), with 1000*4 two-dimensional feature maps; 700*4 of them are used for training set, and the remaining 300*4 are used for test set.

[0116] Therefore, there are a total of 36000*4 drone 2D feature map samples, of which 25200*4 2D feature map samples are used to constitute the training set, and 10800*4 2D feature map samples are used to constitute the test set, that is, the training set and the test set are combined in a ratio of 7:3.

[0117] (S500) Build VGGR16_Channel Attention deep learning network and train it

[0118] S501) Embed a Channel Attention module in the VGG16 deep learning network and coordinate multi-dimensional features to obtain a VGGR16_Channel Attention deep learning network.

[0119] like Figure 4 As shown, the VGGR16_Channel Attention structure and its parameter description:

[0120] 1) Input layer:

[0121] Input size: 224×224×4 (instantaneous amplitude reshaping map, instantaneous phase reshaping map, instantaneous frequency reshaping map, time-frequency map);

[0122] Data type: floating point number;

[0123] Normalization: Normalize the amplitude value of the input feature to the range of [-1,1].

[0124] 2) Convolutional layer block 1:

[0125] Convolutional layer 1-1: 64 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU;

[0126] Convolutional layer 1-2: 64 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU.

[0127] Further explanation: The input layer is adjusted to accept feature maps of 4 channels, that is, the number of input channels of the first convolutional layer that accepts 4-channel input (here refers to convolutional layer 1-1) is adjusted to 4.

[0128] 3) Channel Attention Module 1:

[0129] The compression ratio is 16, that is, 16 1×1 convolution kernels are used to generate channel statistics.

[0130] 4) Pooling layer 1

[0131] 2×2 max pooling with a stride of 2

[0132] 5) Convolutional layer block 2:

[0133] Convolutional layer 2-1: 128 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU;

[0134] Convolutional layer 2-2: 128 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU.

[0135] 6) Channel Attention Module 2:

[0136] The compression ratio is 16.

[0137] 7) Pooling layer 2:

[0138] 2×2 max pooling with a stride of 2

[0139] 8) Convolutional layer block 3:

[0140] Convolution layer 3-1: 256 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU; Convolution layer 3-2: 256 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU; Convolution layer 3-3: 256 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU. 9) Channel attention module 3:

[0141] Compression ratio is 16

[0142] 10) Pooling layer 3:

[0143] 2×2 max pooling with a stride of 2

[0144] 11) Convolutional layer block 4:

[0145] Convolutional layer 4-1: 512 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU; Convolutional layer 4-2: 512 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU; Convolutional layer 4-3: 512 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU. 12) Channel attention module 4:

[0146] The compression ratio is 16.

[0147] 13) Pooling layer 4:

[0148] 2×2 max pooling with a stride of 2.

[0149] 14) Convolutional layer block 5:

[0150] Convolution layer 5-1: 512 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU; Convolution layer 5-2: 512 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU; Convolution layer 5-3: 512 3×3 convolution kernels, stride 1, padding 'SAME', activation function ReLU.

[0151] 15) Channel Attention Module 5:

[0152] The compression ratio is 16.

[0153] 16) Pooling layer 5:

[0154] 2×2 max pooling with a stride of 2.

[0155] 17) Fully connected layer:

[0156] Fully connected layer 1: 4096 nodes, activation function ReLU;

[0157] Fully connected layer 2: 4096 nodes, activation function ReLU;

[0158] Fully connected layer 3 (output layer): 1000 nodes, the output layer contains a Softmax classifier.

[0159] Three fully connected (FC) layers: the first two FCs have 4096 channels each, the third FC performs 1000-way classification and therefore contains 1000 channels, one for each class, and the last layer is a soft-max layer. The configuration of the fully connected layers is similar in all networks, and all hidden layers are equipped with rectified (ReLU) nonlinearities.

[0160] Further explanation: the multi-dimensional feature synergy of the input layer: In the VGGR16_Channel Attention structure, in order to adapt to the 224×224×4 input, the input layer is adjusted to accept the feature map of 4 channels. This design allows the model to process and fuse the information of one-dimensional features (instantaneous amplitude, instantaneous phase, instantaneous frequency) and two-dimensional features (time-frequency diagram), which is expected to improve the performance of the model on various tasks.

[0161] Further explanation, embed the Channel Attention module in the VGG16 deep learning network: embed the Channel Attention module in the VGGR16_Channel Attention deep learning network, and insert the channel attention module after each convolution layer block (that is, before each pooling layer). The channel attention module helps the model focus on more useful feature channels, thereby improving the performance of the model.

[0162] Incorporating the Channel Attention mechanism into the VGG16 network can bring two advantages. First, it can improve the response of key features: by dynamically adjusting the importance of different channel features, the model can pay more attention to the feature channels that have an important impact on the recognition of modulation modes, thereby improving the accuracy and efficiency of recognition. Second, it helps to reduce redundant information: by suppressing unimportant feature channels, the channel attention mechanism can reduce redundant information in the model processing process, thereby speeding up the inference speed and reducing the computational cost.

[0163] The rest of the VGGR16_Channel Attention deep learning network remains consistent with the VGG16 deep learning network and will not be described in detail.

[0164] S502) Training VGGR16_Channel Attention deep learning network

[0165] Use a random sampling algorithm to disrupt the order of sample images in the training set to obtain a randomized training set.

[0166] Set the number of training cycles to 50. After all samples are sent to the neural network, multiple iterations of training are performed to ensure that all data are in the convergence stage in the end.

[0167] Selecting the batch size, which refers to the number of samples selected for a single training iteration, plays a key role in influencing the optimization level and speed of the model. A power-of-two batch size can achieve better performance, so set the batch size to 16.

[0168] Set the learning rate Learning Rate, which is used to control the amplitude of parameter updates in gradient descent. Set LearningRate to 0.001.

[0169] The loss function is changed to a weighted loss function based on mean squared error (MSE) total .

[0170] The training set and validation set of the shuffled drone sample images (instantaneous amplitude reshaping images, instantaneous phase reshaping images, instantaneous frequency reshaping images, and time-frequency images) are input into the VGGR16_Channel Attention deep learning network in batches of 70*4 and 30*4 images respectively, and the parameters are continuously updated iteratively until the weighted loss function MSE is total Until convergence, the trained VGGR16_Channel Attention deep learning network is obtained.

[0171] Further explanation, the weighted loss function based on mean square error MSSE total Design: A weighted loss function is used for different inputs (instantaneous amplitude reshaping map, instantaneous phase reshaping map, instantaneous frequency reshaping map and time-frequency map) to adjust the importance of different features during training. The following is the calculation of the weighted loss function based on the mean square error:

[0172] Assume that w1, w2, w3, and w4 represent the weights of the input feature instantaneous amplitude reshaping map, instantaneous phase reshaping map, instantaneous frequency reshaping map, and time-frequency map, respectively. The weighted loss function based on the mean square error is calculated for each feature separately, and then the weighted sum is calculated.

[0173] For a certain sample, the model's predicted values ​​for the four features are The true values ​​are y1, y2, y3, and y4. Then, the loss function based on the mean square error is calculated as follows:

[0174] MSE total =w1×MSE1+w2×MSE2+w3×MSE3+w4×MSE4

[0175] Among them, the mean square error loss MSE of each feature i as follows:

[0176]

[0177] Here n is the number of elements in the sample, and ik are the predicted value and true value of the kth element of the ith feature, respectively.

[0178] Here, w1=0.2, w2=0.2, w3=0.2, w4=0.4.

[0179] Note: The predicted value is the output result of the VGGR16_Channel Attention deep learning network after processing the original numerical values ​​of the input features (such as a1, a2, ...) through a series of convolutional layers, pooling layers, and fully connected layers; the true value is the target value used to train the network, that is, the result that the network is expected to predict, which here represents the actual category of the sample.

[0180] (S600) Identify the modulation mode of the image transmission signal of the drone to be identified

[0181] The sample images of the drone signal to be identified (instantaneous amplitude reshaping image, instantaneous phase reshaping image, instantaneous frequency reshaping image and time-frequency image) are input into the trained VGGR16_Channel Attention deep learning network in batches of 30*4 images, and the recognition result S700 of the drone image transmission signal is output.

[0182] The recognition effect of the present invention is further described below in conjunction with the experimental platform:

[0183] The experimental platform uses Windows 11 system, and the environment configuration is Python 3.8.19 and PyTorch 2.3.0. The server configuration is 13th Gen Intel(R) Core(TM) i9-13900HX, 16G memory. The graphics card is NVIDIA GeForce RTX 4060Laptop GPU. All models are run on GPU, and all parameters except the model are the same.

[0184] The accuracy of the modulation mode recognition of the UAV image transmission signal by the method of the present invention on the test set of instantaneous amplitude reshaping diagram, instantaneous phase reshaping diagram, instantaneous frequency reshaping diagram and time-frequency diagram of the UAV signal is estimated, and the recognition accuracy of the model for various modulation modes of the UAV image transmission signal is obtained by using the confusion matrix. Figure 5 As shown in the figure, the vertical axis of the confusion matrix is ​​the true label of the model for the drone remote control signal, the horizontal axis is the predicted label of the model test set, the diagonal line represents the correct prediction accuracy of different labels, and the numbers in the grid indicate the prediction accuracy.

[0185] from Figure 5 It can be seen that the confusion matrix of VGGR16_Channel Attention recognition performance shows that the recognition effect of the modulation mode of the image transmission signal with the subcarrier modulation mode of 8PSK is relatively poor, and there will be a small number of image transmission signal modulation modes with subcarriers of 2PSK and 16QAM that will misidentify each other, but the recognition accuracy of all image transmission signal modulation modes is also above 93%.

[0186] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration, characterized in that: The method includes: (S100) UAV communication signal data collection and preprocessing The analog / digital device inside the receiver performs analog-to-digital conversion on the received analog signal, and converts the received signal into the in-phase component through orthogonal demodulation and digital down-conversion. i (t) and the orthogonal component s q (t) Two digital baseband signals, then s i (t), s q (t) The two signals are combined to obtain the time domain signal s(t) = s i (t)+j·s q (t), j represents the imaginary unit; (S200) extracting and reshaping one-dimensional features into a two-dimensional feature map, and performing normalization processing; estimating a two-dimensional time-frequency map of the image transmission signal; wherein the one-dimensional features include: instantaneous amplitude, instantaneous phase and instantaneous frequency; The estimated two-dimensional time-frequency diagram of the image transmission signal includes: S221) Perform time-frequency synchronization optimization transformation on the combined time domain signal to obtain T tfso Time-frequency distribution: The combined time domain signal is analyzed with time as the horizontal axis and frequency as the vertical axis. The collected data is processed by dynamic harmonic analysis and transformation, and each output can obtain T tfso Time spectrum, and then calculate the T of each set of data along the time axis tfso The maximum value of the time spectrum is obtained by splicing the maximum values ​​of each group of data to obtain the T of all sample data. tfso Time-frequency spectrum maximum diagram; The expression of the time-frequency synchronization optimization transformation is: In formula (1), tfr1(t, f) = FFT(s(t)·h(t)), tfr2(i, f) ± FFT(s(t)·h′(t)); s(t) is the time domain signal; h(t) is the window function; h′(t) is the derivative of the window function h(t); FFT is Fourier transform, j is the imaginary unit; the adjustment factor N is the number of points in FFT, h len is the window length; Im represents the imaginary part; S222) Extraction of UAV image transmission signal based on robust principal component analysis RPCA: According to the difference in the correlation between the remote control signal and the image transmission signal in the time-frequency domain, the remote control signal and the image transmission signal are respectively allocated to the sparse matrix and the low-rank matrix using the low-rank matrix recovery method; RPCA simultaneously recovers the low-rank matrix A and the sparse matrix E by minimizing the L2 norm of the reconstruction error to separate the UAV image transmission signal information; the minimized reconstruction error is the time-frequency spectrum matrix T tfso The difference between (t, f) and the sum of the low-rank matrix A and the sparse matrix E; (S300) Feature merging: merging the instantaneous amplitude reshaping map, the instantaneous phase reshaping map, the instantaneous frequency reshaping map and the time-frequency map obtained in step (S200) as four channels into a multi-channel feature map; (S400) Generate a training set and a test set: Each drone signal sample corresponds to four features: an instantaneous amplitude reshaping map, an instantaneous phase reshaping map, an instantaneous frequency reshaping map, and a time-frequency map, which are used to generate a training set and a test set; (S500) constructing a VGGR16_Channel Attention deep learning network and training it: embedding a Channel Attention module in the VGG16 deep learning network, inserting the Channel Attention module after each convolutional layer block and before the pooling layer, and coordinating the multi-dimensional features of the input layer, adjusting the number of input channels of the first convolutional layer that accepts 4-channel input to 4 so that it can accept feature maps of 4 channels, obtaining a VGGR16_Channel Attention deep learning network, and training it; (S600) Identify the modulation mode of the drone image transmission signal to be identified: input the sample image of the drone signal to be identified into the trained VGGR16_Channel Attention deep learning network, and output the recognition result of the drone image transmission signal.

2. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 1 is characterized in that: In step (S200), the one-dimensional features are extracted and reshaped into a two-dimensional feature map, and normalized, including: S211) Feature extraction: extracting the instantaneous amplitude, instantaneous phase and instantaneous frequency of the UAV image transmission signal from the collected time domain signal s(t); For the time domain signal s(t) = s i (t)+j·s q (t), The instantaneous amplitude is: The instantaneous phase is: The instantaneous frequency is: In the formula, arctan( ) means to find the inverse tangent, represents the derivative; S212) Feature reshaping: reshaping each one-dimensional feature to obtain a two-dimensional feature map: an instantaneous amplitude reshaping map, an instantaneous phase reshaping map, and an instantaneous frequency reshaping map; S213) Feature normalization: Each two-dimensional feature map is normalized to ensure that all feature maps are in the same numerical range, and the values ​​in the two-dimensional array are normalized to the range of [-1, 1].

3. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 1 is characterized in that: In step S221), h(t) is taken as 4. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 1 is characterized in that: In step S222), the RPCA problem is transformed into a convex optimization problem, the goal is to minimize the sum of the nuclear norm of the low-rank matrix A and the L1 norm of the sparse matrix E, while ensuring that D = A + E. The optimization problem is expressed as: in, It means finding the minimum value for the low-rank matrix A and the sparse matrix E, and st represents the constraint condition; || || * represents the nuclear norm of the matrix, || ||1 represents the L1 norm of the matrix, and λ is the balance parameter between low-rank terms and sparse terms; Solving equation (2), we can obtain the low-rank matrix A, which is also the separated drone image transmission signal information.

5. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 4 is characterized in that: The λ=0.

1.

6. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 1 is characterized in that: In step (S400), a training set and a test set are combined in a ratio of 7:

3.

7. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 1 is characterized in that: In step (S500), the structure of the VGGR16_Channel Attention deep learning network includes: an input layer, a convolutional layer block one, a channel attention module one, a pooling layer one, a convolutional layer block two, a channel attention module two, a pooling layer two, a convolutional layer block three, a channel attention module three, a pooling layer three, a convolutional layer block four, a channel attention module four, a pooling layer four, a convolutional layer block five, a channel attention module five, a pooling layer five, a fully connected layer and a Softmax classifier, which are connected in sequence.

8. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 1 is characterized in that: In step (S500), the training of the VGGR16_Channel Attention deep learning network includes: Use a random sampling algorithm to disrupt the order of sample images in the training set to obtain a randomized training set; Set the number of training cycles Epoch, after sending all samples into the neural network, after multiple iterations of training, ensure that all data are in the convergence stage at the end; Select the batch size, set the learning rate, and change the loss function to the weighted loss function based on the mean square error (MSE). total ; The shuffled drone sample image training set and validation set are input into the VGGR16_Channel Attention deep learning network, and the parameters are continuously updated iteratively until the weighted loss function converges to obtain the trained VGGR16_ChannelAttention deep learning network.

9. The method for identifying modulation modes of UAV image transmission signals based on multi-dimensional feature collaboration according to claim 8 is characterized in that: Set w1, w2, w3, and w4 to represent the weights of the input feature instantaneous amplitude reshaping map, instantaneous phase reshaping map, instantaneous frequency reshaping map, and time-frequency map, respectively. The weighted loss function based on the mean square error is calculated for each feature separately, and then the weighted sum is calculated: For a certain sample, the model's predicted values ​​for the four features are The true values ​​are y1, y2, y3, and y4 respectively. Then, the loss function based on the mean square error is calculated as: MSE total =w1×MSE1+w2×MSE2+w3×MSE3+w4×MSE4 Among them, the mean square error loss MSEi of each feature is as follows: Where n is the number of elements in the sample, and ik are the predicted value and true value of the kth element of the ith feature, respectively.

10. A UAV image transmission signal modulation mode recognition system based on multi-dimensional feature collaboration, characterized in that: The system uses the method for identifying the modulation mode of the UAV image transmission signal based on multi-dimensional feature collaboration as described in any one of claims 1 to 9 to identify the modulation mode of the UAV image transmission signal.

Images