A method and apparatus for identifying a UAV, and an electronic device

Abstract

The application discloses a kind of unmanned aerial vehicle identification methods, comprising: receiving radio signal by receiver;Radio signal is carried out short-time Fourier transform and obtains time-frequency diagram;The time-frequency diagram is preprocessed;Classification is carried out to the time-frequency diagram using neural network model for classifying time-frequency diagram, obtain the binary unique identifier of characteristic classification result;Extract identification information from binary unique identifier as unmanned aerial vehicle identification result;Neural network model is obtained based on multiple time-frequency diagram samples and labeled information training;Labeled information is the binary unique identifier obtained by encoding multiple label information corresponding to time-frequency diagram sample;Label information at least includes the unmanned aerial vehicle model information and unmanned aerial vehicle distance information corresponding to radio signal sample.The application adopts classification method to identify multiple unmanned aerial vehicle information simultaneously, provides more useful information for implementing unmanned aerial vehicle management;And the algorithm volume of the neural network model used is small, and execution efficiency is high.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned aerial vehicles, and particularly relates to an unmanned aerial vehicle identification method and device and electronic equipment. BACKGROUND

[0002] With the development of unmanned aerial vehicle technology, civilian unmanned aerial vehicles gradually enter people's lives and bring more and more convenience to people's lives. However, due to improper operation methods or lack of safety awareness of some operators, safety hazards and illegal and irregular events occur frequently. Therefore, it is necessary to strengthen the management of unmanned aerial vehicles. When implementing the management of unmanned aerial vehicles, the direction of the unmanned aerial vehicle needs to be detected and the model of the unmanned aerial vehicle needs to be identified.

[0003] In the prior art, the model of an unmanned aerial vehicle can be identified by photographing the unmanned aerial vehicle in flight and using a target detection method based on computer vision. However, the target detection method needs the unmanned aerial vehicle to appear in the photographed scene, and once the unmanned aerial vehicle is blocked by trees or buildings, the method cannot be used. In addition, since the unmanned aerial vehicle target is generally small, it usually only occupies a few pixels in an open scene, and is easily mixed with birds, power lines or balloons and other objects, which is prone to missed detection or false detection, and is not conducive to application in actual scenes.

[0004] In view of the above problems, a scheme for identifying the model of an unmanned aerial vehicle based on deep learning method from the radio signal of the unmanned aerial vehicle has been proposed, but this kind of method can only identify the model information of the unmanned aerial vehicle, and the identification information is not enough for implementing the management of unmanned aerial vehicles. SUMMARY

[0005] In order to solve the above problems existing in the prior art, the present application provides an unmanned aerial vehicle identification method, device and electronic equipment.

[0006] The technical problem to be solved by the embodiments of the present application is solved by the following technical scheme:

[0007] In a first aspect, the present application provides an unmanned aerial vehicle identification method, comprising:

[0008] receiving a radio signal in space by a receiver;

[0009] performing short-time Fourier transform on the radio signal to obtain a time-frequency graph;

[0010] preprocessing the time-frequency graph; the preprocessing includes denoising processing and interference removing processing;

[0011] using a pre-trained neural network model for classifying time-frequency graphs to classify the preprocessed time-frequency graph, to obtain a binary unique identifier representing the classification result;

[0012] extracting identification information from the binary unique identifier to obtain a UAV identification result;

[0013] The neural network model is obtained by training based on a plurality of time-frequency pattern samples and labeled information of the time-frequency pattern samples. The labeled information of the time-frequency pattern sample is a binary unique identifier obtained by encoding a plurality of label information corresponding to the time-frequency pattern sample.

[0014] The plurality of label information at least includes UAV model information and UAV distance information corresponding to the radio signal sample. The radio signal sample is a radio signal corresponding to the time-frequency pattern sample.

[0015] Optionally, the manner of denoising the time-frequency pattern includes:

[0016] For the first column data of the time-frequency pattern, the mean and standard deviation of the column data are calculated, and a first denoising threshold is calculated using the calculated mean and standard deviation; The number of time domain points of the time-frequency pattern; N The first denoising threshold is calculated using the calculated

[0017] The first denoising threshold is calculated using the calculated N The denoising of the time-frequency pattern is simulated using the calculated

[0018] For the first column data of the simulated denoising time-frequency pattern, the mean and standard deviation of the column data are calculated, and a second denoising threshold is calculated using the calculated mean and standard deviation; The second denoising threshold is calculated using the calculated

[0019] The second denoising threshold is calculated using the calculated N The second denoising threshold is calculated using the calculated

[0020] The maximum value of each column data of the time-frequency pattern is extracted to form a maximum value vector with a length of N

[0021] The maximum value vector is subjected to multi-point sliding median filtering to obtain a filtered vector with a length of N The maximum value vector is subjected to multi-point sliding median filtering to obtain a filtered vector with a length of

[0022] The fourth denoising threshold is calculated using the calculated N The fourth denoising threshold is calculated using the calculated

[0023] The smaller one of the third denoising threshold and the fourth denoising threshold is used to denoise the time-frequency pattern.

[0024] Optionally, the manner of denoising the time-frequency pattern includes:

[0025] ​After the time-frequency diagram is denoised, a matrix with a one-dimensional size of J × N is initialized.

[0026] For the first segment data in the first column data of the current time-frequency diagram, the number of non-0 elements in the segment data is counted, and when the counted number is less than 0.3* , the element in the first j row and the first i column of the matrix is set to 0, otherwise the element is set to 1.

[0027] When all elements in the matrix are reset, the matrix is taken as a binary time-frequency diagram after removing the stray signal.

[0028] wherein, , S is the sampling frequency of the receiver, ， represents the unit bandwidth of the frequency band that the receiver can accept.

[0029] Optionally, the manner of denoising the time-frequency diagram further includes:

[0030] For the first column data of the binary time-frequency diagram, a plurality of elements corresponding to the WiFi signal in the column data are set to 0 to remove the WiFi signal in the binary time-frequency diagram.

[0031] Optionally, the preprocessing further includes:

[0032] After removing the WiFi signal in the binary time-frequency diagram, a non-all-zero column in the current binary time-frequency diagram is extracted.

[0033] A new time-frequency diagram with a size of J × J is constructed based on each of the extracted non-all-zero columns, as the time-frequency diagram after the preprocessing.

[0034] Optionally, the plurality of label information specifically includes: device type information, device distance information, and device model information corresponding to the radio signal sample.

[0035] wherein, the device type information is information whether the wireless device sending the radio signal sample is a drone; the device distance information is distance information between the wireless device and the receiver; and the device model information is model information of the wireless device; when the wireless device is a drone, the device distance information is the drone distance information, and the device model information is the drone model information.

[0036] Optionally, the multiple label information further comprises: device state information corresponding to the radio signal sample; the device state information is state information of the wireless device.

[0037] Optionally, the neural network model comprises: a convolution module, a residual module, a first LSTM module, a second LSTM module, and a full connection module.

[0038] The input end of the convolution module receives the preprocessed time-frequency graph; the output end of the convolution module is connected to the input end of the residual module through a 1 / 2 pooling layer; the output end of the residual module is connected to the input end of the first LSTM module through a 1 / 2 pooling layer; the output end of the first LSTM module is connected to the input end of the second LSTM module; and the output end of the second LSTM module is connected to the input end of the full connection module.

[0039] The residual module comprises multiple residual blocks with increasing processing channel numbers, and adjacent residual blocks are connected to each other through 1 / 2 pooling layers.

[0040] The full connection module comprises multiple full connection layers connected in series, wherein the last full connection layer outputs a binary unique identifier representing a classification result.

[0041] In a second aspect, the present application provides a UAV identification device, comprising:

[0042] A receiving module is configured to receive radio signals in space through a receiver.

[0043] A transformation module is configured to perform short-time Fourier transform on the radio signals to obtain time-frequency graphs.

[0044] A preprocessing module is configured to preprocess the time-frequency graphs; the preprocessing comprises denoising and interference removal.

[0045] A classification module is configured to classify the preprocessed time-frequency graphs using a pre-trained neural network model for classifying time-frequency graphs to obtain a binary unique identifier representing a classification result.

[0046] An extraction module is configured to extract identification information from the binary unique identifier to obtain a UAV identification result.

[0047] The neural network model is trained based on multiple time-frequency graph samples and labeled information of the time-frequency graph samples; the labeled information of the time-frequency graph samples is a binary unique identifier obtained by encoding multiple label information corresponding to the time-frequency graph samples.

[0048] The multiple label information at least includes: unmanned aerial vehicle model information corresponding to the radio signal sample and unmanned aerial vehicle distance information; and the radio signal sample is a radio signal corresponding to a time-frequency diagram sample.

[0049] In a third aspect, an electronic device is provided, including: a processor and a memory; the processor and the memory complete mutual communication through a communication bus;

[0050] The memory is used for storing a computer program.

[0051] The processor is used for executing the computer program stored on the memory, and realizing any one of the unmanned aerial vehicle identification methods.

[0052] In the unmanned aerial vehicle identification method provided by the embodiments of the present application, a neural network model for classifying time-frequency diagrams is obtained based on multiple time-frequency diagram samples and label information of the time-frequency diagram samples; wherein the label information of the time-frequency diagram sample is a binary unique identifier obtained by encoding multiple label information corresponding to the time-frequency diagram sample; the label information at least includes: unmanned aerial vehicle model information corresponding to a radio signal sample and unmanned aerial vehicle distance information; thus, using the neural network model to classify the time-frequency diagram converted from the radio signal in space can obtain the binary unique identifier representing the classification result, and then the corresponding unmanned aerial vehicle model information and unmanned aerial vehicle distance information can be extracted from the binary unique identifier. Compared with the prior art, the embodiments of the present application can provide more useful information for implementing unmanned aerial vehicle management.

[0053] Moreover, the embodiments of the present application encode the multiple label information corresponding to the time-frequency diagram sample into a binary unique identifier, ensuring the global uniqueness of each radio signal sample, so that multiple unmanned aerial vehicle information can be identified simultaneously by using a classification method. Compared with the neural network model of target detection, the algorithm size of the neural network model used in the present application is small, and the execution efficiency is greatly improved.

[0054] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings and further detailed description of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a flowchart of a unmanned aerial vehicle identification method provided by the embodiments of the present application;

[0056] Figure 2 is a process schematic diagram of generating a time-frequency diagram in the embodiments of the present application;

[0057] Figure 3 is a process schematic diagram of removing a stray signal in a time-frequency diagram in the embodiments of the present application;

[0058] Figure 4is a process schematic diagram of removing WiFi signals in a time-frequency graph in an embodiment of the present application;

[0059] Figure 5 is a structural schematic diagram of a neural network model used in an embodiment of the present application;

[0060] Figure 6 is a structural schematic diagram of a convolution module and a staggered block in an embodiment of the present application;

[0061] Figure 7 is a structural schematic diagram of three different neural network models used in an embodiment of the present application;

[0062] Figure 8 is a label information grading and subordination relationship of a data set in an embodiment of the present application;

[0063] Figure 9 is a process schematic diagram of an embodiment of the present application;

[0064] Figure 10 is a structural schematic diagram of an electronic device provided by an embodiment of the present application; DETAILED DESCRIPTION

[0065] The present application will be further described in detail below in conjunction with specific embodiments, but the embodiments of the present application are not limited thereto.

[0066] The embodiments of the present application provide a UAV identification method, which can provide more useful information for implementing UAV management. Figure 1 As shown in the figure, the method comprises the following steps:

[0067] S10: receiving a radio signal in space through a receiver.

[0068] Exemplarily, the radio signal in space can be received by a receiver with a radio frequency signal acquisition module. Here, the receiver can be a receiver based on FPGA control processing and containing a radio frequency signal acquisition module. The radio frequency signal acquisition module can be AD9371, but is not limited thereto, and any radio frequency receiving module with wideband gain can be applied to the embodiments of the present application.

[0069] S20: performing short-time Fourier transform on the radio signal to obtain a time-frequency graph.

[0070] Specifically, referring to Figure 2 As shown in the figure, the time-domain sampling data of the receiver is read in a sliding reading manner, and the signal sampling frame number set when reading is m , the sampling frequency point number of each frame is S , and the sampling time point number is N ; the short-time Fourier transform is performed on each frame.N The Fast Fourier Transform of a point yields a point with dimension 1. S × N The time-frequency diagram.

[0071] In practical applications, PyCharm software can be used to perform short-time Fourier transform on radio signals to obtain the corresponding time-frequency diagram of the radio signals.

[0072] S30: Preprocess the time-frequency graph; this preprocessing includes noise reduction and interference removal.

[0073] Here, denoising the time-frequency graph can combat random noise; interference removal can combat interfering signals. This improves the robustness of the UAV recognition method, thereby enhancing UAV recognition performance in complex environments and addressing the problem of significant electromagnetic interference affecting UAV recognition.

[0074] For example, in one implementation, the method for denoising the time-frequency graph may include:

[0075] (1) Regarding the first time-frequency diagram Calculate the mean of the data in the column. and standard deviation The first denoising threshold is calculated using the calculated mean and standard deviation. ; N This represents the number of time-domain points in the time-frequency plot.

[0076] (2) Using the calculation results N The first noise reduction threshold Perform the first denoising process on the time-frequency graph;

[0077] (3) After the first denoising process, for the current time-frequency graph, the first... Calculate the mean of the data in the column. and standard deviation The second denoising threshold is calculated using the calculated mean and standard deviation. ;

[0078] (4) Using the calculation results N The second noise reduction threshold Perform a second denoising process on the current time-frequency graph.

[0079] Alternatively, in another implementation, the method for denoising the time-frequency graph may include:

[0080] (1) Regarding the first time-frequency diagram Calculate the mean of the data in the column. and standard deviation and the calculated mean and standard deviation to calculate a first denoising threshold ;

[0081] (2) simulate denoising of the time-frequency map using the calculated N first denoising thresholds to obtain a simulated denoised time-frequency map;

[0082] (3) for the first column of data of the simulated denoised time-frequency map, calculate the mean and standard deviation of the column of data, and use the calculated mean and standard deviation to calculate a second denoising threshold ;

[0083] (4) perform linear fitting according to the calculated N second denoising thresholds , and take the midpoint of the fitting as a third denoising threshold ;

[0084] (5) extract the maximum value of each column of data of the time-frequency map to form a maximum value vector with a length of N ;

[0085] (6) perform multi-point (e.g. 7-point) sliding median filtering on the maximum value vector to obtain a filtered vector with a length of N ;

[0086] (7) perform linear fitting according to the N elements of the filtered vector, and take the midpoint of the fitting as a fourth denoising threshold ;

[0087] (8) take the smaller of the third denoising threshold and the fourth denoising threshold to perform denoising processing on the time-frequency map.

[0088] wherein, is not easily affected by wideband interference signals, but may not be close enough to the noise floor; is relatively close to the noise floor, but is easily affected by long-time constant frequency signals; therefore, taking the smaller of the two to perform denoising processing on the time-frequency map can simultaneously ensure detection accuracy and stability; that is, the final detection threshold selected here can minimize the false alarm rate within an acceptable false alarm probability range, and is as close to the noise floor as possible, while most of the noise is below the signal detection threshold, thereby having both high stability and accuracy.

[0089] In practical applications, when the threshold is used to denoise the time-frequency graph, specifically, the points higher than the threshold in the time-frequency graph are set to 1, and the points lower than the threshold are set to 0, so as to eliminate random noise.

[0090] After the denoising of the time-frequency graph is completed, the time-frequency graph is further subjected to interference removal processing. Exemplarily, in an implementation mode, the manner of removing interference from the time-frequency graph can include:

[0091] (1) After denoising the time-frequency graph, a matrix with one dimension of J × N is initialized;

[0092] (2) For the first segment data in the first column data of the current time-frequency graph, the number of non-0 elements in the segment data is counted, and when the counted number is less than 0.3 , the element in the first j row and the first i column of the matrix is set to 0, otherwise the element is set to 1. Here, since the bandwidth of the stray signal is extremely narrow, and the bandwidth of the unmanned aerial vehicle remote control signal is greater than 0.3 , therefore, through this step operation, the stray signal can be removed, Figure 3 the implementation principle of this step operation is shown in FIG.

[0093] (3) When all elements in the matrix are reset, the matrix is taken as the binary time-frequency graph after removing the stray signal.

[0094] wherein, , S is the sampling frequency point number of the receiver, ， represents the unit bandwidth of the frequency band that the receiver can accept.

[0095] At this time, the binary time-frequency graph can be taken as the time-frequency graph after the preprocessing in step S30.

[0096] Alternatively, in another implementation mode, if there is also a WiFi signal in the space, after obtaining the above-mentioned binary time-frequency graph, the elements corresponding to the WiFi signal in the first column data of the binary time-frequency graph can be set to 0 to remove the WiFi signal in the binary time-frequency graph.

[0097] Specifically, the center frequency of WiFi signals is similar to that of most drone remote control signals, making them easily detected by the receiver and appearing in the time-frequency graph, interfering with feature extraction. Therefore, it is necessary to remove WiFi signals from the time-frequency graph to preserve the characteristics of the drone remote control signal. For the actual process of removing WiFi signals, see [link to relevant documentation]. Figure 4 As shown, we can first initialize a zero-matrix Ma* with the same dimensions as the binarized time-frequency map Ma. Then, we iterate through the binarized time-frequency map column by column. Whenever a frequency snapshot value of 1 is detected in a column, we count the number of "1"s z in the B frequency points following that frequency point. If the count z < B - 1, the signal at that frequency point is considered a narrowband signal, and the element corresponding to that frequency point in Ma* is set to 1. If z ≥ B - 1, a WiFi signal is considered to be detected at that frequency point, and the element corresponding to that frequency point in Ma* is set to 0. Then, we continue to traverse forward in single-frequency-point increments, checking B frequency points each time, until z < B - 1 is found, at which point the WiFi signal is considered lost. Preferably, B = 5, but it is not limited to this setting.

[0098] Figure 4 The diagram illustrates the principle behind eliminating WiFi signals. To clearly demonstrate the processing steps, Figure 4 The binarized time-frequency plot was displayed with its rows and columns transformed. From Figure 4 As can be seen, in the t-th column of the binarized time-frequency plot, there are 4 "1"s within B=5 frequency points starting from frequency point f. Therefore, the element at the f-th frequency point in the t-th column of Ma* is set to 0. In the t-th column, there are 3 "1"s within 5 frequency points starting from frequency point f+1. Therefore, the element at the f+1-th frequency point in the t-th column of Ma* is set to 1. Similarly, the element at the f+2-th frequency point in the t-th column of Ma* is set to 1. Starting from frequency point f+k, the number of "1"s within every 5 frequency points is less than 4. Therefore, the elements starting from the f+k-th frequency point in the t-th column of Ma* are all set to 1.

[0099] It is understandable that the binarized time-frequency graph after removing the WiFi signal can be used as the preprocessed time-frequency graph.

[0100] Alternatively, in another implementation, after removing the WiFi signal from the binarized time-frequency graph, the non-zero columns in the current binarized time-frequency graph can be further extracted; then, a dimension is constructed based on each extracted non-zero column. J × J The new time-frequency plot is used as the preprocessed time-frequency plot.

[0101] Specifically, if the extracted non-zero columns are insufficient JIf the extracted non-all-zero columns are less than J columns, the time-frequency graph of J may be obtained by inserting all-zero columns J If the extracted non-all-zero columns exceed J columns, the time-frequency graph of J may be obtained by extracting the first J columns J .

[0102] In addition, if the dimension size of the time-frequency graph does not match the input of the neural network model, the time-frequency graph can also be preprocessed in terms of scale correction.

[0103] S40: Classify the preprocessed time-frequency graph using a pre-trained neural network model for classifying time-frequency graphs to obtain a binary unique identifier representing the classification result.

[0104] The neural network model classifies time-frequency graphs according to energy band and texture feature differences. The neural network model is trained based on multiple time-frequency graph samples and labeled information of the time-frequency graph samples. The labeled information of the time-frequency graph samples is a binary unique identifier obtained by encoding multiple label information corresponding to the time-frequency graph samples. It can be understood that different types of label information occupy different bits in the binary unique identifier. The multiple label information at least includes: UAV model information and UAV distance information corresponding to the radio signal sample; the radio signal sample is the radio signal corresponding to the time-frequency graph sample. The time-frequency graph sample can also be processed by the preprocessing method shown in step S30, so that the performance of the neural network model in the training stage and the use stage is consistent.

[0105] It can be understood that using a pre-trained neural network model to classify time-frequency graphs can solve the problem of difficulty in manually extracting signal features.

[0106] Specifically, referring to Figure 5 , the neural network model includes: a convolution module, a residual module, a first LSTM (Long short-term memory) module, a second LSTM module, and a full connection module.

[0107] The input end of the convolution module receives the preprocessed time-frequency graph. The output end of the convolution module is connected to the input end of the residual module through a 1 / 2 pooling layer. The output end of the residual module is connected to the input end of the first LSTM module through a 1 / 2 pooling layer. The output end of the first LSTM module is connected to the input end of the second LSTM module. The output end of the second LSTM module is connected to the input end of the full connection module.

[0108] The residual module includes multiple residual blocks with the number of processing channels increasing sequentially, and adjacent residual blocks are connected to each other through a 1 / 2 pooling layer;

[0109] The fully connected module consists of multiple serially connected fully connected (FC) layers, with the last fully connected layer outputting a binary unique identifier representing the classification result.

[0110] In this embodiment of the invention, to enable the neural network model to have sufficient classification ability, multiple residual blocks are used to form a deeper residual module, thereby solving the gradient vanishing problem of the network without sacrificing its feature extraction ability. See also... Figure 6 As shown, the residual block has a multi-branch structure during the training phase. Specifically, referring to the internal structure of the convolutional module ConvBlock, parallel 1×1 convolutional branches and identity mapping branches are added to each convolutional layer to form a residual block Resblock. In the actual use phase after training, the added parallel branches are disconnected to restore the residual block to a single-branch structure. ReLU is the activation function.

[0111] Among them, the output result of a single residual block during the training phase y Represented as:

[0112] ;

[0113] in, and Corresponding to Figure 6 The model uses 1×1 and 3×3 convolutions. During training, a convolutional neural network architecture is built by simply stacking these modules; in practical use, the residual blocks can be easily transformed into equivalent models. In the form of, and The parameters can be obtained from the trained model through linear combination.

[0114] For example, in one implementation, when the multiple tag information corresponding to the time map sample only includes the UAV model information and UAV distance information corresponding to the radio signal sample, the specific network structure of the neural network model can be found in [reference needed]. Figure 7 The structure (a) is shown in the diagram. At this point, Figure 5 The neural network model shown can simultaneously output two types of identification information: drone model information and drone distance information.

[0115] In another implementation, the various tag information corresponding to the time-frequency map sample can specifically include: device type information, device distance information, and device model information corresponding to the radio signal sample;

[0116] The device type information is information about whether the wireless device sending the radio signal sample is a UAV; the device distance information is distance information between the wireless device and the receiver; the device model information is model information of the wireless device; when the wireless device is a UAV, the device distance information is UAV distance information, and the device model information is UAV model information. In addition, when the wireless device is not a UAV, the device model information can be uniformly set as information different from all UAV model information, and of course, in the case of sufficient sample preparation, the model information of the wireless device can also be directly used.

[0117] Correspondingly, Figure 7 The structure (b) in the structure (b) is a network structure of a neural network model adapted to the device type information, the device distance information, and the device model information. That is, the structure (b) can simultaneously output the three types of identification information of the device type information, the device distance information, and the device model information.

[0118] In another implementation, based on the foregoing implementation, the multiple types of label information corresponding to the time-frequency pattern sample can further include device state information corresponding to the radio signal sample; wherein the device state information is state information of the wireless device, such as a current device in a standby state or a running state.

[0119] Correspondingly, Figure 7 The structure (c) in the structure (c) is a network structure of a neural network model adapted to the device type information, the device distance information, the device model information, and the device state information. That is, the structure (c) can simultaneously output the four types of identification information of the device type information, the device distance information, the device model information, and the device state information.

[0120] Before actually starting to train the neural network model, radio signals of multiple models of UAVs can be collected under different weather conditions, different transmission and reception distances, and different flight states, so as to construct multiple time-frequency pattern samples and label their corresponding label information, form a data set, and train the neural network model by using the data set. Figure 8 The label information classification and dependency relationship of the data set are shown in the table in the structure (c); wherein Category 1 indicates that the wireless device is a UAV, and Category 0 indicates that the wireless device is not a UAV; wherein Category 1 can include multiple UAV models UAV 1-UAV N, and for the case of Category 0, the binary bits of the UAV field are different from the binary bits of UAV 1-UAV N. A single model of UAV can include four working states Mode 1-Mode 4; each working state can include multiple different distances Distance.

[0121] S50: Extract the identification information from the binary unique identifier as the unmanned aerial vehicle identification result.

[0122] Specifically, according to the coding rule of the binary unique identifier, the real classification information corresponding to the various label information is extracted from the binary unique identifier, so as to obtain the identification information.

[0123] In an implementation manner, the trained neural network model can be deployed on a deep learning hardware platform NVIDIJetson tx2, and connected with the unmanned aerial vehicle signal acquisition device, so that the integration operation of signal acquisition, time-frequency graph generation and unmanned aerial vehicle identification can be realized.

[0124] Referring to Figure 9 The neural network model for classifying the time-frequency graph is obtained based on the plurality of time-frequency graph samples and the labeled information of the time-frequency graph samples; wherein the labeled information of the time-frequency graph sample is a binary unique identifier obtained by encoding the plurality of label information corresponding to the time-frequency graph sample; the label information at least includes the unmanned aerial vehicle model information and the unmanned aerial vehicle distance information corresponding to the radio signal sample; thus, the binary unique identifier representing the classification result can be obtained by using the neural network model to classify the time-frequency graph converted from the radio signal in space, and the corresponding unmanned aerial vehicle model information and unmanned aerial vehicle distance information can be extracted from the binary unique identifier. Compared with the prior art, the embodiment of the present application can provide more useful information for implementing unmanned aerial vehicle management.

[0125] Moreover, the embodiment of the present application encodes the plurality of label information corresponding to the time-frequency graph sample into a binary unique identifier, which guarantees the global uniqueness of each radio signal sample, so that the classification method can be used to identify multiple unmanned aerial vehicle information at the same time, and compared with the neural network model of target detection type, the algorithm volume of the neural network model used in the present application is small, and the execution efficiency is greatly improved.

[0126] The embodiment of the present application can accurately and quickly identify the unmanned aerial vehicle according to the radio signal of the unmanned aerial vehicle in the environment of certain communication signal interference, low signal-to-noise ratio and certain time-frequency aliasing, so as to achieve the purpose of unmanned aerial vehicle flight monitoring and countermeasures.

[0127] The unmanned aerial vehicle identification method provided by the embodiment of the present application can be applied to an electronic device. Specifically, the electronic device can be a computer, a terminal device, a server or a specific unmanned aerial vehicle remote control device, an unmanned aerial vehicle countermeasure device, etc., which is not limited here.

[0128] Corresponding to the unmanned aerial vehicle identification method described above, the embodiment of the present application provides an unmanned aerial vehicle identification device, which comprises:

[0129] The receiving module is used to receive radio signals in space via a receiver;

[0130] The transformation module is used to perform a short-time Fourier transform on the radio signal to obtain a time-frequency diagram;

[0131] The preprocessing module is used to preprocess the time-frequency graph; the preprocessing includes noise reduction and interference removal.

[0132] The classification module is used to classify the preprocessed time-frequency graph using a pre-trained neural network model for classifying time-frequency graphs, and obtain a unique binary identifier representing the classification result.

[0133] The extraction module extracts identification information from the binary unique identifier to obtain the UAV identification result;

[0134] The neural network model is trained based on multiple time-frequency map samples and their annotation information; the annotation information of the time-frequency map samples is a binary unique identifier obtained by encoding multiple label information corresponding to the time-frequency map samples.

[0135] The various tag information includes at least: the drone model information and drone distance information corresponding to the radio signal sample; the radio signal sample is the radio signal corresponding to the time-frequency map sample.

[0136] Optionally, the method for denoising the time-frequency graph includes:

[0137] Regarding the first time-frequency diagram Given a column of data, calculate the mean and standard deviation of that column of data, and then use the calculated mean and standard deviation to calculate the first denoising threshold; N This represents the number of time-domain points in the time-frequency plot.

[0138] Calculated N A first denoising threshold is used to simulate the denoising of the time-frequency graph, resulting in a simulated denoised time-frequency graph;

[0139] For the first time-frequency diagram of the simulated denoising Given a column of data, calculate the mean and standard deviation of that column of data, and then use the calculated mean and standard deviation to calculate the second denoising threshold;

[0140] Based on the calculation N A second denoising threshold is linearly fitted, and the midpoint of the fit is taken as the third denoising threshold based on the fitting result;

[0141] Extract the maximum value of each column of data from the time-frequency graph, and construct a structure of length [length missing]. N The maximum value vector;

[0142] Perform multi-point sliding median filtering on the maximum value vector to obtain a length of N The filtered vector;

[0143] Based on the filtered vector N Linear fitting is performed on each element, and the midpoint of the fitting is taken as the fourth noise reduction threshold based on the fitting result.

[0144] The time-frequency graph is denoised using the smaller of the third and fourth denoising thresholds.

[0145] Optionally, the methods for performing interference removal processing on the time-frequency graph include:

[0146] After denoising the time-frequency graph, one dimension is initialized as follows: J × N Matrix;

[0147] For the current time-frequency graph, the first The first column of data Given a data segment, count the number of non-zero elements in that segment. If the count is less than 0.3, the count is considered zero. When, the first of the matrix is... j line, number i Set the element of the column to 0, otherwise set the element to 1;

[0148] When all elements in the matrix have been reset, the matrix is ​​used as a binarized time-frequency graph after removing spurious signals.

[0149] in, , S This refers to the number of sampling frequency points of the receiver. ， This indicates the unit bandwidth of the frequency band that the receiver can accept.

[0150] Optionally, the method for performing interference removal processing on the time-frequency graph further includes:

[0151] For the first binarized time-frequency graph The column of data is set to 0 for several elements corresponding to the WiFi signal, in order to remove the WiFi signal from the binarized time-frequency graph.

[0152] Optionally, the preprocessing further includes:

[0153] After removing the WiFi signal from the binarized time-frequency graph, extract the non-all-zero columns from the current binarized time-frequency graph;

[0154] The dimension is constructed based on each of the extracted non-zero columns. J ×J a new time-frequency graph as the pre-processed time-frequency graph.

[0155] Optionally, the plurality of label information specifically includes: device type information, device distance information and device model information corresponding to the radio signal sample.

[0156] The device type information is information about whether the wireless device sending the radio signal sample is a UAV; the device distance information is distance information between the wireless device and the receiver; and the device model information is model information of the wireless device; when the wireless device is a UAV, the device distance information is UAV distance information, and the device model information is UAV model information.

[0157] Optionally, the plurality of label information further includes: device state information corresponding to the radio signal sample; and the device state information is state information of the wireless device.

[0158] Optionally, the neural network model includes: a convolution module, a residual module, a first LSTM module, a second LSTM module and a full connection module.

[0159] The input end of the convolution module receives the pre-processed time-frequency graph; the output end of the convolution module is connected to the input end of the residual module through a 1 / 2 pooling layer; the output end of the residual module is connected to the input end of the first LSTM module through a 1 / 2 pooling layer; the output end of the first LSTM module is connected to the input end of the second LSTM module; and the output end of the second LSTM module is connected to the input end of the full connection module.

[0160] The residual module includes a plurality of residual blocks with increasing processing channel numbers, and adjacent residual blocks are connected to each other through a 1 / 2 pooling layer.

[0161] The full connection module includes a plurality of serially connected full connection layers, and the last full connection layer outputs a binary unique identifier representing a classification result.

[0162] Based on the same inventive concept, the embodiments of the present application also provide an electronic device, as shown in the accompanying drawings, which comprises a processor 601 and a memory 603. Figure 10 The processor 601 and the memory 603 complete mutual communication through a communication bus,

[0163] The memory 603 is used to store a computer program.

[0164] The processor 601 is used to execute the computer program stored in the memory 603, and implement the method steps of any one of the UAV identification methods.

[0165] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used to represent the communication bus in the figure, but it does not represent that there is only one bus or only one type of bus.

[0166] The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the processor.

[0167] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0168] The application further provides a computer readable storage medium. The computer readable storage medium stores a computer program. When the computer program is executed by a processor, the method steps of any one of the unmanned aerial vehicle identification methods described above are implemented.

[0169] Optionally, the computer readable storage medium can be a Non-Volatile Memory (NVM), such as at least one disk memory.

[0170] Optionally, the computer readable storage medium can be at least one storage device located away from the processor.

[0171] In another embodiment of the application, a computer program product containing instructions is also provided, which, when running on a computer, causes the computer to execute the method steps of any one of the unmanned aerial vehicle identification methods described above.

[0172] It should be noted that the described embodiments of the device / electronic device / storage medium / computer program product are merely exemplary in nature and that various changes can be made to the method and natures of the device / electronic device / storage medium / computer program product by one skilled in the art that would be or become apparent to the skilled person and are intended to be within the scope of the application.

[0173] It should be noted that the terms "first", "second" are used only for the purpose of description and should not be construed as indicating or implying relative importance or a specific number of technical features indicated. Thus, the features defined with "first", "second" can include one or more features explicitly or implicitly. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0174] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present description, the illustrative description of the above terms is not necessarily directed to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the present description.

[0175] Although the present application is described herein in conjunction with various embodiments, other variations of the described embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from an inspection of the drawings and the disclosure. In the description, the word "comprise" does not exclude other components or steps, and "a" or "one" does not exclude a plurality. Some measures are described in mutually different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0176] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, apparatuses (devices), or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects, all of which are referred to herein as "modules" or "systems". Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The computer program is stored / distributed in a suitable medium, together with other hardware, or as part of hardware, and can also take other distribution forms, such as through the Internet or other wired or wireless telecommunications systems.

[0177] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0178] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0179] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0180] The above description is further to the present application in conjunction with specific preferred embodiments, and cannot be deemed to limit the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, several simple derivations or replacements can be made without departing from the concept of the present application, and all of them should be regarded as falling within the protection scope of the present application.

Claims

1. A method for identifying unmanned aerial vehicles (UAVs), characterized in that, include: Receive radio signals in space using a receiver; Perform a short-time Fourier transform on the radio signal to obtain a time-frequency diagram; The time-frequency graph is preprocessed; the preprocessing includes noise reduction and interference removal. The pre-trained neural network model for classifying time-frequency graphs is used to classify the preprocessed time-frequency graphs, and a unique binary identifier representing the classification result is obtained. The identification information is extracted from the binary unique identifier to obtain the UAV identification result; The neural network model is trained based on multiple time-frequency map samples and their annotation information; the annotation information of the time-frequency map samples is a binary unique identifier obtained by encoding multiple label information corresponding to the time-frequency map samples. The various tag information includes at least: the drone model information and drone distance information corresponding to the radio signal sample; the radio signal sample is the radio signal corresponding to the time-frequency map sample; The methods for denoising the time-frequency graph include: For the first time-frequency diagram Given a column of data, calculate the mean and standard deviation of that column of data, and then use the calculated mean and standard deviation to calculate the first denoising threshold; N This represents the number of time-domain points in the time-frequency plot. Calculated N A first denoising threshold is used to simulate the denoising of the time-frequency graph, resulting in a simulated denoised time-frequency graph; For the first time-frequency diagram of the simulated denoising Given a column of data, calculate the mean and standard deviation of that column of data, and then use the calculated mean and standard deviation to calculate the second denoising threshold; Based on the calculation N A second denoising threshold is linearly fitted, and the midpoint of the fit is taken as the third denoising threshold based on the fitting result; Extract the maximum value of each column of data from the time-frequency graph, and construct a structure of length [length missing]. N The maximum value vector; Perform multi-point sliding median filtering on the maximum value vector to obtain a length of N The filtered vector; Based on the filtered vector N Linear fitting is performed on each element, and the midpoint of the fitting is taken as the fourth noise reduction threshold based on the fitting result. The time-frequency graph is denoised using the smaller of the third and fourth denoising thresholds.

2. The UAV identification method according to claim 1, characterized in that, The methods for removing interference from the time-frequency graph include: After denoising the time-frequency graph, one dimension is initialized as follows: J × N Matrix; Regarding the current time-frequency graph, the first The first column of data Given a data segment, count the number of non-zero elements in that segment. If the count is less than 0.3, the count is considered zero. When, the first of the matrix is... j line, number i Set the element of the column to 0, otherwise set the element to 1; When all elements in the matrix have been reset, the matrix is ​​used as a binarized time-frequency graph after removing spurious signals. in, , S This refers to the number of sampling frequency points of the receiver. ， This indicates the unit bandwidth of the frequency band that the receiver can accept.

3. The UAV identification method according to claim 2, characterized in that, The methods for de-interference processing of the time-frequency graph also include: For the first binarized time-frequency graph The column of data is set to 0 for several elements corresponding to the WiFi signal, in order to remove the WiFi signal from the binarized time-frequency graph.

4. The UAV identification method according to claim 3, characterized in that, The preprocessing also includes: After removing the WiFi signal from the binarized time-frequency graph, extract the non-all-zero columns from the current binarized time-frequency graph; The dimension is constructed based on each of the extracted non-zero columns. J × J The new time-frequency diagram is used as the preprocessed time-frequency diagram.

5. The UAV identification method according to claim 1, characterized in that, The various tag information specifically includes: device type information, device distance information, and device model information corresponding to the radio signal sample; The device type information refers to whether the wireless device emitting the radio signal sample is a drone; the device distance information refers to the distance between the wireless device and the receiver; the device model information refers to the model information of the wireless device; when the wireless device is a drone, the device distance information refers to the distance information of the drone, and the device model information refers to the model information of the drone.

6. The UAV identification method according to claim 5, characterized in that, The various tag information also includes: device status information corresponding to the radio signal sample; the device status information is the status information of the wireless device.

7. The UAV identification method according to claim 1, characterized in that, The neural network model includes: a convolutional module, a residual module, a first LSTM module, a second LSTM module, and a fully connected module; The input of the convolutional module receives the preprocessed time-frequency graph; the output of the convolutional module is connected to the input of the residual module through a 1 / 2 pooling layer; the output of the residual module is connected to the input of the first LSTM module through a 1 / 2 pooling layer; the output of the first LSTM module is connected to the input of the second LSTM module; and the output of the second LSTM module is connected to the input of the fully connected module. The residual module includes multiple residual blocks with the number of processing channels increasing sequentially, and adjacent residual blocks are connected to each other through a 1 / 2 pooling layer; The fully connected module includes multiple serially connected fully connected layers, wherein the last fully connected layer outputs a binary unique identifier representing the classification result.

8. A drone identification device, characterized in that, include: The receiving module is used to receive radio signals in space via a receiver; The transformation module is used to perform a short-time Fourier transform on the radio signal to obtain a time-frequency diagram; The preprocessing module is used to preprocess the time-frequency graph; The preprocessing includes denoising and interference removal; wherein, the denoising process for the time-frequency graph includes: For the first time-frequency diagram Given a column of data, calculate the mean and standard deviation of that column of data, and then use the calculated mean and standard deviation to calculate the first denoising threshold; N This represents the number of time-domain points in the time-frequency plot. Calculated N A first denoising threshold is used to simulate the denoising of the time-frequency graph, resulting in a simulated denoised time-frequency graph; For the first time-frequency diagram of the simulated denoising Given a column of data, calculate the mean and standard deviation of that column of data, and then use the calculated mean and standard deviation to calculate the second denoising threshold; Based on the calculation N A second denoising threshold is linearly fitted, and the midpoint of the fit is taken as the third denoising threshold based on the fitting result; Extract the maximum value of each column of data from the time-frequency graph, and construct a structure of length [length missing]. N The maximum value vector; Perform multi-point sliding median filtering on the maximum value vector to obtain a length of N The filtered vector; Based on the filtered vector N Linear fitting is performed on each element, and the midpoint of the fitting is taken as the fourth noise reduction threshold based on the fitting result. The time-frequency graph is denoised using the smaller of the third and fourth denoising thresholds. The classification module is used to classify the preprocessed time-frequency graph using a pre-trained neural network model for classifying time-frequency graphs, and obtain a unique binary identifier representing the classification result. The extraction module extracts identification information from the binary unique identifier to obtain the UAV identification result; The neural network model is trained based on multiple time-frequency map samples and their annotation information; the annotation information of the time-frequency map samples is a binary unique identifier obtained by encoding multiple label information corresponding to the time-frequency map samples. The various tag information includes at least: the drone model information and drone distance information corresponding to the radio signal sample; the radio signal sample is the radio signal corresponding to the time-frequency map sample.

9. An electronic device, characterized in that, include: Processor and memory; The processor and memory communicate with each other via a communication bus; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the UAV identification method according to any one of claims 1 to 7.

Images