Multi-station cooperative unmanned aerial vehicle detection method

Abstract

The invention provides a multi-site cooperative unmanned aerial vehicle detection method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting J to-be-identified radio frequency signals, the to-be-identified radio frequency signals comprising frequency spectrum information and phase information related to an object existing in an environment; performing time-frequency analysis on the J to-be-identified radio frequency signals to obtain a corresponding frequency spectrum waterfall matrix; performing mathematical transformation on the J frequency spectrum waterfall matrixes to obtain a corresponding amplitude frequency spectrum waterfall plot and a corresponding phase frequency spectrum waterfall plot; based on the J amplitude spectrum waterfall plots and the J phase spectrum waterfall plots, amplitude spectrum input features and phase spectrum input features are determined respectively; inputting the amplitude spectrum input features and the phase spectrum input features into a depth space-time frequency attention detection model obtained by pre-training, and outputting an unmanned aerial vehicle judgment signal; the unmanned aerial vehicle judgment signal comprises unmanned aerial vehicle existence judgment information and unmanned aerial vehicle type judgment information, so that the unmanned aerial vehicle can be accurately detected in a complex environment.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle positioning technology, and in particular to a multi-site coordinated unmanned aerial vehicle detection method, device, electronic equipment and storage medium. Background Art

[0002] With the development of artificial intelligence technology, drones at this stage are showing a trend of intelligentization, clustering, practicality and modularization. In order to achieve the control of drones, a large number of drone detection and countermeasure technologies have been proposed, including radar, radio spectrum, optoelectronic and acoustic detection technologies, etc., so as to carry out targeted countermeasures against drones.

[0003] However, in complex urban environments, these technologies face a series of challenges, including complex and changeable electromagnetic environments, severe drone signal fading, and multiple drone signal types during drone detection. A single sensing station is unlikely to effectively address these challenges. Therefore, using multiple sensing stations to collaboratively receive data to significantly improve the detection and identification of drone signals has become an effective solution. Current methods for joint detection of multiple sensing stations include maximum ratio merging and selective merging, but in the case of unknown dynamic signal waveforms, drones are still difficult to detect accurately.

[0004] Therefore, how to accurately detect drones in complex environments has become a technical problem that the industry urgently needs to solve. Summary of the invention

[0005] The present invention provides a multi-site coordinated unmanned aerial vehicle detection method, device, electronic equipment and storage medium to solve the problem of how to accurately detect unmanned aerial vehicles in a complex environment.

[0006] According to a first aspect of the present invention, a multi-site collaborative drone detection method is provided, wherein J sensing sites in the environment are used to detect drones in the environment, wherein the sensing sites are used to receive radio frequency signals to be identified related to objects in the environment, and the method comprises:

[0007] Collect J radio frequency signals to be identified, where the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment, where J is an integer greater than or equal to 1;

[0008] Performing time-frequency analysis on the J radio frequency signals to be identified to obtain corresponding spectrum waterfall matrices;

[0009] Performing mathematical transformation on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams;

[0010] Based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, the amplitude spectrum input features and the phase spectrum input features are determined respectively; wherein the amplitude spectrum input features include the sorting information, time information and frequency point information corresponding to the J amplitude spectrum waterfall diagrams, and the phase spectrum input features include the sorting information, time information and frequency point information corresponding to the J phase spectrum waterfall diagrams;

[0011] The amplitude spectrum input features and the phase spectrum input features are input into a pre-trained deep spatiotemporal frequency attention detection model to output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information.

[0012] Optionally, the deep spatiotemporal-frequency attention detection model includes a spatial attention module, a time-frequency module, a first convolutional neural network, a second convolutional neural network, and a fully connected neural network;

[0013] The step of inputting the amplitude spectrum input feature and the phase spectrum input feature into a pre-trained deep spatiotemporal frequency attention detection model and outputting a drone judgment signal comprises the following steps:

[0014] Based on the spatial attention network parameters in the spatial attention module, weighted processing is performed on the amplitude spectrum input feature to obtain the spatial attention distribution of the amplitude spectrum input feature;

[0015] Based on the spatial attention distribution and the time-frequency attention network parameters in the time-frequency module, weighted processing is performed on the amplitude spectrum input feature to obtain the time-frequency attention distribution of the amplitude spectrum input feature;

[0016] Based on the spatial attention distribution and the time-frequency attention distribution, the amplitude spectrum input feature and the phase spectrum input feature are subjected to feature refinement processing to obtain amplitude spectrum input refined features and phase spectrum input refined features;

[0017] Inputting the refined features of the amplitude spectrum input into a first convolutional neural network for feature learning processing, and inputting the refined features of the phase spectrum input into a second convolutional neural network for feature learning processing, to obtain a first learning feature and a second learning feature;

[0018] The first learning feature and the second learning feature are input into the fully connected neural network, and the fully connected neural network outputs the drone judgment signal.

[0019] Optionally, obtain a pre-trained deep spatiotemporal attention detection model, including the following steps:

[0020] Acquire a training set, wherein the training set comprises acquiring a plurality of radio frequency signal samples;

[0021] Preprocessing the multiple radio frequency signal samples to obtain corresponding radio frequency signal sample data, each radio frequency signal sample data including its corresponding amplitude spectrum input feature and phase spectrum input feature;

[0022] Calibrate the RF signal sample data in the training set based on the real UAV existence judgment information corresponding to the RF signal sample and the real UAV type judgment information corresponding to the calibrated RF signal sample;

[0023] The amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal sample data are used as the input of the deep spatiotemporal frequency attention detection model, and the real drone existence judgment information corresponding to the RF signal sample and the real drone type judgment information corresponding to the calibrated RF signal sample are used as the output of the deep spatiotemporal frequency attention detection model, and a mapping relationship is constructed between the amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal sample data, the real drone existence judgment information, and the real drone type judgment information corresponding to the calibrated RF signal sample;

[0024] The network weight parameters of the deep spatiotemporal-frequency attention detection model are iteratively updated based on the optimization algorithm to obtain the final deep spatiotemporal-frequency attention detection model, wherein the network weight parameters include the spatial attention network parameters, the temporal-frequency attention network parameters, the network weight parameters of the first convolutional neural network, the network weight parameters of the second convolutional neural network and the network weight parameters of the fully connected neural network.

[0025] Optionally, after obtaining the pre-trained deep spatiotemporal frequency attention detection model, the deep spatiotemporal frequency attention detection model is further verified, and the verification process includes the following steps:

[0026] Acquire a test set, wherein the test set includes acquiring a plurality of radio frequency signal test samples;

[0027] Preprocessing the multiple radio frequency signal test samples to obtain corresponding radio frequency signal test sample data, each radio frequency signal test sample data including its corresponding amplitude spectrum input feature and phase spectrum input feature;

[0028] Calibrate the RF signal test sample data in the test set based on the real UAV existence judgment information corresponding to the RF signal test sample and the real UAV type judgment information corresponding to the calibration RF signal sample;

[0029] The amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal test sample data are used as the input of the final deep spatiotemporal attention detection model, and the corresponding drone judgment signal is output;

[0030] Counting the number of drone judgment signals that are the same as the corresponding real drone existence judgment information and the real drone type judgment information corresponding to the calibration radio frequency signal sample, to obtain the accuracy of the final deep spatiotemporal attention detection model;

[0031] Based on the accuracy of the final deep spatiotemporal attention detection model, the qualified result of the final deep spatiotemporal attention detection model is determined, where:

[0032] If the accuracy of the final deep space-time-frequency attention detection model is greater than the accuracy threshold, the final deep space-time-frequency attention detection model is qualified.

[0033] Optionally, performing time-frequency analysis on the J radio frequency signals to be identified to obtain a corresponding spectrum waterfall matrix; specifically including:

[0034] Performing short-time Fourier transform on each radio frequency signal to be identified to obtain a corresponding short-time Fourier transform equation;

[0035] Perform time-frequency analysis on the short-time Fourier transform equation to obtain a corresponding spectrum waterfall matrix.

[0036] Optionally, the short-time Fourier transform equation for performing time-frequency analysis on the radio frequency signal x(t) to be identified is:

[0037]

[0038] Among them, STFT(t, f) is the frequency spectrum of the RF signal to be identified at a given time t, f is the frequency, h(τ-t) is the analysis window function, and x(τ) is the local signal of the RF signal to be identified x(t) intercepted corresponding to the analysis window function.

[0039] Optionally, a time-frequency analysis is performed on the short-time Fourier transform equation corresponding to the m-th sensing site to obtain a corresponding spectrum waterfall matrix, specifically including:

[0040] Perform time-frequency analysis on the short-time Fourier transform equation corresponding to the mth sensing station and construct the corresponding first matrix:

[0041]

[0042] Among them, STFT m To perform the short-time Fourier transform equation for the mth sensing station;

[0043] Simplifying equation (2), we get the corresponding spectrum waterfall matrix:

[0044]

[0045] Wherein, m, n, k are all integers, and 0≤m≤J-1;

[0046] n is the time window sequence number, and 0≤n≤N-1, N is the total number of time windows;

[0047] k is the frequency point number, and 0≤k≤K-1, K is the total number of frequency points;

[0048] bw is the analysis bandwidth.

[0049] Optionally, a mathematical transformation is performed on the spectrum waterfall matrix corresponding to the mth sensing site to obtain a corresponding amplitude spectrum waterfall diagram and a phase spectrum waterfall diagram, specifically including:

[0050] Perform the first mathematical transformation on the spectrum waterfall matrix corresponding to the mth sensing site to obtain the corresponding amplitude spectrum waterfall diagram:

[0051]

[0052] Perform a second mathematical transformation on the spectrum waterfall matrix corresponding to the mth sensing site to obtain the corresponding amplitude spectrum waterfall diagram:

[0053]

[0054] Optionally, the drone judgment signal is a one-dimensional vector signal.

[0055] According to a second aspect of the present invention, a drone detection system is provided for detecting drones in an environment using J sensing stations in the environment, wherein the sensing stations are used to receive radio frequency signals to be identified related to objects in the environment, and the system comprises:

[0056] A sensing station is used to collect J radio frequency signals to be identified, where the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment, where J is an integer greater than or equal to 1;

[0057] A first data preprocessing module, configured to perform time-frequency analysis on the J radio frequency signals to be identified to obtain a corresponding spectrum waterfall matrix;

[0058] A second data preprocessing module is used to perform mathematical transformation on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams;

[0059] An input feature preprocessing module, used to determine the amplitude spectrum input feature and the phase spectrum input feature based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, respectively; wherein the amplitude spectrum input feature includes the sorting information, time information and frequency point information corresponding to the J amplitude spectrum waterfall diagrams, and the phase spectrum input feature includes the sorting information, time information and frequency point information corresponding to the J phase spectrum waterfall diagrams;

[0060] The drone judgment module is used to input the amplitude spectrum input features and the phase spectrum input features into a pre-trained deep spatiotemporal frequency attention detection model, and output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information.

[0061] According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect of the present invention when executing the program.

[0062] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the methods described in the first aspect of the present invention.

[0063] In the multi-site collaborative drone detection method, device, electronic device and storage medium provided by the present invention, the method uses J sensing sites in the environment to detect drones in the environment, and the sensing sites are used to receive radio frequency signals to be identified related to objects in the environment, and the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment by collecting J radio frequency signals to be identified; time-frequency analysis is performed on the J radio frequency signals to be identified to obtain corresponding spectrum waterfall matrices; mathematical transformation is performed on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams; based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, amplitude spectrum input features and phase spectrum input features are respectively determined; the amplitude spectrum input features and the phase spectrum input features are input into a pre-trained deep spatiotemporal attention detection model to output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information, so that drones can be accurately detected in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 It is a flowchart of a multi-site collaborative UAV detection method in a first embodiment of the present invention;

[0066] Figure 2 is a schematic diagram of a detection environment in an embodiment of the present invention;

[0067] Figure 3 Schematic diagram of the construction of a deep spatiotemporal attention detection model in an embodiment of the present invention;

[0068] Figure 4 is a flow chart of a multi-site collaborative UAV detection method in a second embodiment of the present invention;

[0069] Figure 5 is a flow chart of a multi-site collaborative UAV detection method in a third embodiment of the present invention;

[0070] Figure 6 is a flow chart of a multi-site collaborative UAV detection method in a fourth embodiment of the present invention;

[0071] Figure 7 is a flow chart of a multi-site collaborative UAV detection method in a fifth embodiment of the present invention;

[0072] Figure 8 is a block diagram of determining input features in an embodiment of the present invention;

[0073] Fig. 9 is a schematic diagram of the structure of a drone detection system in an embodiment of the present invention;

[0074] Fig.10 is a schematic diagram of the structure of an exemplary electronic device in one embodiment of the present invention;

[0075] Description of reference numerals:

[0076] 81-perception site;

[0077] 82-First data preprocessing module

[0078] 83-Second data preprocessing module

[0079] 84-Input feature preprocessing module

[0080] 85-UAV judgment module

[0081] 91-processor;

[0082] 92- memory;

[0083] 93-bus. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0086] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0087] In view of the problem that it is difficult to accurately detect drones in complex environments in the prior art. The present invention provides a multi-site collaborative drone detection method, device, electronic device and storage medium, which uses J sensing sites in the environment to detect drones in the environment, and the sensing sites are used to receive radio frequency signals to be identified related to objects in the environment, and collect J radio frequency signals to be identified, and the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment; perform time-frequency analysis on the J radio frequency signals to be identified to obtain corresponding spectrum waterfall matrices; perform mathematical transformation on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams; based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, respectively determine the amplitude spectrum input features and the phase spectrum input features; input the amplitude spectrum input features and the phase spectrum input features into a pre-trained deep spatiotemporal attention detection model, and output a drone judgment signal; the drone judgment signal includes the drone existence judgment information and the drone type judgment information, so that the drone can be accurately detected in a complex environment.

[0088] Please refer to Figure 1 The embodiment of the present invention provides a multi-site coordinated drone detection method, which uses J sensing sites in the environment to detect drones in the environment, and the sensing sites are used to receive radio frequency signals to be identified related to objects in the environment. The method includes:

[0089] S1: Collect J radio frequency signals to be identified, where the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment, where J is an integer greater than or equal to 1;

[0090] S2: Perform time-frequency analysis on the J radio frequency signals to be identified to obtain corresponding spectrum waterfall matrices;

[0091] S3: performing mathematical transformation on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams;

[0092] S4: Based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, respectively determine the amplitude spectrum input feature and the phase spectrum input feature;

[0093] The amplitude spectrum input feature includes the sorting information, time information and frequency information corresponding to the J amplitude spectrum waterfall diagrams, and the phase spectrum input feature includes the sorting information, time information and frequency information corresponding to the J phase spectrum waterfall diagrams;

[0094] S5: Input the amplitude spectrum input features and the phase spectrum input features into the pre-trained deep spatiotemporal-frequency attention detection model, and output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information.

[0095] In one embodiment, the drone judgment signal is a one-dimensional vector signal. As an example, if the drone judgment signal corresponds to the third type of drone, the drone judgment signal can be set to (1, 0, 0, 1, 0, 0...0). In this label, the first element of the vector is a label of whether it is a drone signal. If it is, it is marked as 1, otherwise it is 0, and the subsequent vector elements (0, 0, 1, 0, 0...0) represent drones belonging to the third type. Of course, the present invention does not limit the specific type of drone judgment signal, and the staff in this field can select a suitable signal format as needed.

[0096] The present invention uses J sensing stations in the environment to detect drones in the environment because there are geographical differences between the sensing stations, so the information perceived by different sensing stations will be significantly different. Based on this significantly different information, drones in the environment can be detected. In an example, this method uses Figure 2 The J sensing stations in the environment shown in the figure detect the drones in the environment. Please refer to Figure 2 ,exist Figure 2 The environment shown also contains interference sources ( Figure 2 The communication base station in the middle is adjacent to each sensing site, which will make the electromagnetic environment around the sensing site more complicated, thereby affecting the sensing link and communication link of the sensing site.

[0097] Based on this, the method provided by the embodiment of the present invention is further described:

[0098] Regarding the deep spatiotemporal attention detection model, in one implementation, please refer to Figure 3 , which shows a flow chart of the deep spatiotemporal-frequency attention detection model, which includes a spatial attention module 10, a time-frequency module 20, a first convolutional neural network 30, a second convolutional neural network 40 and a fully connected neural network 50.

[0099] In this case, please refer to Figure 4 , and combined with Figure 3 In step S5, the amplitude spectrum input feature and the phase spectrum input feature are input into the pre-trained deep spatiotemporal attention detection model to output the drone judgment signal, including the following steps:

[0100] S51: Based on the spatial attention network parameters in the spatial attention module, weighted processing is performed on the amplitude spectrum input feature to obtain the spatial attention distribution of the amplitude spectrum input feature;

[0101] Specifically, based on the network parameters of the spatial attention network 101 in the spatial attention module, the amplitude spectrum input feature is weighted, and the spatial attention distribution of the amplitude spectrum input feature is obtained in the spatial attention unit 102.

[0102] α S =W S AWF

[0103] Among them, W S is the spatial attention network parameter, AWF is the amplitude spectrum input feature; wherein, the spatial attention distribution of can be understood as the spatial attention distribution of different perception sites;

[0104] S52: Based on the spatial attention distribution and the time-frequency attention network parameters in the time-frequency module, weighted processing is performed on the amplitude spectrum input feature to obtain the time-frequency attention distribution of the amplitude spectrum input feature;

[0105] Specifically, based on the spatial attention distribution and the network parameters of the time-frequency attention network 201 in the time-frequency module, the amplitude spectrum input feature is weighted to obtain the time-frequency attention distribution of the amplitude spectrum input feature.

[0106] α TF =W TF α S AWF

[0107] Among them, W TF are the parameters of the time-frequency attention network;

[0108] S53: Based on the spatial attention distribution and the time-frequency attention distribution, perform feature refinement processing on the amplitude spectrum input feature and the phase spectrum input feature to obtain amplitude spectrum input refined features and phase spectrum input refined features;

[0109] Specifically, based on the spatial attention distribution and the time-frequency attention distribution, the amplitude spectrum input feature and the phase spectrum input feature are subjected to feature refinement processing to obtain the amplitude spectrum input refined feature

[0110] AWF′=α TF α S AWF

[0111] Phase spectrum input refinement feature

[0112] PWF′=α TF α S ·PWF

[0113] S54: inputting the refined features of the amplitude spectrum input into the first convolutional neural network for feature learning processing, and inputting the refined features of the phase spectrum input into the second convolutional neural network for feature learning processing, to obtain a first learning feature and a second learning feature;

[0114] S55: Input the first learning feature and the second learning feature into the fully connected neural network, and the fully connected neural network outputs the drone judgment signal.

[0115] Among them, regarding the implementation of the first convolutional neural network and the second convolutional neural network, in one implementation, the first convolutional neural network and the second convolutional neural network will perform multi-layer convolution, pooling and activation operations on the input image or other data, so as to extract feature maps of different levels and scales, and the convolutional neural network will expand the feature map of the last layer into a one-dimensional vector (the same as the aforementioned one-dimensional vector signal), and use the one-dimensional vector as the input of the fully connected neural network. The fully connected neural network performs tasks such as classification or regression on the input one-dimensional vector, thereby realizing the recognition of drone signals.

[0116] When obtaining the pre-trained deep spatiotemporal attention detection model, in one implementation, please refer to Figure 5 , including the following steps:

[0117] S61: Acquire a training set, where the training set includes acquiring a plurality of radio frequency signal samples;

[0118] S62: preprocessing the multiple RF signal samples to obtain corresponding RF signal sample data, each RF signal sample data including its corresponding amplitude spectrum input feature and phase spectrum input feature;

[0119] S63: Calibrate the RF signal sample data in the training set based on the real UAV existence judgment information corresponding to the RF signal sample and the real UAV type judgment information corresponding to the calibrated RF signal sample;

[0120] S64: Using the amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal sample data as the input of the deep spatiotemporal frequency attention detection model, using the real drone existence judgment information corresponding to the RF signal sample and the real drone type judgment information corresponding to the calibrated RF signal sample as the output of the deep spatiotemporal frequency attention detection model, and constructing a mapping relationship between the amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal sample data, the real drone existence judgment information, and the real drone type judgment information corresponding to the calibrated RF signal sample;

[0121] S65: Iteratively update the network weight parameters of the deep spatiotemporal-frequency attention detection model based on the optimization algorithm to obtain the final deep spatiotemporal-frequency attention detection model, wherein the network weight parameters include the spatial attention network parameters, the temporal-frequency attention network parameters, the network weight parameters of the first convolutional neural network, the network weight parameters of the second convolutional neural network and the network weight parameters of the fully connected neural network.

[0122] The optimization algorithm may be a gradient algorithm, or other algorithms, such as the Adam optimization algorithm, etc., and the present invention is not limited thereto.

[0123] After obtaining the pre-trained deep spatiotemporal attention detection model, in one implementation, please refer to Figure 6 , and also includes verifying the deep spatiotemporal attention detection model, the verification process has the following steps:

[0124] S71: Acquire a test set, where the test set includes acquiring a plurality of radio frequency signal test samples;

[0125] S72: Preprocess the multiple RF signal test samples to obtain corresponding RF signal test sample data, each RF signal test sample data including its corresponding amplitude spectrum input feature and phase spectrum input feature;

[0126] S73: Calibrate the radio frequency signal test sample data in the test set based on the real drone existence judgment information corresponding to the radio frequency signal test sample and the real drone type judgment information corresponding to the calibration radio frequency signal sample;

[0127] S74: using the amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal test sample data as inputs of a final deep spatiotemporal attention detection model, and outputting a corresponding drone judgment signal;

[0128] S75: Counting the number of the drone judgment signals that are the same as the corresponding real drone existence judgment information and the real drone type judgment information corresponding to the calibration radio frequency signal sample, and obtaining the final accuracy of the deep spatiotemporal attention detection model;

[0129] S76: Based on the accuracy of the final deep spatiotemporal frequency attention detection model, determine the qualified result of the final deep spatiotemporal frequency attention detection model, wherein: if the accuracy of the final deep spatiotemporal frequency attention detection model is greater than the accuracy threshold, enter S77; otherwise, enter S78;

[0130] S77: The final deep spatiotemporal attention detection model is qualified;

[0131] S78: The final deep spatiotemporal attention detection model is unqualified.

[0132] Now, how to perform time-frequency analysis on the J radio frequency signals to be identified is described:

[0133] In one implementation, please refer to Figure 7 , step S2 specifically includes:

[0134] S21: Performing short-time Fourier transform on each radio frequency signal to be identified to obtain a corresponding short-time Fourier transform equation;

[0135] S22: Performing time-frequency analysis on the short-time Fourier transform equation to obtain a corresponding spectrum waterfall matrix.

[0136] Among them, the short-time Fourier transform equation for time-frequency analysis of the radio frequency signal x(t) to be identified is:

[0137]

[0138] Among them, STFT(t, f) is the frequency spectrum of the RF signal to be identified at a given time t, f is the frequency, h(τ-t) is the analysis window function, and x(τ) is the local signal of the RF signal to be identified x(t) intercepted corresponding to the analysis window function.

[0139] Please combine Figure 8 , which shows a block diagram of how to perform time-frequency analysis on the J radio frequency signals to be identified to determine the amplitude spectrum input features and the phase spectrum input features. Figure 8 In the example, the radio frequency signal to be identified is an I / Q signal.

[0140] On this basis, the time-frequency analysis of the short-time Fourier transform equation corresponding to the mth sensing station is realized to obtain the corresponding spectrum waterfall matrix, which specifically includes:

[0141] Perform time-frequency analysis on the short-time Fourier transform equation corresponding to the mth sensing station and construct the corresponding first matrix:

[0142]

[0143] Among them, STFT m is the short-time Fourier transform equation for the mth sensing station; Δt is the time interval, Δf is the frequency interval;

[0144] Simplifying equation (2), we get the corresponding spectrum waterfall matrix:

[0145]

[0146] Wherein, m, n, k are all integers, and 0≤m≤J-1;

[0147] n is the time window sequence number, and 0≤n≤N-1, N is the total number of time windows;

[0148] k is the frequency point number, and 0≤k≤K-1, K is the total number of frequency points;

[0149] bw is the analysis bandwidth.

[0150] Continuing to perform mathematical transformation on the spectrum waterfall matrix corresponding to the mth sensing site, the corresponding amplitude spectrum waterfall diagram and phase spectrum waterfall diagram can be obtained. In one implementation, the method specifically includes:

[0151] Perform the first mathematical transformation on the spectrum waterfall matrix corresponding to the mth sensing site to obtain the corresponding amplitude spectrum waterfall diagram:

[0152]

[0153] Perform a second mathematical transformation on the spectrum waterfall matrix corresponding to the mth sensing site to obtain the corresponding amplitude spectrum waterfall diagram:

[0154]

[0155] Please continue to refer to Figure 8 The J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams are all two-dimensional spectrum waterfall diagrams. By combining the J amplitude spectrum waterfall diagrams, the amplitude spectrum input feature can be obtained, which is three-dimensional data related to all times, frequencies and sites; similarly, by combining the J phase spectrum waterfall diagrams, the phase spectrum input feature can be obtained, which is also three-dimensional data related to all times, frequencies and sites.

[0156] In addition, please refer to Fig. 9The present invention also provides a drone detection system for detecting drones in an environment using J sensing stations in the environment, wherein the sensing stations are used to receive radio frequency signals to be identified related to objects in the environment, and the system comprises:

[0157] A sensing site 81 is used to collect J radio frequency signals to be identified, where the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment, where J is an integer greater than or equal to 1;

[0158] A first data preprocessing module 82, configured to perform time-frequency analysis on the J radio frequency signals to be identified to obtain a corresponding spectrum waterfall matrix;

[0159] A second data preprocessing module 83 is used to perform mathematical transformation on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams;

[0160] An input feature preprocessing module 84 is used to determine the amplitude spectrum input feature and the phase spectrum input feature based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, respectively; wherein the amplitude spectrum input feature includes the sorting information, time information and frequency point information corresponding to the J amplitude spectrum waterfall diagrams, and the phase spectrum input feature includes the sorting information, time information and frequency point information corresponding to the J phase spectrum waterfall diagrams;

[0161] The drone judgment module 85 is used to input the amplitude spectrum input features and the phase spectrum input features into a pre-trained deep spatiotemporal frequency attention detection model, and output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information.

[0162] In addition, an embodiment of the present invention further provides an electronic device, please refer to FIG. 11 , the electronic device includes a memory 92, a processor 91, and a program stored in the memory 92 and executable on the processor 91, and the processor 91 can implement the steps of the positioning mark graphic recognition method in the aforementioned solution of the present invention when executing the program. The processor can communicate with the memory 92 via a bus 93.

[0163] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the positioning mark graphic recognition method in the aforementioned solution of the present invention are implemented.

[0164] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0165] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0166] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0167] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0168] To summarize, in the solution provided by the embodiment of the present invention, J sensing sites in the environment are used to detect drones in the environment, and the sensing sites are used to receive radio frequency signals to be identified related to objects in the environment, and the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment by collecting J radio frequency signals to be identified; time-frequency analysis is performed on the J radio frequency signals to be identified to obtain corresponding spectrum waterfall matrices; mathematical transformation is performed on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams; based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, the amplitude spectrum input features and the phase spectrum input features are respectively determined; the amplitude spectrum input features and the phase spectrum input features are input into a pre-trained deep spatiotemporal-frequency attention detection model to output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information, so that drones can be accurately detected in complex environments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-site collaborative drone detection method, which uses J sensing sites in the environment to detect drones in the environment, wherein the sensing sites are used to receive radio frequency signals to be identified related to objects in the environment, and is characterized in that: The method comprises: Collect J radio frequency signals to be identified, where the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment, where J is an integer greater than or equal to 1; Performing time-frequency analysis on the J radio frequency signals to be identified to obtain corresponding spectrum waterfall matrices; Performing mathematical transformation on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams; Based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, the amplitude spectrum input features and the phase spectrum input features are determined respectively; wherein the amplitude spectrum input features include the sorting information, time information and frequency point information corresponding to the J amplitude spectrum waterfall diagrams, and the phase spectrum input features include the sorting information, time information and frequency point information corresponding to the J phase spectrum waterfall diagrams; The amplitude spectrum input features and the phase spectrum input features are input into a pre-trained deep spatiotemporal frequency attention detection model to output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information.

2. The multi-site coordinated drone detection method according to claim 1, characterized in that: The deep spatiotemporal-frequency attention detection model includes a spatial attention module, a time-frequency module, a first convolutional neural network, a second convolutional neural network and a fully connected neural network; The step of inputting the amplitude spectrum input feature and the phase spectrum input feature into a pre-trained deep spatiotemporal frequency attention detection model and outputting a drone judgment signal comprises the following steps: Based on the spatial attention network parameters in the spatial attention module, weighted processing is performed on the amplitude spectrum input feature to obtain the spatial attention distribution of the amplitude spectrum input feature; Based on the spatial attention distribution and the time-frequency attention network parameters in the time-frequency module, weighted processing is performed on the amplitude spectrum input feature to obtain the time-frequency attention distribution of the amplitude spectrum input feature; Based on the spatial attention distribution and the time-frequency attention distribution, the amplitude spectrum input feature and the phase spectrum input feature are subjected to feature refinement processing to obtain amplitude spectrum input refined features and phase spectrum input refined features; Inputting the refined features of the amplitude spectrum input into a first convolutional neural network for feature learning processing, and inputting the refined features of the phase spectrum input into a second convolutional neural network for feature learning processing, to obtain a first learning feature and a second learning feature; The first learning feature and the second learning feature are input into the fully connected neural network, and the fully connected neural network outputs the drone judgment signal.

3. The multi-site coordinated drone detection method according to claim 2, characterized in that: Obtaining a pre-trained deep spatiotemporal attention detection model involves the following steps: Acquire a training set, wherein the training set comprises acquiring a plurality of radio frequency signal samples; Preprocessing the multiple radio frequency signal samples to obtain corresponding radio frequency signal sample data, each radio frequency signal sample data including its corresponding amplitude spectrum input feature and phase spectrum input feature; Calibrate the RF signal sample data in the training set based on the real UAV existence judgment information corresponding to the RF signal sample and the real UAV type judgment information corresponding to the calibrated RF signal sample; The amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal sample data are used as the input of the deep spatiotemporal frequency attention detection model, and the real drone existence judgment information corresponding to the RF signal sample and the real drone type judgment information corresponding to the calibrated RF signal sample are used as the output of the deep spatiotemporal frequency attention detection model, and a mapping relationship is constructed between the amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal sample data, the real drone existence judgment information, and the real drone type judgment information corresponding to the calibrated RF signal sample; The network weight parameters of the deep spatiotemporal-frequency attention detection model are iteratively updated based on the optimization algorithm to obtain the final deep spatiotemporal-frequency attention detection model, wherein the network weight parameters include the spatial attention network parameters, the temporal-frequency attention network parameters, the network weight parameters of the first convolutional neural network, the network weight parameters of the second convolutional neural network and the network weight parameters of the fully connected neural network.

4. The multi-site coordinated drone detection method according to claim 3, characterized in that: After obtaining the pre-trained deep spatiotemporal frequency attention detection model, the deep spatiotemporal frequency attention detection model is further verified. The verification process includes the following steps: Acquire a test set, wherein the test set includes acquiring a plurality of radio frequency signal test samples; Preprocessing the multiple radio frequency signal test samples to obtain corresponding radio frequency signal test sample data, each radio frequency signal test sample data including its corresponding amplitude spectrum input feature and phase spectrum input feature; Calibrate the RF signal test sample data in the test set based on the real UAV existence judgment information corresponding to the RF signal test sample and the real UAV type judgment information corresponding to the calibration RF signal sample; The amplitude spectrum input features and the phase spectrum input features corresponding to the RF signal test sample data are used as the input of the final deep spatiotemporal attention detection model, and the corresponding drone judgment signal is output; Counting the number of drone judgment signals that are the same as the corresponding real drone existence judgment information and the real drone type judgment information corresponding to the calibration radio frequency signal sample, to obtain the accuracy of the final deep spatiotemporal attention detection model; Based on the accuracy of the final deep spatiotemporal attention detection model, the qualified result of the final deep spatiotemporal attention detection model is determined, where: If the accuracy of the final deep space-time-frequency attention detection model is greater than the accuracy threshold, the final deep space-time-frequency attention detection model is qualified.

5. The multi-site coordinated drone detection method according to claim 1, characterized in that: Performing time-frequency analysis on the J radio frequency signals to be identified to obtain a corresponding spectrum waterfall matrix; specifically including: Performing short-time Fourier transform on each radio frequency signal to be identified to obtain a corresponding short-time Fourier transform equation; Perform time-frequency analysis on the short-time Fourier transform equation to obtain a corresponding spectrum waterfall matrix.

6. The multi-site coordinated drone detection method according to claim 5, characterized in that: The short-time Fourier transform equation for time-frequency analysis of the RF signal x(t) to be identified is: Among them, STFT(t,f) is the frequency spectrum of the RF signal to be identified at a given time t, f is the frequency, h(τ-t) is the analysis window function, and x(τ) is the local signal of the RF signal to be identified x(t) intercepted corresponding to the analysis window function.

7. The multi-site coordinated UAV detection method according to claim 6, characterized in that: Perform time-frequency analysis on the short-time Fourier transform equation corresponding to the mth sensing station to obtain the corresponding spectrum waterfall matrix, which specifically includes: Perform time-frequency analysis on the short-time Fourier transform equation corresponding to the mth sensing station and construct the corresponding first matrix: Among them, STFT m To perform the short-time Fourier transform equation for the mth sensing station; Simplifying equation (2), we get the corresponding spectrum waterfall matrix: Wherein, m, n, k are all integers, and 0≤m≤J-1; n is the time window sequence number, and 0≤n≤N-1, N is the total number of time windows; k is the frequency point number, and 0≤k≤K-1, K is the total number of frequency points; bw is the analysis bandwidth.

8. The multi-site coordinated drone detection method according to claim 7, characterized in that: The spectrum waterfall matrix corresponding to the mth sensing site is mathematically transformed to obtain the corresponding amplitude spectrum waterfall diagram and phase spectrum waterfall diagram, including: Perform the first mathematical transformation on the spectrum waterfall matrix corresponding to the mth sensing site to obtain the corresponding amplitude spectrum waterfall diagram: Perform a second mathematical transformation on the spectrum waterfall matrix corresponding to the mth sensing site to obtain the corresponding amplitude spectrum waterfall diagram:

9. The multi-site coordinated drone detection method according to claim 1, characterized in that: The drone judgment signal is a one-dimensional vector signal.

10. A drone detection system, used to detect drones in an environment using J sensing stations in the environment, wherein the sensing stations are used to receive radio frequency signals to be identified related to objects in the environment, characterized in that: The system includes: A sensing station is used to collect J radio frequency signals to be identified, where the radio frequency signals to be identified include spectrum information and phase information related to objects in the environment, where J is an integer greater than or equal to 1; A first data preprocessing module, configured to perform time-frequency analysis on the J radio frequency signals to be identified to obtain a corresponding spectrum waterfall matrix; A second data preprocessing module is used to perform mathematical transformation on the J spectrum waterfall matrices to obtain corresponding amplitude spectrum waterfall diagrams and phase spectrum waterfall diagrams; An input feature preprocessing module, used to determine the amplitude spectrum input feature and the phase spectrum input feature based on the J amplitude spectrum waterfall diagrams and the J phase spectrum waterfall diagrams, respectively; wherein the amplitude spectrum input feature includes the sorting information, time information and frequency point information corresponding to the J amplitude spectrum waterfall diagrams, and the phase spectrum input feature includes the sorting information, time information and frequency point information corresponding to the J phase spectrum waterfall diagrams; The drone judgment module is used to input the amplitude spectrum input features and the phase spectrum input features into a pre-trained deep spatiotemporal frequency attention detection model, and output a drone judgment signal; the drone judgment signal includes drone existence judgment information and drone type judgment information.

11. An electronic device, characterized in that: The method comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 9 when executing the program.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images