Unmanned aerial vehicle signal detection method and system based on deep learning

Abstract

The invention relates to an unmanned aerial vehicle signal detection method and system based on deep learning, and the method comprises the steps: collecting a received signal, carrying out the vector accumulation after obtaining a signal time-frequency diagram, carrying out the conjugate multiplication of the signal time-frequency diagrams of two radio frequency channels, and solving arc tangent; and obtaining a time-phase difference map, normalizing the time-phase difference map into a grey-scale map, sending the grey-scale map into a trained deep learning model for unmanned aerial vehicle target detection, keeping a high-confidence target frame, then carrying out secondary detection to obtain a candidate target frame, and finally carrying out non-maximum suppression processing to obtain a final effective target frame, so that the algorithm is insensitive to sudden change of pixels, and the accuracy of the algorithm is improved. And by using the trained deep learning model, a convolution kernel type needing to be considered and used is avoided, so that the detection processing becomes more simplified, the method effectively adapts to a lower signal-to-noise ratio, and the recognition performance of the image transmission signal of the unmanned aerial vehicle under the lower signal-to-noise ratio is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of MIMO radar target detection, and relates to a method and system for detecting UAV signals based on deep learning. Background Art

[0002] With the increasingly widespread application of UAV technology, UAVs have brought more and more new challenges to the safety of the low-altitude area. Therefore, anti-UAV systems have emerged as the times require, and the detection of UAV signals is the operation basis of anti-UAV systems. Accurately detecting UAV signals is an important and arduous task. The detection of UAVs mainly includes the detection of active devices and passive devices. Active devices have the advantage of high accuracy, but are easily affected in areas with complex terrain, resulting in the inability to correctly observe the target. Therefore, passive devices are playing an increasingly important role at present.

[0003] UAV control mainly relies on the video data transmitted back by the video transmission signal. Especially for high-speed racing drones, it completely depends on the video transmission signal feedback to the pilot so that the pilot can perceive the surrounding environment of the UAV. Since the video transmission signal is sent by the UAV, the detection of the video transmission signal is the basis for locating the target UAV. The video transmission signal transmits a relatively large amount of data, has a relatively wide signal bandwidth and pulse width, and has the characteristic of continuous signal transmission. Generally, the OFDM (Orthogonal Frequency Division) method is used for modulation, and this modulation method is shown as a relatively regular rectangle on the time-frequency diagram.

[0004] At present, the recognition of the video transmission signal of UAVs is mainly achieved by performing edge detection after binarization on the time-frequency diagram or directly performing image convolution. Edge detection is prone to failure and introduce interference signals at low signal-to-noise ratios, and it is also very difficult to select the binarization threshold; image convolution requires the use of different convolution kernels for convolution, and the algorithm implementation is complex. Therefore, how to improve the recognition performance of the video transmission signal of UAVs at low signal-to-noise ratios has become a technical problem to be solved at present. Summary of the Invention

[0005] In view of the problems existing in the above-mentioned traditional technologies, the present invention proposes a method for detecting UAV signals based on deep learning and a system for detecting UAV signals based on deep learning, which can improve the recognition performance of the video transmission signal of UAVs at low signal-to-noise ratios.

[0006] To achieve the above object, the embodiments of the present invention adopt the following technical solutions: On the one hand, a method for detecting UAV signals based on deep learning is provided, including the steps of: Loading a trained deep learning model; the deep learning model is trained by using a pre-collected and labeled training data set; After collecting the received signal, it is sampled by ADC, windowed, and then subjected to short-time Fourier transform to obtain the time-frequency diagram of the signal; After performing vector accumulation on the time-frequency diagram of the signal, the time-frequency diagrams of the received signals received by the two RF channels are conjugated and multiplied and then the arctangent is taken to obtain the time-phase difference diagram; After converting the time-phase difference diagram into a grayscale image, it is fed into the deep learning model to obtain the upper left vertex coordinates and lower right vertex coordinates of a series of target boxes, and the confidence scores corresponding to the targets framed by the target boxes; After retaining the target boxes with confidence scores higher than the set confidence threshold, determine the categories of all the retained target boxes; Traverse the categories of all the retained target boxes, and perform secondary detection on all the retained target boxes to obtain a list of candidate target boxes corresponding to the categories; Perform non-maximum suppression processing on the list of candidate target boxes corresponding to the categories to obtain the valid target boxes corresponding to the categories.

[0007] On the other hand, a drone signal detection system based on deep learning is also provided, including: A model loading module for loading a trained deep learning model; the deep learning model is obtained by training using a pre-collected and labeled training data set; A time-frequency conversion module for collecting the received signal and then obtaining the time-frequency diagram of the signal through ADC sampling, windowing, and short-time Fourier transform; A vector diagram module for performing vector accumulation on the time-frequency diagram of the signal, conjugating and multiplying the time-frequency diagrams of the received signals received by the two RF channels, and taking the arctangent to obtain the time-phase difference diagram; A model detection module for converting the time-phase difference diagram into a grayscale image and then feeding it into the deep learning model to obtain the upper left vertex coordinates and lower right vertex coordinates of a series of target boxes, and the confidence scores corresponding to the targets framed by the target boxes; A target selection module for determining the categories of all the retained target boxes after retaining the target boxes with confidence scores higher than the set confidence threshold; A secondary detection module for traversing the categories of all the retained target boxes and performing secondary detection on all the retained target boxes to obtain a list of candidate target boxes corresponding to the categories; A non-maximum suppression processing module for performing non-maximum suppression processing on the list of candidate target boxes corresponding to the categories to obtain the valid target boxes corresponding to the categories.

[0008] One of the above technical solutions has the following advantages and beneficial effects: The above-mentioned UAV signal detection method and system based on deep learning collect received signals, perform ADC sampling, windowing, and short-time Fourier transform on the received signals to obtain a signal time-frequency diagram, and then perform vector accumulation to improve the image quality. Then, the signal time-frequency diagrams of the received signals received by the two RF channels are conjugated and multiplied, and the arctangent is calculated to obtain a time-phase difference diagram, which is normalized to a grayscale diagram and sent to a trained deep learning model for UAV target detection. After retaining the target boxes with high confidence, secondary detection is performed to obtain candidate target boxes, and finally non-maximum suppression processing is performed to obtain the final effective target boxes. By detecting and clustering UAV signals on the time-phase difference diagram, the algorithm is insensitive to pixel mutations, and the use of a trained deep learning model also avoids the need to consider the type of convolution kernel used, thus simplifying the detection process, effectively adapting to a low signal-to-noise ratio (such as below 10 dB), having a low false alarm and missed alarm rate, and improving the recognition performance of the UAV's video transmission signal under a low signal-to-noise ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0010] Figure 1 It is a flowchart of a UAV signal detection method based on deep learning in an embodiment; Figure 2 It is a schematic flowchart of creating a data set before training in an embodiment; Figure 3 It is a curve of the learning rate and loss change during training in an embodiment; Figure 4 It is a system framework and architecture flowchart applied by the method in an embodiment; Figure 5 It is a comparison schematic diagram of the time-frequency diagram and the time-phase difference diagram in an embodiment; Figure 6 It is a specific detection process schematic diagram of a UAV signal detection method based on deep learning in an embodiment; Figure 7 It is a precision-recall curve in an embodiment; Figure 8 It is the preliminary inspection result in an embodiment; Figure 9 It is an Elbow method curve in an embodiment; Figure 10 It is the clustering result in an embodiment; Figure 11 The secondary detection result in one embodiment; Figure 12 The final result in one embodiment; Figure 13 It is the module block diagram of a deep learning-based UAV signal detection system in one embodiment. Detailed implementation manners

[0011] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the description of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0012] It should be noted that referring to "embodiment" herein means that a specific feature, structure or characteristic described in connection with the embodiment may be included in at least one embodiment of the present invention. Displaying this phrase at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art can understand that the embodiments described herein can be combined with other embodiments. The term "and / or" used in the description and appended claims of the present invention refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0013] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0014] In one embodiment, as Figure 1 shown, a deep learning-based UAV signal detection method is provided, which may include the following steps S10 to S22: S10, load a trained deep learning model; the deep learning model is trained by using a pre-collected and labeled training data set; wherein, after receiving a signal collected in advance and performing a series of processes, a corresponding grayscale image is obtained, and then the video transmission signal of the UAV (i.e., target annotation) is marked on the grayscale image. A series of grayscale images after marking are used as a data set to pre-train the deep learning model, so that during the actual detection process, the received signal from the actual detection airspace is collected, a corresponding grayscale image is obtained after processing and sent into the trained deep learning model for rapid detection.

[0015] S12, after collecting the received signal, perform ADC sampling, windowing, and short-time Fourier transform to obtain a signal time-frequency diagram; S14. After performing vector accumulation on the signal time-frequency diagram, conjugate multiply the signal time-frequency diagrams of the received signals from the two RF channels and take the arctangent to obtain a time-phase difference diagram; S16. After converting the time-phase difference diagram into a grayscale image, input it into a deep learning model to obtain the coordinates of the upper left vertex and the lower right vertex of a series of target boxes, and the confidence score corresponding to the target enclosed by the target box; S18. After retaining the target boxes with confidence scores higher than the set confidence threshold, determine the categories of all the retained target boxes; S20. Traverse the categories of all the retained target boxes, perform secondary detection on all the retained target boxes to obtain a list of candidate target boxes corresponding to the categories; S22. Perform non-maximum suppression processing on the list of candidate target boxes corresponding to the categories to obtain valid target boxes corresponding to the categories.

[0016] In the above method for detecting UAV signals based on deep learning, after collecting the received signal, perform ADC sampling, windowing, and short-time Fourier transform to obtain the signal time-frequency diagram, and then perform vector accumulation to improve the image quality. Then, conjugate multiply the signal time-frequency diagrams of the received signals from the two RF channels and take the arctangent to obtain the time-phase difference diagram and normalize it into a grayscale image, input it into the trained deep learning model for UAV target detection, retain the target boxes with high confidence, perform secondary detection to obtain candidate target boxes, and finally perform non-maximum suppression processing to obtain the final valid target boxes. By detecting and clustering UAV signals on the time-phase difference diagram, the algorithm is insensitive to pixel mutations, and using the trained deep learning model also avoids the need to consider the type of convolution kernel used, thus simplifying the detection process, effectively adapting to a low signal-to-noise ratio (such as below 10 dB), having a low false alarm rate and a low missed alarm rate, and improving the recognition performance of the video transmission signal of the UAV under a low signal-to-noise ratio.

[0017] In one embodiment, before the above step S10, the following model training steps may further be included: After receiving the signal, perform ADC sampling, windowing, and short-time Fourier transform to obtain the signal time-frequency diagram; Perform vector accumulation on the signal time-frequency diagram; Conjugate multiply the signal time-frequency diagrams of the received signals from the two RF channels and take the arctangent to obtain the time-phase difference diagram and then convert it into a grayscale image; Use the grayscale images corresponding to multiple groups of received signals as the training data set for target annotation; Input the training data set after target annotation into the deep learning model for training. Until the detection error of the deep learning model for the target is lower than the set error threshold, export the trained deep learning model.

[0018] It can be understood that, as Figure 2 shown, in the model training stage, first, the received signal can be sampled by ADC (analog-to-digital conversion), windowed (such as a Hamming window), and then subjected to short-time Fourier transform to obtain a signal time-frequency diagram (in matrix form). At this time, the data in the signal time-frequency diagram is complex. The signal time-frequency diagram is two-dimensional and consists of many pixel points. Each pixel point has a coordinate indicating its position on the signal time-frequency diagram.

[0019] Then, in order to improve the quality of the signal time-frequency diagram, vector accumulation needs to be performed on the signal time-frequency diagram; the width of the time-phase difference diagram is w o , and the height is h o ; Vector accumulation means setting a window size, with the width w t and height h t of the window such that the width w o of the time-phase difference diagram can be w t divided evenly, and the result is w ; the height h o of the time-phase difference diagram can be h t divided evenly, and the result is h ; Therefore, the original signal time-frequency diagram can be divided into windows, and by summing the data in each window, a new signal time-frequency diagram with an image size of can be formed.

[0020] The signal time-frequency diagrams of the received signals from two RF channels are conjugate multiplied and the arctangent is taken to obtain the time-phase difference diagram. Then, the time-phase difference diagram is converted into a grayscale image through normalization; since the phase difference range is between (-π, π), the following processing is required: (1) where is the grayscale image obtained after processing, w , h are the width and height of the time-phase difference diagram, and is the phase difference value of the point at the coordinate x , y .

[0021] The grayscale images created from multiple groups of signals are used as the training dataset for UAV video transmission signal target annotation.

[0022] The grayscale images of the labeled training dataset are fed into a deep learning model for training. Existing deep learning models applicable to the object detection of the present invention can be but are not limited to the YOLO model, the Faster RCNN model, the Mask RCNN model, or the SSD model. In this specification, the Faster RCNN model is taken as an example.

[0023] When the detection error of the object during training is lower than the set error threshold, the trained deep learning model is exported; otherwise, after adjusting the model parameters of the deep learning model, return to the step of labeling the grayscale images created from multiple groups of signals as the training dataset and retrain until the detection accuracy meets the standard. As Figure 3 shown are the learning rate and loss change curves during training. Among them, the detection error of the object can be evaluated from aspects such as the loss function of the model and the detection accuracy rate. On the one hand, training will end when the loss value no longer has optimization during the model training process. It can be considered that the loss value is an adaptive threshold (that is, the loss value will continuously decrease during the training process, but when reaching the performance bottleneck of the model, the loss value will jitter within a small range or remain at a fixed value. If the loss value does not decrease for several consecutive rounds (or finally completes all training rounds), it is considered that the model has reached the performance limit of this training, and continuing to train will not improve, so training ends). On the other hand, the error threshold is specifically determined according to actual needs. For example, but not limited to, when the detection accuracy rate of the object exceeds 85% (that is, the detection error is less than 15%), it can be determined that the model has been trained well and can be used.

[0024] As Figure 4 shown, regarding the above-mentioned deep learning-based UAV signal detection method, in its detection and clustering stage (that is, steps S10 to S22), it includes four process links: object detection, clustering, secondary detection, and non-maximum suppression processing (NMS).

[0025] Among them, object detection includes: loading the trained deep learning model; collecting the received signal, and obtaining the signal time-frequency diagram after the received signal is sampled by ADC, windowed, and subjected to short-time Fourier transform; performing vector accumulation on the signal time-frequency diagram; conjugate multiplying the signal time-frequency diagrams of the received signals received by two RF channels and taking the arctangent to obtain the time-phase difference spectrum, as Figure 5 shown; converting the time-phase difference spectrum into a grayscale image according to Equation (1); feeding the grayscale image into the trained deep learning model to obtain the upper left vertex coordinates and lower right vertex coordinates of a series of object boxes, and the confidence score corresponding to the object framed by the object box; among them, the object box can be denoted as , and the confidence score can be denoted as . The coordinates of each object box include the upper left vertex coordinates and Lower right corner vertex coordinates.

[0026] Retain the bounding boxes with confidence scores higher than the set confidence threshold to obtain the target coordinates corresponding to this part of the bounding boxes; the confidence threshold can be determined according to the false alarm and missed alarm requirements to ensure that the targets above this threshold can be ensured to be valid targets (reduce the false alarm rate). For example, but not limited to, if a low false alarm rate is required, the confidence threshold can be set to 0.7, otherwise it can be set to 0.3 or other values. This step is implemented by Equation (2): (2) Where, where The function returns the number that meets the conditions, is the set confidence threshold. represents the bounding boxes corresponding to the confidence scores higher than the confidence threshold.

[0027] Regarding clustering: First, judge whether clustering is needed. Specifically, consider the abscissa of the upper left corner vertex coordinates of the bounding boxes corresponding to the starting frequency of the signal, and the video transmission signal of the drone is OFDM modulated and there will be no frequency hopping. According to the distribution of the starting frequency of the corresponding signal to judge whether clustering is needed: (3) Where, N is the number of retained bounding boxes i.e., the number of bounding boxes with confidence scores higher than the confidence threshold, is the mean value of the distribution of the abscissa of the upper left corner vertex coordinates of the bounding boxes of, represents the th abscissa of the upper left corner vertex coordinates of the bounding box, is the abscissa of the distribution standard deviation.

[0028] In one embodiment, regarding the above step S18, during the process of determining the categories of all retained bounding boxes, if the distribution standard deviation of the abscissa of the upper left corner vertex coordinates of the bounding boxes is less than the set standard deviation threshold, it is determined that the number of categories of all retained bounding boxes is 1 and directly enter the processing flow of secondary detection.

[0029] It can be understood that if the distribution standard deviation is less than the set standard deviation threshold (THS), it can be considered that the target box is concentrated in frequency distribution and has only one category, and no clustering processing is required, so it directly enters the secondary detection processing flow. Among them, the standard deviation threshold can be set according to the accuracy requirements of target classification, and can generally be set to 20% of the pixels occupied by the average width of the target box. The smaller the standard deviation threshold, the data originally in one category may be divided into multiple categories. If the standard deviation threshold is too large, the opposite is true.

[0030] In one embodiment, regarding the above step S18, in the process of determining the categories of all retained target boxes, if the distribution standard deviation of the horizontal coordinates of the upper left corner vertex coordinates of the target box is not less than the set standard deviation threshold, all retained target boxes are clustered using K-Means classification and Elbow method to determine the number of clustered categories.

[0031] Specifically, if the distribution standard deviation is not less than the set standard deviation threshold, the next step is to determine the number of clusters.

[0032] Determine the number of clusters: Because the signals of different drones may appear at the same time in the time dimension, adding the time dimension coordinates to the clustering will lead to a decrease in the clustering effect, so only the frequency dimension data is used in the clustering process. Use K-Means classification (i.e. the commonly used K-Means clustering, which is a commonly used unsupervised learning algorithm) and Elbow method (i.e. a method for determining the optimal number of clusters K in K-Means clustering) to select the appropriate number of cluster categories. Among them, the Elbow method mainly determines the optimal K value based on the sum of squared errors (SSE) of clustering. SSE refers to the sum of squares of the distances from each data point to the center point of the cluster to which it belongs. As the K value increases, the SSE will gradually decrease, because the more clusters there are, the more similar the data points in each cluster are, and the closer the distance between the data point and the cluster center is. However, when the K value increases to a certain extent, the reduction of SSE will gradually slow down. The Elbow method is to find a point similar to the "elbow" in the curve by drawing a relationship curve between SSE and K value. The K value corresponding to this point is the optimal number of cluster categories.

[0033] Specifically, assuming the number of clusters is , , perform the following iterations: use K-Means clustering to cluster the starting frequency and ending frequency of the target box, that is, initialize k Cluster centers (which can be recorded as Afterwards, the Euclidean distance from each object to each cluster center is calculated: (4) in, Indicates Attributes of an object Attributes Indicates the Attributes of the cluster center Indicates the Euclidean distance from the object to the cluster center. Here, the 2 on the summation symbol indicates that the object has two attributes here, namely the abscissa of the upper left vertex coordinate of the target box and the abscissa of the lower right vertex coordinate, corresponding to the starting frequency and the ending frequency of the signal. After each classification, calculate the sum of squared errors : (5) As increases, the sample division becomes finer, and the SSE will decrease. If the SSE value of the current th iteration is significantly reduced compared to the SSE value of the th iteration (such as less than 1 / 10 of the th iteration), then stop the iteration, which is the true number of clusters. At this time, the clustering result of K-Means clustering is the true clustering result

[0034] Secondary detection: Traverse all the categories of clusters, and calculate the mean of the abscissas of the upper left vertex coordinates of all target boxes, the mean of the widths of the target boxes, the starting frequency and bandwidth of the corresponding signals in each category respectively: (6) Among them, is the mean of the abscissas of the upper left vertex coordinates of the th category, is the mean of the widths of the target boxes in the th category, is the abscissa of the upper left vertex of the th target box in the th category, is the abscissa of the lower right vertex coordinate of the th target box in the th category. is the number of target boxes in the th category.

[0035] In one embodiment, the above-mentioned UAV signal detection method based on deep learning may further include the following processing steps: Traverse the target boxes with confidence scores lower than the confidence threshold, and classify the target boxes with width deviation less than the width deviation threshold and abscissa deviation of the upper left vertex coordinate less than the coordinate deviation threshold as valid target boxes. The width deviation threshold can be set to ±5% of the width. The higher this value is, the higher the false alarm rate and the lower the missed alarm rate. Conversely, the missed alarm rate is high and the false alarm rate is low. The coordinate deviation threshold is similar.

[0036] It can be understood that by traversing the target bounding boxes with confidence scores lower than the confidence threshold, the target bounding boxes with smaller width deviation and abscissa deviation of the abscissa of the upper left vertex coordinate are classified as valid target bounding boxes: (7) Among them, is the th target bounding box with a confidence score lower than the confidence threshold, is the set of the k th class of valid target bounding boxes, is 's width, is the abscissa of the abscissa of the upper left vertex coordinate, is the threshold of the width error of the target bounding box, is the threshold of the abscissa error of the abscissa of the upper left vertex coordinate of the target bounding box, is the absolute value function. Specifically, because low-confidence targets may be directly filtered out in the previous processing, in order to detect the targets that may be missed (i.e., false alarms) in the model detection results and reduce the false alarm rate in this embodiment, it is also possible to re-detect according to the rules of the already detected targets, as shown in the aforementioned formula (7).

[0037] Non-maximum suppression processing (NMS): After the secondary detection, there may be overlapping target bounding boxes, and NMS processing is required to remove the false alarm signals. First, traverse all classes, calculate the area for all target bounding boxes in each class, which can be denoted as : (8) Among them, is the abscissa of the upper left vertex coordinate of the th target bounding box in the th class, is the abscissa of the lower right vertex coordinate of the th target bounding box in the th class.

[0038] Specifically, after sorting the target bounding boxes in each class in descending order of confidence score, they are used as all candidate bounding boxes corresponding to each class. For each class, first consider the first candidate bounding box in all its candidate bounding boxes as a valid target bounding box, and then calculate the overlapping area between the other remaining candidate bounding boxes and the first candidate bounding box, denoted as : (9) Among them, is the coordinate of the upper left vertex of the overlapping part between all other remaining candidate bounding boxes in the th class and the first candidate bounding box, is the The coordinates of the lower-right vertices of the overlapping parts of all other remaining candidate boxes of the class with the first candidate box (valid target box). Is the upper-left vertex coordinate of the first candidate box among all the remaining candidate boxes of class Is the lower-right vertex coordinate of the first candidate box among all the remaining candidate boxes of class Is the upper-left vertex coordinate of the remaining candidate boxes other than the first candidate box in class Is the lower-right vertex coordinate of the remaining candidate boxes other than the first candidate box in class

[0039] Calculate the intersection over union (IoU) of the candidate boxes for each class based on the overlapping area: (10) Where Represents the IoU of the first candidate box with the other remaining candidate boxes in class Represents the area of the first candidate box among all the candidate boxes in class Represents the area of all the remaining candidate boxes other than the first candidate box in class

[0040] For each class, remove all candidate boxes with an IoU greater than the set IoU threshold (since the video transmission signals of a single drone do not overlap, this threshold can be set below 20%). Send the first candidate box to the valid target box queue corresponding to the class. If the number of remaining candidate boxes is 0, it means that all candidate boxes of the class have been processed, and the valid target boxes in the valid target box queue are the final target boxes of the class. If the number of remaining candidate boxes is not 0, then return to the processing step of sorting the target boxes in descending order of confidence score to continue the iteration until all candidate boxes of the class have been processed.

[0041] In some embodiments, as Figure 6 shown, is a schematic diagram of the specific detection process of the above-mentioned drone signal detection method based on deep learning. As can be seen from the precision-recall curve shown in Figure 7 the recall rate and accuracy of the above-mentioned drone signal detection method based on deep learning both exceed 90%. Among them, the preliminary inspection results of the above-mentioned drone signal detection method during the test can be as shown in Figure 8 Figure 9 is the Elbow method curve,[[]] Figure 10 is the clustering result,[[]] Figure 11 is the secondary detection result,[[]] Figure 12 ​​​​​​​​As the final result, where the blue box is the video transmission signal of UAV target 0, the red box is the video transmission signal of UAV target 1, and UAV target 0 and UAV target 1 are two different UAV targets.

[0042] Traditional edge detection algorithms (such as the Sobel edge detection operator) are sensitive to pixel mutations. However, in the signal time-frequency diagram and the above-mentioned time-phase difference diagram, there are more pixel mutations due to the existence of noise, resulting in the failure of traditional edge detection algorithms. The above-mentioned UAV signal detection method based on deep learning in the present invention is not sensitive to pixel mutations, so there is no such sensitivity problem, and it is not necessary to consider the type of convolution kernel used like traditional image convolution algorithms. The processing implementation is more simplified and can effectively filter out some non-UAV signals, with a lower false alarm rate.

[0043] It should be understood that although the various steps in the above process Figure 1 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above process Figure 1 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0044] In one embodiment, as Figure 13As shown in the figure, a drone signal detection system 100 based on deep learning is provided, which includes a model loading module 11, a time-frequency conversion module 13, a vector spectrogram module 15, a model detection module 17, a target selection module 19, a secondary detection module 21, and a non-maximum suppression processing module 23. Among them, the model loading module 11 is used to load a trained deep learning model; the deep learning model is obtained by training using a pre-collected and labeled training data set. The time-frequency conversion module 13 is used to obtain a signal time-frequency diagram after collecting the received signal, performing ADC sampling, windowing, and short-time Fourier transform. The vector spectrogram module 15 is used to perform vector accumulation on the signal time-frequency diagram, conjugate multiply the signal time-frequency diagrams of the received signals received by two radio frequency channels, and take the arctangent to obtain a time-phase difference spectrogram. The model detection module 17 is used to convert the time-phase difference spectrogram into a grayscale image and send it into the deep learning model to obtain the upper left vertex coordinates and lower right vertex coordinates of a series of target boxes, and the confidence score corresponding to the target framed by the target box. The target selection module 19 is used to retain the target boxes with a confidence score higher than the set confidence threshold, and then determine the categories of all the retained target boxes. The secondary detection module 21 is used to traverse the categories of all the retained target boxes, perform secondary detection on all the retained target boxes, and obtain a list of candidate target boxes corresponding to the categories. The non-maximum suppression processing module 23 is used to perform non-maximum suppression processing on the list of candidate target boxes corresponding to the categories to obtain valid target boxes corresponding to the categories.

[0045] For the above-mentioned drone signal detection system 100 based on deep learning, after collecting the received signal, ADC sampling, windowing, and short-time Fourier transform are performed to obtain a signal time-frequency diagram, and then vector accumulation is performed to improve the image quality. Then, the signal time-frequency diagrams of the received signals received by two radio frequency channels are conjugate multiplied and the arctangent is taken to obtain a time-phase difference spectrogram, which is normalized to a grayscale image and sent into the trained deep learning model for drone target detection. After retaining the target boxes with high confidence, secondary detection is performed to obtain candidate target boxes, and finally non-maximum suppression processing is performed to obtain the final valid target boxes. By detecting and clustering the drone signals on the time-phase difference spectrogram, the algorithm is insensitive to pixel mutations, and using the trained deep learning model also avoids the need to consider the type of convolution kernel used, thus making the detection process simpler, effectively adapting to a lower signal-to-noise ratio (such as below 10 dB), having a lower false alarm rate and missed alarm rate, and improving the recognition performance of the drone's video transmission signal under a lower signal-to-noise ratio.

[0046] In one embodiment, during the process of determining the categories of all the retained target boxes, if the standard deviation of the distribution of the abscissa of the upper left vertex coordinates of the target box is less than the set standard deviation threshold, it is determined that the categories of all the retained target boxes are 1 and directly enter the processing flow of secondary detection.

[0047] In one embodiment, during the process of determining the categories of all the retained target boxes, if the standard deviation of the distribution of the abscissa of the coordinates of the upper left vertex of the target box is not less than the set standard deviation threshold, K-Means clustering and the Elbow method are used to cluster all the retained target boxes to determine the number of clustering categories.

[0048] In one embodiment, the target selection module 19 is further configured to traverse the target boxes with confidence scores lower than the confidence threshold, and classify the target boxes with a width deviation less than the width deviation threshold and an abscissa deviation of the coordinates of the upper left vertex less than the coordinate deviation threshold as valid target boxes.

[0049] In one embodiment, the above-mentioned deep learning-based UAV signal detection system 100 may further include a model training module, which is configured to perform ADC sampling, windowing, and short-time Fourier transform on the received signal to obtain a signal time-frequency diagram, perform vector accumulation on the signal time-frequency diagram, conjugate multiply the signal time-frequency diagrams of the received signals received by two RF channels, and calculate the arctangent to obtain a time-phase difference diagram and convert it into a grayscale image. The grayscale images corresponding to multiple groups of received signals are used as a training data set for target annotation and then sent into a deep learning model for training. When the detection error of the deep learning model for the target is lower than the set error threshold, the trained deep learning model is exported.

[0050] For the specific limitations of the above-mentioned deep learning-based UAV signal detection system 100, reference may be made to the corresponding limitations of the deep learning-based UAV signal detection method in the foregoing text, which will not be elaborated herein.

[0051] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus dynamic random access memory (Rambus DRAM, abbreviated as RDRAM), and interface dynamic random access memory (DRDRAM), etc.

[0052] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0053] The above embodiments only represent several implementation manners of the present invention, and the description is relatively specific and detailed, but it cannot be construed as a limitation on the protection scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and they all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. A method for detecting drone signals based on deep learning, characterized in that, Including the steps: Loading a trained deep learning model; the deep learning model is obtained by training using a pre-collected and labeled training data set; After collecting the received signal, performing ADC sampling, windowing, and short-time Fourier transform to obtain a signal time-frequency diagram; After performing vector accumulation on the signal time-frequency diagram, conjugately multiplying and taking the arctangent of the signal time-frequency diagrams of the received signals received by two RF channels to obtain a time-phase difference diagram; Converting the time-phase difference diagram into a grayscale image and feeding it into the deep learning model to obtain the upper left vertex coordinates and lower right vertex coordinates of a series of target boxes, and the confidence score corresponding to the target framed by the target box; After retaining the target boxes with confidence scores higher than the set confidence threshold, determining the categories of all the retained target boxes; Traversing the categories of all the retained target boxes, and performing secondary detection on all the retained target boxes to obtain a list of candidate target boxes corresponding to the categories; Performing non-maximum suppression processing on the list of candidate target boxes corresponding to the categories to obtain valid target boxes corresponding to the categories.

2. The method for detecting drone signals based on deep learning according to claim 1, wherein During the process of determining the categories of all the retained target boxes, if the standard deviation of the distribution of the abscissa of the upper left vertex coordinates of the target box is less than the set standard deviation threshold, it is determined that the number of categories of all the retained target boxes is 1 and directly enter the processing flow of secondary detection.

3. The method for detecting drone signals based on deep learning according to claim 2, wherein During the process of determining the categories of all the retained target boxes, if the standard deviation of the distribution of the abscissa of the upper left vertex coordinates of the target box is not less than the set standard deviation threshold, use K-Means classification and the Elbow method to cluster all the retained target boxes to determine the number of clustering categories.

4. The method for detecting drone signals based on deep learning according to any one of claims 1 to 3, characterized in that, Also including the steps: Traversing the target boxes with confidence scores lower than the confidence threshold, and classifying the target boxes with a width deviation less than the width deviation threshold and an abscissa deviation of the upper left vertex coordinates less than the coordinate deviation threshold as valid target boxes.

5. The method for detecting drone signals based on deep learning according to claim 1, wherein Before the step of loading the trained deep learning model, it also includes the steps: Performing ADC sampling, windowing, and short-time Fourier transform on the received signal to obtain a signal time-frequency diagram; Performing vector accumulation on the signal time-frequency diagram; Conjugately multiplying and taking the arctangent of the signal time-frequency diagrams of the received signals received by two RF channels, and converting the obtained time-phase difference diagram into a grayscale image; Using the grayscale images corresponding to multiple groups of received signals as a training data set for target annotation; Feeding the target-annotated training data set into the deep learning model for training, and exporting the trained deep learning model until the detection error of the deep learning model for the target is lower than the set error threshold.

6. A drone signal detection system based on deep learning, characterized in that, Including: A model loading module for loading a trained deep learning model; The deep learning model is obtained by training using a pre-collected and labeled training data set; A time-frequency conversion module for obtaining a signal time-frequency diagram after collecting the received signal and performing ADC sampling, windowing, and short-time Fourier transform; A vector diagram module for performing vector accumulation on the signal time-frequency diagram, and then conjugately multiplying and taking the arctangent of the signal time-frequency diagrams of the received signals received by two RF channels to obtain a time-phase difference diagram; The model detection module is used to convert the time-phase difference spectrogram into a grayscale image and then send it into the deep learning model to obtain the coordinates of the upper left vertex and the lower right vertex of a series of target boxes, and the confidence score corresponding to the target framed by the target box; The target selection module is used to determine the categories of all the remaining target boxes after retaining the target boxes with confidence scores higher than the set confidence threshold; The secondary detection module is used to traverse the categories of all the remaining target boxes and perform secondary detection on all the remaining target boxes to obtain a list of candidate target boxes corresponding to the categories; The non-maximum suppression processing module is used to perform non-maximum suppression processing on the list of candidate target boxes corresponding to the categories to obtain the valid target boxes corresponding to the categories.

7. The drone signal detection system based on deep learning according to claim 6, wherein During the process of determining the categories of all the remaining target boxes, if the standard deviation of the distribution of the abscissa of the upper left vertex coordinates of the target box is less than the set standard deviation threshold, it is determined that the number of categories of all the remaining target boxes is 1 and directly enter the processing flow of secondary detection.

8. The drone signal detection system based on deep learning according to claim 7, characterized in that, During the process of determining the categories of all the remaining target boxes, if the standard deviation of the distribution of the abscissa of the upper left vertex coordinates of the target box is not less than the set standard deviation threshold, the K-Means classification and the Elbow method are used to cluster all the remaining target boxes to determine the number of clustering categories.

9. The drone signal detection system based on deep learning according to any one of claims 6 to 8, characterized in that, The target selection module is also used to traverse the target boxes with confidence scores lower than the confidence threshold, and classify the target boxes with width deviation less than the width deviation threshold and abscissa deviation of the upper left vertex coordinates less than the coordinate deviation threshold as valid target boxes.

10. The drone signal detection system based on deep learning according to claim 6, characterized in that, It further includes a model training module, which is used to obtain a signal time-frequency diagram after receiving a signal, performing ADC sampling, windowing, and short-time Fourier transform on it, performing vector accumulation on the signal time-frequency diagram, then conjugating and multiplying the signal time-frequency diagrams of the received signals received by two radio frequency channels and taking the arctangent to obtain a time-phase difference spectrogram and converting it into a grayscale image. The grayscale images corresponding to multiple groups of received signals are used as a training data set for target annotation and then sent into the deep learning model for training. Until the detection error of the deep learning model for the target is lower than the set error threshold, the trained deep learning model is exported.

Images