UAV automatic identification method, device, electronic equipment and computer medium

Abstract

The embodiments of the present disclosure disclose a method, device, electronic device and computer medium for automatic identification of drones. A specific implementation of the method includes: in response to determining that a radio detection device detects a radio signal, preprocessing the radio signal to generate preprocessed radio spectrum data; extracting radio signal features in the preprocessed radio spectrum data; inputting the radio signal features into a pre-trained drone signal recognition model to obtain a drone signal recognition result; in response to determining that the drone signal recognition result represents the drone signal, controlling a high-definition camera device associated with the radio detection device to capture a video of the surrounding environment at the first time in real time; and generating drone detection information corresponding to the first time based on a three-dimensional feature information group. This implementation improves the accuracy of drone recognition.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the computer field, and in particular to a method, device, electronic device and computer medium for automatically identifying a drone. Background Art

[0002] Drone technology has developed rapidly in recent years, and its application scope has been expanding from military use to civilian and commercial fields. However, the popularity of drones has also brought security risks, including illegal intrusion of drones into sensitive areas, threats to public safety, infringement of privacy, and potential terrorist activities. At present, the method commonly used to identify drones is to identify drones by drone signal frequency bands.

[0003] However, it is found in practice that when the above method is used for drone identification, the following technical problems often occur: drones are identified only by the drone signal frequency band, resulting in errors in identification.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a method, device, electronic device and computer-readable medium for automatic identification of drones to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a method for automatic identification of drones, the method comprising: in response to determining that a radio detection device detects a radio signal, preprocessing the radio signal to generate preprocessed radio spectrum data; extracting radio signal features in the preprocessed radio spectrum data; inputting the radio signal features into a pre-trained drone signal recognition model to obtain a drone signal recognition result; in response to determining that the drone signal recognition result represents the drone signal, controlling a high-definition camera device associated with the radio detection device to capture an environmental video of the surrounding environment at a first time in real time; generating two-dimensional feature information corresponding to each frame of video image in the environmental video to obtain a two-dimensional feature information sequence; generating a drone three-dimensional feature information group corresponding to the drone detection object set at the first time according to a historical three-dimensional environmental feature information group; generating a three-dimensional feature information group at the first time according to the drone three-dimensional feature information group, a pre-constructed drone query information group and the two-dimensional feature information sequence; generating drone detection information corresponding to the first time according to the three-dimensional feature information group.

[0008] In a second aspect, some embodiments of the present disclosure provide a device for automatic identification of drones, the device comprising: a preprocessing unit, configured to, in response to determining that a radio detection device detects a radio signal, preprocess the radio signal to generate preprocessed radio spectrum data; an extraction unit, configured to extract radio signal features from the preprocessed radio spectrum data; an input unit, configured to input the radio signal features into a pre-trained drone signal identification model to obtain a drone signal identification result; a control unit, configured to, in response to determining that the drone signal identification result represents the drone signal, control a high-definition camera device associated with the radio detection device to shoot in real time An environmental video of the surrounding environment at a first time; a first generating unit, configured to generate two-dimensional feature information corresponding to each frame of video image in the above environmental video, and obtain a two-dimensional feature information sequence; a second generating unit, configured to generate a drone three-dimensional feature information group corresponding to the drone detection object set at the above first time according to the historical three-dimensional environmental feature information group; a third generating unit, configured to generate a three-dimensional feature information group at the above first time according to the drone three-dimensional feature information group, a pre-constructed drone query information group and the above two-dimensional feature information sequence; a fourth generating unit, configured to generate the drone detection information corresponding to the above first time according to the three-dimensional feature information group.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0011] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the automatic identification method of drones in some embodiments of the present disclosure, the accuracy of drone identification is improved. Specifically, the reason for the error in identification is that the drone is identified only by the drone signal frequency band, and the identification has errors. Based on this, the automatic identification method of drones in some embodiments of the present disclosure, first, in response to determining that the radio detection device detects a radio signal, the radio signal is preprocessed to generate preprocessed radio spectrum data. Secondly, the radio signal features in the preprocessed radio spectrum data are extracted. Then, the radio signal features are input into the pre-trained drone signal recognition model to obtain the drone signal recognition result. Thus, by detecting the radio signal of the drone, its spectrum features are identified. Using spectrum analysis technology, the 20M-8G frequency band is covered to meet the needs of complex scenarios. The drone spectrum graph is trained and annotated using a deep learning algorithm to achieve accurate identification. Thirdly, in response to determining that the drone signal recognition result represents the drone signal, the high-definition camera device associated with the radio detection device is controlled to capture the environmental video of the surrounding environment at the first time in real time. Afterwards, the two-dimensional feature information corresponding to each frame of the video image in the environmental video is generated to obtain a two-dimensional feature information sequence. Then, based on the historical three-dimensional environmental feature information group, a drone three-dimensional feature information group corresponding to the drone detection object set at the first time is generated. Thus, it is possible to further determine whether there is a drone in the surrounding airspace through image recognition and three-dimensional recognition technology. Finally, based on the drone three-dimensional feature information group, the pre-constructed drone query information group and the two-dimensional feature information sequence, a three-dimensional feature information group at the first time is generated. Based on the three-dimensional feature information group, the drone detection information corresponding to the first time is generated. Thus, drone identification can be performed from two perspectives: drone signal and environmental image, which improves the accuracy of drone identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flow chart of some embodiments of the method for automatic identification of drones according to the present disclosure;

[0014] Figure 2 is a schematic diagram of the structure of some embodiments of the automatic identification device of a drone according to the present disclosure;

[0015] Figure 3 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure;

[0016] Figure 4 This is a screenshot of an internal exemplary code for drone signal recognition in the drone automatic recognition method disclosed in the present invention. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 1 is a flow chart of some embodiments of the method for automatically identifying a drone according to the present disclosure. 100 of some embodiments of the method for automatically identifying a drone according to the present disclosure is shown. The method for automatically identifying a drone comprises the following steps:

[0024] Step 101, in response to determining that a radio detection device detects a radio signal, preprocessing the radio signal to generate preprocessed radio spectrum data.

[0025] In some embodiments, the execution subject (e.g., computing device) of the automatic identification method of a drone may, in response to determining that a radio detection device detects a radio signal, pre-process the radio signal to generate pre-processed radio spectrum data. The radio detection device may be a device for detecting a drone radio signal. For example, the radio detection device may be a radio detector or a wireless signal detector. For example, pre-processing the radio signal may include: denoising, filtering, and normalization.

[0026] Step 102: extract radio signal features from the pre-processed radio spectrum data.

[0027] In some embodiments, the above-mentioned execution entity can extract radio signal features from the above-mentioned pre-processed radio spectrum data. For example, features in the pre-processed radio spectrum data, such as frequency, amplitude, phase, modulation mode, etc., can be extracted and converted into feature vectors. The above-mentioned execution entity can extract radio signal features from the above-mentioned pre-processed radio spectrum data through a preset radio spectrum feature extraction method. The radio spectrum feature extraction method may include: cyclic spectrum feature extraction, time domain feature extraction (multi-layer perceptron (MLP), convolutional neural network (CNN)). The radio signal features in the above-mentioned pre-processed radio spectrum data can also be extracted through Fourier Transform (Fourier Transform) and Fast Fourier Transform (Fast Fourier Transform, FFT).

[0028] Step 103: input the above radio signal features into a pre-trained drone signal recognition model to obtain a drone signal recognition result.

[0029] In some embodiments, the execution subject may input the radio signal features into a pre-trained drone signal recognition model to obtain a drone signal recognition result. The drone signal recognition model may be a pre-trained neural network model that takes the radio signal features as input and the drone signal recognition result as output. For example, the drone signal recognition model may be a pre-trained CNN model. Figure 4 The example shows some code screenshots for drone signal recognition. Figure 4 The code for drone signal recognition shown in the example is just a screenshot of the internal exemplary code.

[0030] Among them, the drone signal recognition model can be trained through the following steps:

[0031] The first step is to obtain the drone radio signal spectrum group set. Among them, the drone radio signal spectrum group in the drone radio signal spectrum group set corresponds to a drone radio signal type. For example, collect a large number of radio spectrum signals of different types of drones, covering multiple frequency bands (20M-8G). When collecting data, it is necessary to ensure the diversity of different drone types, distances, and environmental conditions.

[0032] The second step is to label each drone radio signal spectrum in the drone radio signal spectrum set to obtain a labeled drone radio signal spectrum set as a drone radio signal spectrum sample set. For example, the collected drone radio signal spectrum is labeled, including labels such as drone model and signal type, to form a labeled data set.

[0033] The third step is to determine the initial drone signal recognition model. The initial drone signal recognition model includes: an initial signal feature extraction network, an initial multi-level signal feature hybrid autoencoder network, and an initial signal recognition network. For example, the initial signal feature extraction network can be a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), etc.

[0034] The third step may include:

[0035] The first sub-step is to determine the initial signal feature extraction network.

[0036] The second sub-step is to determine the initial multi-level signal feature hybrid autoencoder network. The initial multi-level signal feature hybrid autoencoder network includes: an initial encoding network (encoder) and an initial decoding network (decoder). The initial encoding network includes: an initial signal feature decoupling network (including an autoencoder network and an LSTM network) and an initial signal feature fusion network (VAE (Variational AutoEncoder, variational encoder). The initial signal feature decoupling network includes: an initial static signal feature decoupling network and an initial dynamic signal feature decoupling network. The initial signal feature fusion network includes: an initial fusion network and an initial reparameterization network. The initial static signal feature decoupling network can be an Auto-Encoder model. The initial dynamic signal feature decoupling network can be an LSTM (Long Short-Term Memory) model. The initial fusion network can be a Feature Pyramid Network (FPN) or a Self-Attention Mechanism network. The initial reparameterization network can include: DyRep (Dynamic Re-parameterization) and ACNet (Asymmetric Convolution Net).

[0037] The third sub-step is to determine the initial signal recognition network. First, determine the signal perception classification network. For example, the signal perception classification network can be a multi-layer perception classifier based on KAN (Kolmogorov-Arnold Networks, a new neural network architecture). Secondly, initialize the above-mentioned signal perception classification network to generate an initial signal perception classification network. The above-mentioned signal perception classification network can be initialized by the Kaiming initialization algorithm to generate an initial signal perception classification network. Next, the above-mentioned initial signal perception classification network is subjected to noise processing to generate a noise signal perception classification network. The above-mentioned initial signal perception classification network can be subjected to noise processing by a Gaussian noise algorithm to generate a noise signal perception classification network. Then, the above-mentioned noise signal perception classification network is subjected to parameter regularization processing to generate a signal perception classification regularized network. The above-mentioned noise signal perception classification network can be subjected to parameter regularization processing by an L1 regularization (L1 norm regularization) algorithm to generate a signal perception classification regularized network. Finally, the above-mentioned signal perception classification regularized network is subjected to activation adjustment processing to generate an initial signal recognition network. The activation function can be used to perform activation adjustment processing on the signal perception classification regularization network to generate an initial signal recognition network.

[0038] In the fourth sub-step, the initial signal feature extraction network, the initial multi-level signal feature hybrid autoencoder network, and the initial signal recognition network are integrated to generate an initial UAV signal recognition model.

[0039] In the fourth step, for each drone radio signal spectrum sample group in the drone radio signal spectrum sample group set, a target drone radio signal spectrum sample is selected from the drone radio signal spectrum sample group. A drone radio signal spectrum sample can be randomly selected from the drone radio signal spectrum sample group as the target drone radio signal spectrum sample.

[0040] The fifth step is to train the initial drone signal recognition model based on the radio signal spectrum samples of each target drone to obtain a trained drone signal recognition model.

[0041] The fifth step may include the following sub-steps:

[0042] In the first sub-step, for each target drone radio signal spectrum sample in each target drone radio signal spectrum sample, the following training steps are performed:

[0043] 1. Input the sample UAV radio signal spectrum included in the target UAV radio signal spectrum sample into the initial signal feature extraction network to obtain the initial UAV radio signal feature information.

[0044] 2. Input the initial UAV radio signal feature information into the initial multi-level signal feature hybrid autoencoder network to obtain the initial multi-level UAV radio signal feature information;

[0045] 3. Input the initial multi-level UAV radio signal feature information into the initial signal recognition network to obtain the initial UAV radio signal recognition result.

[0046] 4. Based on the preset loss function, determine the signal loss value between the initial drone radio signal recognition result and the sample label included in the target drone radio signal spectrum sample. The loss function can be: mean square error loss (MSELoss) function, cross-entropy loss (Cross-Entropy Loss) function, and mean absolute error (MAE) function.

[0047] In the second sub-step, in response to determining that the sum of the signal loss values ​​is less than or equal to a preset loss value, the initial drone signal recognition model is determined to be a trained drone signal recognition model.

[0048] Therefore, the present invention constructs a multi-level signal feature hybrid autoencoder network, which can fully capture the multi-level information in the signal and enhance the efficiency and accuracy of the autoencoder in feature untangling. In addition, a multi-layer signal recognition network is also constructed, which can help the model capture the complex nonlinear relationship in the signal data and improve the accuracy of signal recognition.

[0049] Step 104 , in response to determining that the drone signal recognition result represents the drone signal, controlling the high-definition camera device associated with the radio detection device to capture a video of the surrounding environment at the first time in real time.

[0050] In some embodiments, the above-mentioned execution subject can control the high-definition camera device associated with the above-mentioned radio detection device to capture the environmental video of the surrounding environment at the first time in real time in response to determining that the drone signal recognition result represents the drone signal. The high-definition camera device may refer to a high-definition camera set around the radio detection device for monitoring the surrounding low-altitude area. The high-definition camera device can rotate 360 ​​degrees to shoot and has a 3D shooting function. The first time may refer to a time period of a preset duration from the time point when the drone signal recognition result is determined to represent the drone signal. The preset duration may be 1 minute.

[0051] Step 105: Generate two-dimensional feature information corresponding to each frame of the video image in the above-mentioned environment video to obtain a two-dimensional feature information sequence.

[0052] In some embodiments, the execution subject may generate two-dimensional feature information corresponding to each frame of the video image in the environment video to obtain a two-dimensional feature information sequence. The two-dimensional feature information may represent feature information in a two-dimensional (2D) space corresponding to the video image. The two-dimensional feature information may be information in vector form that represents image semantics corresponding to the video image.

[0053] As an example, the above-mentioned execution entity can input each video image into a convolutional neural network model to generate two-dimensional feature information and obtain a two-dimensional feature information set.

[0054] In practice, the execution subject may generate the two-dimensional feature information corresponding to each frame of the video image in the environment video through the following steps:

[0055] In the first step, for each frame of video image, perform the following steps:

[0056] First, the video image is divided and processed according to the preset number of divided regions to obtain a group of divided region images. The preset number of divided regions may represent the number of divided region images into which the video image is divided. For example, the preset number of divided regions may be 6. The specific value of the preset number of divided regions is not limited. Each of the divided region images may represent an image divided from the video image.

[0057] Second, for each divided region image in the divided region image group, the following steps are iterated:

[0058] 1. Determine the histogram information of the divided region image. The histogram information may represent the histogram of the divided region image. In practice, the execution subject may determine the histogram information of the divided region image through the cv2.calcHist() function in the OpenCV library.

[0059] 2. Perform equalization processing on the above histogram information to obtain equalized histogram information.

[0060] 3. Perform grayscale value mapping on the equalized histogram information to obtain the grayscale level corresponding to the above-mentioned divided area image.

[0061] 4. Determine the threshold information corresponding to the divided area image according to the grayscale level, the quantity information of the divided area image and the cropping factor information. The threshold information may represent the histogram cropping threshold. The quantity information of the divided area image may represent the number of pixels in the divided area image. In practice, first, the execution entity may determine the ratio of the number of pixels of the divided area image to the grayscale level as the target ratio. Then, the product of the target ratio and the cropping factor information is determined as the threshold information.

[0062] 5. Update the divided region image according to the threshold information to obtain an updated divided region image. The execution subject may perform cropping on the divided region image according to the threshold information by a threshold segmentation algorithm to obtain a cropped divided region image. Then, an interpolation algorithm may be used to perform pixel allocation processing on each pixel in the cropped divided region image to obtain a pixel allocated divided region image as the updated divided region image.

[0063] 6. Perform histogram equalization processing on the updated divided region image to obtain a histogram equalized updated divided region image as the divided region image.

[0064] 7. Update the number of iterations and obtain the updated number of iterations as the number of iterations.

[0065] Third, the obtained divided area images are fused and processed, and the obtained divided area images after fusion processing are merged into fused image information. The above-mentioned execution subject can fuse the obtained divided area images through an image fusion algorithm to obtain the obtained divided area images after fusion processing as fused image information. For example, the image fusion algorithm can be a spatial domain fusion algorithm.

[0066] Fourth, in response to determining that the number of iterations is less than the preset number of iterations, the fused image information is used as the wound surface injection image information, and the above division step is performed again.

[0067] Fifth, in response to determining that the number of iterations is greater than or equal to the preset number of iterations, the fused image information is determined as an enhanced video image.

[0068] The second step is to generate two-dimensional feature information corresponding to each enhanced video image to obtain a two-dimensional feature information sequence. For example, the enhanced video image can be input into a pre-trained two-dimensional image feature extraction model to generate the two-dimensional feature information. The two-dimensional image feature extraction model can be a neural network model that extracts image feature information from a two-dimensional image. In practice, the two-dimensional image feature extraction model can be a multi-layer convolutional layer connected in series. The two-dimensional feature information can characterize the semantic content of the image features in the enhanced video image.

[0069] Step 106: Generate a drone three-dimensional feature information group corresponding to the drone detection object set at the first time according to the historical three-dimensional environment feature information group.

[0070] In some embodiments, the execution subject may generate a drone three-dimensional feature information group corresponding to the drone detection object set at the first time according to the historical three-dimensional environment feature information group. The historical three-dimensional environment feature information group may represent the detection content information of each drone detection object at the historical time. In practice, the drone detection object in each drone detection object corresponds to the historical three-dimensional environment feature information in the historical three-dimensional environment feature information group. The historical three-dimensional feature information may be feature information of the corresponding drone detection object in the corresponding three-dimensional (3D) space at the historical time. The historical three-dimensional feature information may be information in the form of a vector. At the historical time, the drone has a corresponding flight position and posture, and a corresponding three-dimensional space coordinate system may be established for the current high-definition camera device. That is, the historical three-dimensional feature information may be information representing the corresponding detection features of the drone detection object in the corresponding three-dimensional space coordinate system at the historical time. Each drone detection object at the historical time has a corresponding detection confidence, that is, each historical three-dimensional feature information has a corresponding confidence, representing the accuracy of the historical three-dimensional feature information representing the corresponding detection features of the drone detection object in the corresponding three-dimensional coordinate system. The detection confidence may be a value between 0 and 1, and the higher the value, the higher the accuracy of the corresponding historical three-dimensional feature information. Among them, the drone detection object set is each drone detection object detected at a historical time. The drone three-dimensional feature information group can represent the feature information of each drone detection object in the drone detection object set in the three-dimensional coordinate system corresponding to the first time. The drone three-dimensional feature information can be feature information in the form of a vector. The three-dimensional coordinate system can be a three-dimensional coordinate system established based on a high-definition camera at the first time. For example, first, a first image set corresponding to the first time and a second image set corresponding to the historical time are obtained. According to the above-mentioned first image set and the above-mentioned second image set, the position transformation information generation model is used to determine the change information of each object between the first image set and the second image set. In practice, the position transformation information generation model can be a neural network model that generates position transformation information. In practice, the position transformation information generation model can include: a position detection model and a position mapping model. Among them, the position detection model can be a neural network model that generates position information in an image. The position mapping model can be a mapping model that generates position transformation between position information between two identical drone objects. The position mapping model can be a multi-layer series of fully connected layers. Finally, according to the above change information, the historical three-dimensional feature information group is mapped to generate a three-dimensional feature information group corresponding to the drone detection object set at the first time as the drone three-dimensional feature information group.

[0071] Step 107, generating the three-dimensional feature information group at the first time according to the three-dimensional feature information group of the drone, the pre-constructed drone query information group and the two-dimensional feature information sequence.

[0072] In some embodiments, the execution subject may generate the three-dimensional feature information group at the first time according to the three-dimensional feature information group of the drone, the pre-constructed drone query information group and the two-dimensional feature information sequence. The drone query information in the drone query information group corresponds to the estimated drones in each estimated drone at the first time. Each estimated drone may be each drone estimated to appear at the first time. The drone query information may be information in the form of a vector representing the drone feature information. The drone feature information may represent feature information of various relevant features of the drone. For example, the various relevant features may include: drone position, drone attitude features.

[0073] In practice, the execution entity may generate the three-dimensional feature information group at the first time by the following steps:

[0074] The first step is to combine the above-mentioned drone three-dimensional feature information group and the above-mentioned drone query information group to generate a combined information group.

[0075] In the second step, the combined information group is input into the first attention mechanism model to generate attention mechanism information. The first attention mechanism model may be a pre-trained neural network model based on the attention mechanism. In practice, the first attention mechanism model may be a Transformer model.

[0076] The third step is to transform the two-dimensional feature information sequence into a three-dimensional space dimension to generate three-dimensional feature information as candidate three-dimensional feature information. The three-dimensional feature conversion model acquired in advance can be used to transform the two-dimensional feature information sequence into a three-dimensional space dimension to generate three-dimensional feature information as candidate three-dimensional feature information.

[0077] In the fourth step, the candidate three-dimensional feature information and the attention mechanism information are input into the second attention mechanism model to generate a three-dimensional feature information group. The second attention mechanism model may be a pre-trained neural network model based on the attention mechanism. For example, the second attention mechanism model may be a Transformer model.

[0078] Step 108: Generate the drone detection information corresponding to the first time according to the three-dimensional feature information group.

[0079] In some embodiments, the execution entity may generate the drone detection information corresponding to the first time based on the three-dimensional feature information group.

[0080] In practice, the above-mentioned execution entity can generate the drone detection information corresponding to the above-mentioned first time through the following steps:

[0081] In the first step, each frame of the video image in the above environment video is denoised to generate a denoised video image and obtain a denoised video image sequence.

[0082] In the second step, the denoised video image sequence is subjected to target detection by a pre-trained target detection model to generate a target detection image group. The target detection image includes a target selection frame. The target detection model can be a pre-trained neural network model that takes the denoised video image as input and the target detection image as output. For example, the target detection model can be a YOLOv5 model.

[0083] The third step is to extract features of the image area within the target selection frame of each target detection image in the target detection image group to obtain the target image area features. Feature extraction: Extract features of the image area within the target selection frame and use a convolutional neural network (CNN) to extract the spatial features of the drone, such as shape, texture, color, etc.

[0084] In the fourth step, each target image region feature is classified by a pre-trained target classification detection model to generate a feature classification detection result and obtain a feature classification detection result group. The target classification detection model can be a pre-trained neural network model that takes the target image region feature as input and takes the feature classification detection result as output. For example, the target classification detection model can be a 3D CNN model or a CNN+LSTM model. The feature classification detection result can characterize the type of the drone.

[0085] In the fifth step, the drone detection information corresponding to the first time is generated according to the three-dimensional feature information group and the feature classification detection result group. As an example, first, the execution subject can determine the similarity between the historical three-dimensional feature information in the historical three-dimensional feature information group and the three-dimensional feature information group to obtain a similarity set. Then, the historical three-dimensional feature information with a corresponding similarity greater than a preset similarity is selected from the historical three-dimensional feature information group as the target three-dimensional feature information to obtain at least one target three-dimensional feature information. The at least one target three-dimensional feature information is spliced ​​with the corresponding three-dimensional feature information with a similarity association relationship to generate a spliced ​​three-dimensional feature information group. Finally, the spliced ​​three-dimensional feature information group is input into the regression model and the classification model to output regression information and classification information. Among them, the regression information can be the position information of the corresponding drone. The classification information can be the corresponding drone confidence. Afterwards, the regression information and the classification information are integrated with the corresponding drone information to generate the first drone detection information. Finally, the first drone detection information and the feature classification detection result group are merged into the drone detection information.

[0086] Further references Figure 2As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a drone automatic identification device. These drone automatic identification device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the drone automatic identification device can be specifically applied to various electronic devices.

[0087] like Figure 2 As shown, some embodiments of the automatic identification device 200 of drones include: a preprocessing unit 201, an extraction unit 202, an input unit 203, a control unit 204, a first generation unit 205, a second generation unit 206, a third generation unit 207 and a fourth generation unit 208. The preprocessing unit 201 is configured to, in response to determining that the radio detection device detects a radio signal, preprocess the radio signal to generate preprocessed radio spectrum data; the extraction unit 202 is configured to extract the radio signal features in the preprocessed radio spectrum data; the input unit 203 is configured to input the radio signal features into a pre-trained drone signal identification model to obtain a drone signal identification result; the control unit 204 is configured to, in response to determining that the drone signal identification result represents the drone signal, control the high-definition camera associated with the radio detection device to capture the surrounding environment video at the first time in real time. ; The first generating unit 205 is configured to generate two-dimensional feature information corresponding to each frame of video image in the above-mentioned environmental video to obtain a two-dimensional feature information sequence; the second generating unit 206 is configured to generate a drone three-dimensional feature information group corresponding to the drone detection object set at the above-mentioned first time according to the historical three-dimensional environmental feature information group; the third generating unit 207 is configured to generate a three-dimensional feature information group at the above-mentioned first time according to the above-mentioned drone three-dimensional feature information group, a pre-constructed drone query information group and the above-mentioned two-dimensional feature information sequence; the fourth generating unit 208 is configured to generate the drone detection information corresponding to the above-mentioned first time according to the above-mentioned three-dimensional feature information group.

[0088] It is understandable that the units recorded in the drone automatic identification device 200 are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the drone automatic identification device 200 and the units contained therein, and will not be repeated here.

[0089] Reference below Figure 3, which shows a schematic diagram of the structure of an electronic device 300 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0090] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0091] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0092] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0093] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0094] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0095] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to determining that the radio detection device detects a radio signal, preprocess the radio signal to generate preprocessed radio spectrum data; extract radio signal features in the preprocessed radio spectrum data; input the radio signal features into a pre-trained drone signal recognition model to obtain a drone signal recognition result; in response to determining that the drone signal recognition result represents the drone signal, control the high-definition camera associated with the radio detection device to capture the environment video of the surrounding environment at the first time in real time; generate two-dimensional feature information corresponding to each frame of the video image in the environment video to obtain a two-dimensional feature information sequence; generate a drone three-dimensional feature information group corresponding to the drone detection object set at the first time according to the historical three-dimensional environment feature information group; generate a three-dimensional feature information group at the first time according to the drone three-dimensional feature information group, the pre-constructed drone query information group and the two-dimensional feature information sequence; generate the drone detection information corresponding to the first time according to the three-dimensional feature information group.

[0096] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be arranged in a processor, for example, may be described as: a processor comprising: a preprocessing unit, an extraction unit, an input unit, a control unit, a first generating unit, a second generating unit, a third generating unit and a fourth generating unit. The names of these units do not constitute limitations on the units themselves in certain circumstances. For example, the preprocessing unit may also be described as "a unit that preprocesses the above-mentioned radio signal in response to determining that the radio detection device detects a radio signal to generate preprocessed radio spectrum data".

[0099] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0100] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for automatically identifying a drone, comprising: In response to determining that the radio detection device detects a radio signal, preprocessing the radio signal to generate preprocessed radio spectrum data; extracting radio signal features from the preprocessed radio spectrum data; Inputting the radio signal features into a pre-trained drone signal recognition model to obtain a drone signal recognition result; In response to determining that the drone signal identification result represents the drone signal, controlling a high-definition camera device associated with the radio detection device to capture a video of the surrounding environment in real time at the first instant; Generate two-dimensional feature information corresponding to each frame of video image in the environmental video to obtain a two-dimensional feature information sequence; Generate a drone three-dimensional feature information group corresponding to the drone detection object set at the first time according to the historical three-dimensional environment feature information group; Generate a three-dimensional feature information group at the first time according to the three-dimensional feature information group of the drone, the pre-constructed drone query information group and the two-dimensional feature information sequence; generating drone detection information corresponding to the first time according to the three-dimensional feature information group; The step of generating the three-dimensional feature information group at the first time according to the three-dimensional feature information group of the drone, the pre-constructed drone query information group and the two-dimensional feature information sequence includes: Combining the drone three-dimensional feature information group and the drone query information group to generate a combined information group; Inputting the combined information group into a first attention mechanism model to generate attention mechanism information; Converting the two-dimensional feature information sequence into a three-dimensional space to generate three-dimensional feature information as candidate three-dimensional feature information; Inputting the candidate three-dimensional feature information and the attention mechanism information into a second attention mechanism model to generate a three-dimensional feature information group; The step of generating the drone detection information corresponding to the first time according to the three-dimensional feature information group includes: Performing denoising processing on each frame of video image in the environmental video to generate a denoised video image and obtain a denoised video image sequence; Performing target detection on the denoised video image sequence using a pre-trained target detection model to generate a target detection image group, wherein the target detection image includes a target selection frame; For each target detection image in the target detection image group, feature extraction is performed on an image region within a target selection frame in the target detection image to obtain target image region features; Using a pre-trained target classification detection model, each target image region feature is classified to generate a feature classification detection result, thereby obtaining a feature classification detection result group; The drone detection information corresponding to the first time is generated according to the three-dimensional feature information group and the feature classification detection result group.

2. The method according to claim 1, wherein: Before inputting the radio signal feature into a pre-trained drone signal recognition model to obtain a drone signal recognition result, the method further includes: Obtain a drone radio signal spectrum group set, wherein a drone radio signal spectrum group in the drone radio signal spectrum group set corresponds to a drone radio signal type; Labeling each drone radio signal spectrum in the drone radio signal spectrum set to obtain a drone radio signal spectrum set with labels as a drone radio signal spectrum sample set; Determine an initial UAV signal recognition model, wherein the initial UAV signal recognition model includes: an initial signal feature extraction network, an initial multi-level signal feature hybrid autoencoder network, and an initial signal recognition network; For each drone radio signal spectrum sample group in the drone radio signal spectrum sample group set, selecting a target drone radio signal spectrum sample from the drone radio signal spectrum sample group; The initial drone signal recognition model is trained according to the radio signal spectrum samples of each target drone to obtain a trained drone signal recognition model.

3. The method according to claim 2, wherein: The initial drone signal recognition model is trained according to the spectrum samples of the radio signals of each target drone to obtain a trained drone signal recognition model, including: For each of the target drone radio signal spectrum samples in each target drone radio signal spectrum sample, perform the following training steps: Inputting the sample UAV radio signal spectrum included in the target UAV radio signal spectrum sample into the initial signal feature extraction network to obtain initial UAV radio signal feature information; Inputting the initial UAV radio signal feature information into the initial multi-level signal feature hybrid autoencoder network to obtain the initial multi-level UAV radio signal feature information; Inputting the initial multi-level UAV radio signal feature information into the initial signal recognition network to obtain the initial UAV radio signal recognition result; Based on a preset loss function, determine a signal loss value between an initial drone radio signal recognition result and a sample label included in a target drone radio signal spectrum sample; In response to determining that the sum of the signal loss values ​​is less than or equal to a preset loss value, the initial drone signal recognition model is determined to be a trained drone signal recognition model.

4. The method according to claim 2, wherein: The step of determining the initial UAV signal recognition model comprises: Determine an initial signal feature extraction network; Determine an initial multi-level signal feature hybrid autoencoding network, wherein the initial multi-level signal feature hybrid autoencoding network includes: an initial encoding network and an initial decoding network, the initial encoding network includes: an initial signal feature decoupling network and an initial signal feature fusion network, the initial signal feature decoupling network includes: an initial static signal feature decoupling network, an initial dynamic signal feature decoupling network, and the initial signal feature fusion network includes: an initial fusion network and an initial reparameterization network; Determine the initial signal recognition network; The initial signal feature extraction network, the initial multi-level signal feature hybrid autoencoder network, and the initial signal recognition network are fused to generate an initial UAV signal recognition model.

5. A drone automatic identification device, applied to the method according to claim 1, comprising: a preprocessing unit configured to, in response to determining that the radio detection device detects a radio signal, preprocess the radio signal to generate preprocessed radio spectrum data; an extraction unit configured to extract radio signal features from the pre-processed radio spectrum data; An input unit is configured to input the radio signal feature into a pre-trained drone signal recognition model to obtain a drone signal recognition result; a control unit configured to, in response to determining that the drone signal identification result represents a drone signal, control a high-definition camera device associated with the radio detection device to capture a video of the surrounding environment in real time at a first time; A first generating unit is configured to generate two-dimensional feature information corresponding to each frame of video image in the environment video to obtain a two-dimensional feature information sequence; The second generating unit is configured to generate a drone three-dimensional feature information group corresponding to the drone detection object set at the first time according to the historical three-dimensional environment feature information group; A third generating unit is configured to generate a three-dimensional feature information group at the first time according to the drone three-dimensional feature information group, the pre-constructed drone query information group and the two-dimensional feature information sequence; The fourth generating unit is configured to generate drone detection information corresponding to the first time according to the three-dimensional feature information group.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images