Method, device, equipment, storage medium and program product for identifying unmanned aerial vehicle

Abstract

The application discloses a method, device and equipment for identifying a UAV, a storage medium and a program product, and is applied to a general base station. The technical scheme comprises the following steps: acquiring a time domain signal waveform, a spherical waveform and a visual waveform of a flying object respectively, the time domain signal waveform is used for describing the motion characteristics of the flying object, the spherical waveform is used for describing the geometric characteristics of the flying object, and the visual waveform is used for describing the visual characteristics of the flying object; splicing the time domain signal waveform, the spherical waveform and the visual waveform to obtain a spliced waveform; inputting the spliced waveform into a unified identification model to obtain the probability that the flying object is a UAV; and determining that the flying object is a UAV when the probability is greater than a preset threshold. In this way, the flying object can be identified by comprehensively using multi-modal data, so that the identification accuracy is ensured.

Description

Technical Field

[0001] This application belongs to the field of drone identification technology, and in particular relates to a method, device, equipment, storage medium and program product for drone identification. Background Technology

[0002] In low-altitude drone identification, the dynamic characteristics of the drone can usually be collected by radar or radio signals, and then the target image can be collected by camera. The drone can be identified by combining the dynamic characteristics and the target image with a deep learning model.

[0003] However, in complex environments such as urban airspace or natural areas, the signals and images collected by drones are easily interfered with by birds and other objects, resulting in low drone recognition accuracy. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and program product for drone identification, which can identify flying objects by integrating multimodal data, thereby ensuring identification accuracy.

[0005] In a first aspect, embodiments of this application provide a method for identifying unmanned aerial vehicles (UAVs), applied to a sensing base station, the method comprising: The time-domain signal waveform, spherical waveform, and visual waveform of the flying object are acquired respectively. The time-domain signal waveform is used to describe the motion characteristics of the flying object, the spherical waveform is used to describe the geometric characteristics of the flying object, and the visual waveform is used to describe the visual characteristics of the flying object. The time-domain signal waveform, the spherical waveform, and the visual waveform are spliced ​​together to obtain a spliced ​​waveform. The spliced ​​waveform is input into a unified recognition model to obtain the probability that the flying object is a drone; If the probability is greater than a preset threshold, the flying object is determined to be a drone.

[0006] In one possible implementation, acquiring the time-domain signal waveform, the spherical waveform, and the visual waveform respectively includes: Acquire motion data, point cloud data, and appearance images of the flying object; The time-domain signal waveform is generated based on the motion data; The spherical waveform is generated based on the point cloud data; The visual waveform is generated based on the appearance image.

[0007] In one possible implementation, generating the time-domain signal waveform based on the motion data includes: Perform a Fast Fourier Transform on the motion data to obtain the motion spectrum; The time-domain signal waveform is obtained by performing an inverse transform on the motion spectrum.

[0008] In one possible implementation, generating the spherical waveform based on the point cloud data includes: Calculate the statistical characteristics of point cloud data; The statistical features are input into the fully connected layer to obtain the feature embedding vector; The spherical waveform is generated based on the feature embedding vector.

[0009] In one possible implementation, generating the visual waveform based on the appearance image includes: The appearance image is input into the feature extractor to obtain the high-dimensional feature vector output by the feature extractor; The visual waveform is generated based on the high-dimensional feature vector.

[0010] In one possible implementation, the step of stitching together the time-domain signal waveform, the spherical waveform, and the visual waveform to obtain a stitched waveform includes: Fast Fourier transform is performed on the time-domain signal waveform, the spherical waveform, and the visual waveform respectively to obtain the first spectrum corresponding to the time-domain signal waveform, the second spectrum corresponding to the spherical waveform, and the third spectrum corresponding to the visual waveform; The first spectrum is obtained by extracting the first spectrum according to the preset low-frequency range; The second spectrum is obtained by truncating the second spectrum according to the preset intermediate frequency range; The third fused spectrum is obtained by extracting the third spectrum according to the preset high frequency range; wherein the frequency of the preset high frequency range is greater than the frequency of the preset mid frequency range, and the frequency of the preset mid frequency range is greater than the frequency of the preset low frequency range. The first fused spectrum, the second fused spectrum, and the third fused spectrum are spliced ​​together to obtain the fused spectrum; The spliced ​​waveform is obtained by performing an inverse fast Fourier transform on the fused spectrum.

[0011] In one possible implementation, the unified recognition model includes an input layer, an encoder layer, and an output layer, wherein the encoder layer includes a self-attention module and a gating unit; The step of inputting the spliced ​​waveform into a unified recognition model to obtain the probability that the flying object is a drone includes: The spliced ​​waveform is input into the input layer of the unified recognition model; The spliced ​​waveform is linearly projected through the input layer to obtain the embedding vector; The embedding vector is input into the encoder layer; The attention feature vector is calculated using the self-attention module and attention mechanism in the encoder layer. The attention feature vector is input into the gating unit; The weight coefficients of the attention feature vector are calculated using the gating unit; The weight coefficients and the attention feature vector are multiplied element-wise to obtain the updated feature vector; The weight coefficients of the updated feature vector are used as the input data for the next self-attention module, until the feature vector output by the last self-attention module is reached; The feature vector is input into the output layer to obtain the probability that the flying object is a drone.

[0012] In one possible implementation, inputting the embedding vector into the encoder layer includes: The embedding vector is input into the residual network to obtain the residual result; The sum of the residual result and the embedding vector is calculated to obtain the first sum. The first sum value is input into the encoder layer.

[0013] In one possible implementation, before inputting the spliced ​​waveform into a unified recognition model to obtain the probability that the flying object is a drone, the method further includes: Obtain training waveform samples; The training waveform samples are input into the neural network model to obtain the output of the neural network; Based on the output results and the training waveform samples, calculate the overall loss function and the sparse loss function corresponding to the gating unit; The network parameters of the neural network model are adjusted according to the overall loss function and the sparse loss function to obtain the unified recognition model.

[0014] Secondly, embodiments of this application provide a device for drone identification, applied to a sensing base station, the device comprising: The acquisition module is used to acquire the time-domain signal waveform, spherical waveform, and visual waveform of the flying object, respectively. The time-domain signal waveform is used to describe the motion characteristics of the flying object, the spherical waveform is used to describe the geometric characteristics of the flying object, and the visual waveform is used to describe the visual characteristics of the flying object. The splicing module is used to splice the time-domain signal waveform, the spherical waveform, and the visual waveform to obtain a spliced ​​waveform; The prediction module is used to input the spliced ​​waveform into a unified recognition model to obtain the probability that the flying object is a drone; The determination module is used to determine that the flying object is a drone when the probability is greater than a preset threshold.

[0015] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions; The method for drone identification, as described in the first aspect, is implemented when the processor executes computer program instructions.

[0016] Fourthly, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method for drone identification as described in the first aspect is implemented.

[0017] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a method for identifying unmanned aerial vehicles as described in the first aspect.

[0018] This application provides a method, apparatus, device, storage medium, and program product for drone identification. A sensing base station acquires a time-domain signal waveform describing the motion characteristics of an object, a spherical waveform describing the aggregate features of the object, and a visual waveform describing the visual features of the object. These waveforms are then stitched together to obtain a stitched waveform. The stitched waveform comprehensively reflects the motion, geometric, and visual features of the object. A unified identification model is then used to identify whether the object is a drone based on the stitched waveform. Thus, by stitching together waveforms from multiple modalities, multimodal data can be used to identify the object, ensuring accurate identification. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a method for drone identification provided in an embodiment of this application; Figure 2 This is a schematic flowchart of a waveform signal generation method provided in an embodiment of this application; Figure 3 This is an exemplary schematic diagram of a time-domain signal waveform provided in an embodiment of this application; Figure 4 This is an exemplary schematic diagram of a spherical waveform provided in an embodiment of this application; Figure 5This is an exemplary schematic diagram of a visual waveform provided in an embodiment of this application; Figure 6 This is a flowchart illustrating a method for generating spliced ​​waveforms according to an embodiment of this application; Figure 7 This is a flowchart illustrating the specific implementation of S103 in the drone identification method provided in this application embodiment; Figure 8 This is an exemplary schematic diagram of the gate control unit provided in the embodiments of this application; Figure 9 This is a schematic flowchart of a model training method provided in an embodiment of this application; Figure 10 This is an exemplary schematic diagram of a method for drone identification provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of a drone identification device provided in an embodiment of this application; Figure 12 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0023] To address the problems in the prior art, embodiments of this application provide a method, apparatus, device, storage medium, and program product for drone identification. Figure 1A flowchart illustrating a drone identification method according to an embodiment of this application is shown. This drone identification method can be applied to a sensing base station or a sensing functional unit, such as... Figure 1 As shown, the method includes: S101. Acquire the time-domain signal waveform, spherical waveform, and visual waveform of the flying object respectively.

[0024] Among them, the time-domain signal waveform is used to describe the motion characteristics of the flying object, the spherical waveform is used to describe the geometric characteristics of the flying object, and the visual waveform is used to describe the visual characteristics of the flying object.

[0025] The motion characteristics of an object are determined based on its motion data. For example, motion data can be flight trajectory data, velocity data, or acceleration data, etc., and this application does not limit this.

[0026] Geometric features are used to describe the structure of an object and can be determined using point cloud data. Visual waveforms are determined from images of the object's appearance.

[0027] S102. The time-domain signal waveform, spherical waveform, and visual waveform are spliced ​​together to obtain the spliced ​​waveform.

[0028] Among them, the stitched waveform can comprehensively reflect the motion characteristics, geometric characteristics and visual characteristics of the flying object. Subsequent use of the stitched waveform for UAV identification can improve the accuracy of the identification results.

[0029] S103. Input the spliced ​​waveform into the unified recognition model to obtain the probability that the flying object is a drone.

[0030] The unified recognition model is obtained by training the Transformer model.

[0031] S104. If the probability is greater than a preset threshold, determine that the flying object is a drone.

[0032] The preset threshold is set based on experience. In actual implementation, the preset threshold can be 50%. When the probability is greater than or equal to 50%, the flying object is determined to be a drone; when the probability is less than 50%, the flying object is determined not to be a drone.

[0033] Using the method provided in this application, the sensing base station acquires a time-domain signal waveform describing the motion characteristics of an object, a spherical waveform describing the aggregate characteristics of the object, and a visual waveform describing the visual characteristics of the object. Then, the time-domain signal waveform, the spherical waveform, and the visual waveform are stitched together to obtain a stitched waveform. This stitched waveform comprehensively reflects the motion characteristics, geometric characteristics, and visual characteristics of the object. A unified recognition model is then used to identify whether the object is a drone based on the stitched waveform. Thus, by stitching together waveforms from multiple modalities, multimodal data can be comprehensively used to identify the object, thereby ensuring recognition accuracy.

[0034] In some embodiments of this application, the acquisition of the time-domain signal waveform, spherical waveform, and visual waveform of the flying object in S101 can be specifically implemented as S1011-S1014, such as... Figure 2 As shown: S1011. Acquire motion data, point cloud data, and appearance images of the flying object.

[0035] In one example, motion data could be time-series data such as flight trajectory, velocity, and acceleration collected by radar. Point cloud data is obtained through radar or ultrasonic sensors. Appearance images are obtained through cameras.

[0036] S1012. Generate time-domain signal waveforms based on motion data.

[0037] Specifically, a fast Fourier transform is performed on the motion data to obtain the motion spectrum; an inverse transform is performed on the motion spectrum to obtain the time-domain signal waveform.

[0038] Specifically, based on the data acquisition time of the motion data, a time series of motion data is constructed, and then a fast Fourier transform is performed on the time series data to obtain the motion spectrum.

[0039] The Fast Fourier Transform can be expressed as: in, Represents the motion spectrum. This represents the nth motion data point.

[0040] Correspondingly, the inverse transform can be expressed as: In one example, the time-domain signal waveform is as follows: Figure 3 As shown, Figure 3The example illustrates the time-domain signal waveforms corresponding to UAV and bird characteristics. The vertical axis represents the characteristic amplitude, and the horizontal axis represents the data sampling points. Since UAVs are mechanically driven, their corresponding time-domain signal waveforms typically exhibit stable dominant harmonics; therefore, the time-domain signal waveforms corresponding to UAV characteristics are stable sine waves. However, birds possess biological flight patterns, resulting in a more complex and variable spectrum of their dynamic characteristics; consequently, the time-domain signal waveforms corresponding to bird characteristics are unstable sine waves.

[0041] S1013. Generate a spherical waveform based on point cloud data.

[0042] The process involves calculating the statistical features of the point cloud data; inputting the statistical features into a fully connected layer to obtain a feature embedding vector; and generating a spherical waveform based on the feature embedding vector.

[0043] Specifically, the geometric center point of the point cloud data is calculated. For each point cloud, the distance from the point cloud to the geometric center point is calculated. Then, based on the distance from each point cloud to the geometric center point, statistical characteristics are obtained, which are the variance, mean, and deviation of the distance.

[0044] Among them, the distance data is The mean is The variance is .

[0045] The dimension of the feature embedding vector is determined by the fully connected layer, which is pre-trained based on experience. The parameter of each dimension in the feature embedding vector represents the amplitude of the spherical waveform. ,frequency and phase .

[0046] Based on the above amplitude ,frequency and phase The generated spherical waveform can be represented as: .

[0047] In one example, such as Figure 4 As shown, Figure 4 The diagram illustrates spherical waveforms corresponding to drone and bird features. The vertical axis represents the feature amplitude, and the horizontal axis represents the data sampling points. Drones have a relatively regular shape; therefore, their geometric features will produce a low-variance, highly periodic sine wave. Birds, due to their irregular flight patterns, will produce a high-variance, chaotic waveform based on their geometric features.

[0048] S1014. Generate a visual waveform based on the appearance image.

[0049] The process involves inputting the appearance image into a feature extractor to obtain a high-dimensional feature vector output by the feature extractor; and generating a visual waveform based on the high-dimensional feature vector.

[0050] Specifically, the visual waveform can be represented as: .in, This represents a high-dimensional feature vector.

[0051] In one example, such as Figure 5 As shown, Figure 5 The diagram illustrates the visual waveforms corresponding to drone and bird features. The vertical axis represents the feature amplitude, and the horizontal axis represents the data sampling points. The visual waveform reflects the differences in color patches at different locations in the acquired appearance image. Due to its symmetrical mechanical structure, the texture of the propeller of a drone is relatively symmetrical and regular; therefore, the visual waveform corresponding to a drone also exhibits strong symmetry. In contrast, the visual waveform corresponding to a bird is more chaotic due to the more complex details of its feathers.

[0052] Using the method provided in this application, after acquiring the motion data, point cloud data, and appearance image of the flying object, a Fast Fourier Transform and Inverse Fourier Transform are performed on the motion data to convert it into a signal waveform. Feature extraction is performed on the statistical features of the point cloud data to convert it into a spherical waveform. Feature extraction is performed on the visual image to convert it into a visual waveform. In this way, the motion data, point cloud data, and appearance image can be converted into corresponding waveforms, facilitating subsequent processing by a unified recognition model.

[0053] Based on the calculated time-domain signal waveform, spherical waveform, and visual waveform, the three waveforms are fused to obtain a spliced ​​waveform. Specifically, for S102 above, the time-domain signal waveform, spherical waveform, and visual waveform are spliced ​​to obtain a spliced ​​waveform, as shown below. Figure 6 As shown, it can be implemented as follows: S1021. Perform Fast Fourier Transform on the time-domain signal waveform, the spherical waveform, and the visual waveform respectively to obtain the first spectrum corresponding to the time-domain signal waveform, the second spectrum corresponding to the spherical waveform, and the third spectrum corresponding to the visual waveform.

[0054] S1022. Extract the first spectrum according to the preset low-frequency range to obtain the first fused spectrum.

[0055] The preset low-frequency range is set based on experience. In one example, the preset low-frequency range can be 0-10Hz.

[0056] S1023. Extract the second spectrum according to the preset intermediate frequency range to obtain the second fused spectrum.

[0057] The preset intermediate frequency range is set based on experience. In one example, the preset intermediate frequency range can be 10-40Hz.

[0058] S1024. Extract the third spectrum according to the preset high-frequency range to obtain the third fused spectrum.

[0059] The preset high-frequency range is set based on experience. In one example, the preset high-frequency range could be 40-50Hz.

[0060] Among them, the frequency of the preset high frequency range is greater than the frequency of the preset mid frequency range, and the frequency of the preset mid frequency range is greater than the frequency of the preset low frequency range.

[0061] S1025. The first fused spectrum, the second fused spectrum, and the third fused spectrum are spliced ​​together to obtain the fused spectrum.

[0062] Understandably, the low-frequency spectrum better reflects the motion characteristics of flying objects, the mid-frequency spectrum has higher resolution for geometric features, and the high-frequency spectrum can carry richer visual feature details. Therefore, by stitching together the first, second, and third fused spectra, the motion characteristics, visual feature details, and geometric features of flying objects can be reflected in the fused spectrum, thus ensuring the accuracy of subsequent recognition results.

[0063] S1026. Perform a fast inverse Fourier transform on the fused spectrum to obtain the spliced ​​waveform.

[0064] Using the method provided in this application, after performing Fast Fourier Transform on the time-domain signal waveform, spherical waveform, and visual waveform respectively, the corresponding spectra can be obtained. Therefore, the spectrum corresponding to each waveform can be truncated according to preset flat-path intervals: a first fused spectrum is obtained by truncating the first spectrum according to a preset low-frequency interval; a second fused spectrum is obtained by truncating the second spectrum according to a preset mid-frequency interval; and a third fused spectrum is obtained by truncating the third spectrum according to a preset high-frequency interval. Then, the truncated spectra are stitched together according to the corresponding frequency intervals. Thus, the stitched waveform can be obtained by performing an inverse transform on the stitched spectrum. In this way, the stitched waveform integrates information on the three key physical attributes—dynamic features, geometric features, and visual features—with high fidelity, thereby improving the accuracy of subsequent recognition results.

[0065] In some embodiments of this application, the unified identification model includes an input layer, an encoder layer, and an output layer, wherein the encoder layer includes a self-attention module and a gating unit. Accordingly, in S103 above, the spliced ​​waveform is input into the unified identification model to obtain the probability that the flying object is a drone, such as... Figure 7 As shown, this can be implemented as S1031-S1039: S1031. Input the spliced ​​waveform into the input layer of the unified recognition model.

[0066] The input layer uniformly divides the spliced ​​waveform into multiple segments. The specific number of segments depends on the number of self-attention modules in the encoder layer. For example, if the self-attention module is an 8-head multi-head self-attention module, then the number of segments is 8.

[0067] S1032. The spliced ​​waveform is linearly projected through the input layer to obtain the embedding vector.

[0068] Specifically, the input layer performs linear projection processing on each slice to obtain the sequence. This sequence is the embedding vector.

[0069] S1033, Input the embedded vector into the encoder layer.

[0070] The encoder layer includes multiple encoders, each with the same structure, and each encoder is a multi-head self-attention module with the same structure.

[0071] S1034. The attention feature vector is calculated through the self-attention module and attention mechanism in the encoder layer.

[0072] Specifically, for each self-attention module in the encoder layer, the embedding vector corresponding to each slice is input into the corresponding self-attention module. Each self-attention module processes the embedding vector corresponding to one slice. The output of the self-attention module is: ,in, . To query the transformation matrix, The key transformation matrix is... The value transformation matrix, Let be the attention weights of the feature vectors of the i-th segment.

[0073] S1035. Input the attention feature vector into the gating unit.

[0074] The gating unit is an Adaptive Dynamic Gating Unit (ADU), which generates a weighting coefficient between 0 and 1. This weighting coefficient is used to control the flow of information and determine which information is retained, enhanced, suppressed, or discarded.

[0075] Specifically, the gating unit is a trainable linearly transformed fully connected layer.

[0076] S1036. Calculate the weight coefficients of the attention feature vector through the gating unit.

[0077] S1037. Multiply the weight coefficients and attention feature vectors element by element to obtain the updated feature vector.

[0078] In one example, assuming the visual image is a rainy day image, the gating unit can reduce the weight coefficient of the image content containing raindrops in the rainy day image, thereby filtering out the rainy background in the rainy day image and retaining the image content of the flying object.

[0079] Understandably, the gating unit can determine which features in the attention feature vector are retained and which are discarded, thereby obtaining an updated feature vector. This allows the network to adaptively adjust the information flow without the need for costly mean and variance calculations, thus saving computational resources, improving computational efficiency, and enhancing computational accuracy.

[0080] S1038. Update the feature vector and weight coefficients as input data for the next self-attention module, until the feature vector output by the last self-attention module.

[0081] S1039. Input the feature vector into the output layer to obtain the probability that the flying object is a drone.

[0082] The unified recognition model, using the method provided in this application, includes an input layer, an encoder layer, and an output layer. The encoder layer includes a self-attention module and a gating unit. After inputting the spliced ​​waveform into the input layer, the spliced ​​waveform can be processed into an embedding vector. Then, the self-attention module of the encoder layer processes the embedding vector based on an attention mechanism to obtain an attention feature vector. Next, the gating unit dynamically adjusts the weight coefficients of the attention feature vector so that the unified recognition model can focus on features that can accurately identify flying objects, thereby ensuring the accuracy of subsequent recognition results.

[0083] In one example, the normalization layer and activation function layer in the traditional Transformer network are replaced with gating units, such as... Figure 8 As shown, Figure 8 The image on the left shows a traditional Transfoemer network. Before the self-attention module is a normalization layer (Layer Norm), which includes a normalization processing network (normalize) and a linear transformation network (scale & shift). Figure 8 The right side shows the Transfoemer network with added gating units. Before each self-attention module, the traditional normalization layer is replaced with a gating unit based on the DyT algorithm, which includes content branches and a linear transformation network.

[0084] Specifically, the gate unit based on the DyT algorithm adjusts the weight coefficients using the following formula: in, This indicates updating the feature vector. This indicates a gated branch used to generate weight coefficients. This represents a content branch, used to perform non-linear transformations and scaling on the attention features of the input. This indicates element-wise multiplication.

[0085] In some embodiments of this application, regarding the above-mentioned S1033, embedding the vector input encoder layer, it can be specifically implemented as follows: Step 1: Input the embedding vector into the residual network to obtain the residual results.

[0086] Step 2: Calculate the sum of the residual result and the embedding vector to obtain the first sum.

[0087] The first sum can be expressed as: in, Indicates the first sum value. Represents the residual result. This represents the embedding vector.

[0088] Step 3: Input the first sum into the encoder layer.

[0089] By using the method provided in the embodiments of this application, the embedded vector is added to the backbone network through residual connections, which enhances the model's sensitivity to dynamic changes and further improves the accuracy of the recognition results.

[0090] The following combination Figure 9 The training process of the unified recognition model described above is introduced, such as... Figure 5 As shown, the method includes: S901. Obtain training waveform samples.

[0091] The training waveform samples include positive and negative samples. The positive samples are the stitched waveforms corresponding to the UAV, and the negative samples are the stitched waveforms corresponding to other flying objects.

[0092] S902. Input the training waveform samples into the neural network model to obtain the output of the neural network.

[0093] The neural network model used is a Transformer model, with the normalization layer and activation function layer replaced by gating units. This makes the training process of the Transformer model more stable and subsequent computations more efficient.

[0094] S903. Based on the output results and training waveform samples, calculate the overall loss function and the sparse loss function corresponding to the gating unit.

[0095] The sparse loss function of the gated unit can be expressed as: in, This represents the sparse loss function.

[0096] S904. Adjust the network parameters of the neural network model according to the overall loss function and the sparse loss function to obtain a unified recognition model.

[0097] The method provided in this application involves obtaining training waveform samples and then inputting them into a neural network model for training. The overall loss function of the neural network model and the sparse loss function corresponding to each gating unit are calculated based on the output of the neural network and the training waveform samples. This allows for the training of newly added gating units in the neural network model, thereby improving the accuracy of the model training results.

[0098] The following combination Figure 10 This application introduces a method for drone identification provided in its embodiments, such as... Figure 10 As shown, the method includes: S1001, Data Acquisition.

[0099] Motion data is collected via radar, point cloud data is acquired via radar or ultrasonic sensors, and appearance images are captured via cameras.

[0100] S1002, Generate a spherical waveform.

[0101] S1003, Generate time-domain signal waveform.

[0102] S1004, Generate visual waveform.

[0103] Specifically, the methods for generating spherical waveforms, time-domain signal waveforms, and visual waveforms are described in the relevant embodiments above, and will not be repeated here.

[0104] S1005. Perform frequency domain transformation to obtain their respective spectra.

[0105] Among them, frequency domain transformation refers to applying Fast Fourier Transform to different waveforms to obtain the first spectrum, the second spectrum, and the third spectrum.

[0106] S1006. Use a frequency interleaving algorithm to extract different spectra.

[0107] The frequency interleaving algorithm refers to the process of extracting the corresponding spectrum using different frequency ranges, that is, extracting the first, second and third spectra according to different frequency ranges and then splicing them together to form a fused spectrum.

[0108] S1007, Generate spliced ​​waveform.

[0109] After generating the fused spectrum as described above, performing an inverse Fourier transform on the fused spectrum yields the spliced ​​waveform.

[0110] S1008. Import the spliced ​​waveform into the input layer.

[0111] The input layer slices the spliced ​​waveform to obtain multiple slices, and then linearly projects each slice into a feature vector. By combining the feature vectors of each slice in sequence, an embedding vector is obtained. This embedding vector is then input into the encoder layer.

[0112] S1009, Input encoder layer.

[0113] S1010, assigned to each encoder.

[0114] Among them, the sensing base station can be assigned to different multi-head self-attention modules in sequence according to the order of the feature vectors in the embedding vector.

[0115] S1011, Input multi-head self-attention module.

[0116] S1012, The DyT algorithm based on ADU is adopted.

[0117] Specifically, the DyT algorithm based on ADU is used to control the attention feature vector output by the multi-head self-attention module and generate corresponding weight coefficients.

[0118] S1013, Input modal gating attention layer.

[0119] In this process, the weight coefficients are calculated and then multiplied element-wise with the attention feature vector to obtain the updated feature vector.

[0120] S1014, Input feedforward network.

[0121] S1015. The DyT algorithm based on ADU is adopted.

[0122] S1016. Add the original feature vector through residual connection.

[0123] S1017, Input / Output Layer.

[0124] S1018, obtain the output probability.

[0125] The method provided in this application replaces the normalization layer and activation function layer in the Transformer model with a gating unit. This allows for the selection of attention feature vectors output by the self-attention module, thereby reducing computational overhead and improving model training efficiency and stability. Furthermore, by fusing three waveforms—time-domain signal waveform, spherical waveform, and visual waveform—a stitched waveform is obtained, achieving collaborative recognition of multi-dimensional information. A frequency interleaving algorithm ensures that features from different modalities do not interfere with each other during waveform fusion, guaranteeing the information density of the stitched waveform and carrying more information, thus improving the accuracy of the recognition results.

[0126] Based on the same concept, embodiments of this application also provide a device for drone identification, applied to a sensing base station, such as... Figure 11 As shown, the device includes: The acquisition module 1101 is used to acquire the time-domain signal waveform, spherical waveform, and visual waveform of the flying object, respectively. The time-domain signal waveform is used to describe the motion characteristics of the flying object, the spherical waveform is used to describe the geometric characteristics of the flying object, and the visual waveform is used to describe the visual characteristics of the flying object. The splicing module 1102 is used to splice the time-domain signal waveform, the spherical waveform, and the visual waveform to obtain a spliced ​​waveform; The prediction module 1103 is used to input the spliced ​​waveform into a unified recognition model to obtain the probability that the flying object is a drone; The determination module 1104 is used to determine that the flying object is a drone when the probability is greater than a preset threshold.

[0127] In one possible implementation, module 1101 is specifically used for: Acquire motion data, point cloud data, and appearance images of the flying object; The time-domain signal waveform is generated based on the motion data; The spherical waveform is generated based on the point cloud data; The visual waveform is generated based on the appearance image.

[0128] In one possible implementation, module 1101 is specifically used for: Perform a Fast Fourier Transform on the motion data to obtain the motion spectrum; The time-domain signal waveform is obtained by performing an inverse transform on the motion spectrum.

[0129] In one possible implementation, module 1101 is specifically used for: Calculate the statistical characteristics of point cloud data; The statistical features are input into the fully connected layer to obtain the feature embedding vector; The spherical waveform is generated based on the feature embedding vector.

[0130] In one possible implementation, module 1101 is specifically used for: The appearance image is input into the feature extractor to obtain the high-dimensional feature vector output by the feature extractor; The visual waveform is generated based on the high-dimensional feature vector.

[0131] In one possible implementation, the splicing module 1102 is specifically used for: Fast Fourier transform is performed on the time-domain signal waveform, the spherical waveform, and the visual waveform respectively to obtain the first spectrum corresponding to the time-domain signal waveform, the second spectrum corresponding to the spherical waveform, and the third spectrum corresponding to the visual waveform; The first spectrum is obtained by extracting the first spectrum according to the preset low-frequency range; The second spectrum is obtained by truncating the second spectrum according to the preset intermediate frequency range; The third fused spectrum is obtained by extracting the third spectrum according to the preset high frequency range; wherein the frequency of the preset high frequency range is greater than the frequency of the preset mid frequency range, and the frequency of the preset mid frequency range is greater than the frequency of the preset low frequency range. The first fused spectrum, the second fused spectrum, and the third fused spectrum are spliced ​​together to obtain the fused spectrum; The spliced ​​waveform is obtained by performing an inverse fast Fourier transform on the fused spectrum.

[0132] In one possible implementation, the unified recognition model includes an input layer, an encoder layer, and an output layer, wherein the encoder layer includes a self-attention module and a gating unit; Prediction module 1103 is specifically used for: The spliced ​​waveform is input into the input layer of the unified recognition model; The spliced ​​waveform is linearly projected through the input layer to obtain the embedding vector; The embedding vector is input into the encoder layer; The attention feature vector is calculated using the self-attention module and attention mechanism in the encoder layer. The attention feature vector is input into the gating unit; The weight coefficients of the attention feature vector are calculated using the gating unit; The weight coefficients and the attention feature vector are multiplied element-wise to obtain the updated feature vector; The weight coefficients of the updated feature vector are used as the input data for the next self-attention module, until the feature vector output by the last self-attention module is reached; The feature vector is input into the output layer to obtain the probability that the flying object is a drone.

[0133] In one possible implementation, the prediction module 1103 is specifically used for: The embedding vector is input into the residual network to obtain the residual result; The sum of the residual result and the embedding vector is calculated to obtain the first sum. The first sum value is input into the encoder layer.

[0134] In one possible implementation, before inputting the spliced ​​waveform into a unified recognition model to obtain the probability that the flying object is a drone, the device further includes: The acquisition module 1101 is also used to acquire training waveform samples; The input module is used to input the training waveform samples into the neural network model to obtain the output result of the neural network; The calculation module is used to calculate the overall loss function and the sparse loss function corresponding to the gating unit based on the output results and the training waveform samples. The adjustment module is used to adjust the network parameters of the neural network model according to the overall loss function and the sparse loss function to obtain the unified recognition model.

[0135] It should be noted that the device for drone identification is the same as the method for drone identification described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0136] Figure 12 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0137] The terminal device may include a processor 1201 and a memory 1202 storing computer program instructions.

[0138] Specifically, the processor 1201 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0139] Memory 1202 may include mass storage for data or instructions. For example, and not limitingly, memory 1202 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1202 may include removable or non-removable (or fixed) media. Where appropriate, memory 1202 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1202 is non-volatile solid-state memory.

[0140] In certain embodiments, memory 1202 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0141] The processor 1201 implements any of the UAV identification methods described in the above embodiments by reading and executing computer program instructions stored in the memory 1202.

[0142] In one example, the terminal device may also include a communication interface 1203 and a bus 1204. For example, Figure 12 As shown, the processor 1201, memory 1202, and communication interface 1203 are connected through bus 1204 and complete communication with each other.

[0143] The communication interface 1203 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0144] Bus 1204 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1204 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0145] Furthermore, in conjunction with the drone identification methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the drone identification methods in the above embodiments.

[0146] This application also provides a computer program product, including a computer program, which, when processed and executed, implements any of the drone identification methods described in the above embodiments.

[0147] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0148] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0149] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0150] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0151] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for identifying unmanned aerial vehicles (UAVs), characterized in that, Applied to a sensing base station, the method includes: The time-domain signal waveform, spherical waveform, and visual waveform of the flying object are acquired respectively. The time-domain signal waveform is used to describe the motion characteristics of the flying object, the spherical waveform is used to describe the geometric characteristics of the flying object, and the visual waveform is used to describe the visual characteristics of the flying object. The time-domain signal waveform, the spherical waveform, and the visual waveform are spliced ​​together to obtain a spliced ​​waveform. The spliced ​​waveform is input into a unified recognition model to obtain the probability that the flying object is a drone; If the probability is greater than a preset threshold, the flying object is determined to be a drone; The process of stitching together the time-domain signal waveform, the spherical waveform, and the visual waveform to obtain a stitched waveform includes: Fast Fourier transform is performed on the time-domain signal waveform, the spherical waveform, and the visual waveform respectively to obtain the first spectrum corresponding to the time-domain signal waveform, the second spectrum corresponding to the spherical waveform, and the third spectrum corresponding to the visual waveform; The first spectrum is obtained by extracting the first spectrum according to the preset low-frequency range; The second spectrum is obtained by truncating the second spectrum according to the preset intermediate frequency range; The third fused spectrum is obtained by extracting the third spectrum according to the preset high frequency range; wherein the frequency of the preset high frequency range is greater than the frequency of the preset mid frequency range, and the frequency of the preset mid frequency range is greater than the frequency of the preset low frequency range. The first fused spectrum, the second fused spectrum, and the third fused spectrum are spliced ​​together to obtain the fused spectrum; The spliced ​​waveform is obtained by performing an inverse fast Fourier transform on the fused spectrum.

2. The method according to claim 1, characterized in that, The acquisition of time-domain signal waveforms, spherical waveforms, and visual waveforms includes: Acquire motion data, point cloud data, and appearance images of the flying object; The time-domain signal waveform is generated based on the motion data; The spherical waveform is generated based on the point cloud data; The visual waveform is generated based on the appearance image.

3. The method according to claim 2, characterized in that, The step of generating the time-domain signal waveform based on the motion data includes: Perform a Fast Fourier Transform on the motion data to obtain the motion spectrum; The time-domain signal waveform is obtained by performing an inverse transform on the motion spectrum.

4. The method according to claim 2, characterized in that, The step of generating the spherical waveform based on the point cloud data includes: Calculate the statistical characteristics of point cloud data; The statistical features are input into the fully connected layer to obtain the feature embedding vector; The spherical waveform is generated based on the feature embedding vector.

5. The method according to claim 2, characterized in that, The step of generating the visual waveform based on the appearance image includes: The appearance image is input into the feature extractor to obtain a high-dimensional feature vector output by the feature extractor; The visual waveform is generated based on the high-dimensional feature vector.

6. The method according to claim 1, characterized in that, The unified recognition model includes an input layer, an encoder layer, and an output layer. The encoder layer includes a self-attention module and a gating unit. The step of inputting the spliced ​​waveform into a unified recognition model to obtain the probability that the flying object is a drone includes: The spliced ​​waveform is input into the input layer of the unified recognition model; The spliced ​​waveform is linearly projected through the input layer to obtain the embedding vector; The embedding vector is input into the encoder layer; The attention feature vector is calculated using the self-attention module and attention mechanism in the encoder layer. The attention feature vector is input into the gating unit; The weight coefficients of the attention feature vector are calculated using the gating unit; The weight coefficients and the attention feature vector are multiplied element-wise to obtain the updated feature vector; The updated feature vector and the weight coefficients are used as input data for the next self-attention module, until the feature vector output by the last self-attention module is reached; The feature vector is input into the output layer to obtain the probability that the flying object is a drone.

7. The method according to claim 6, characterized in that, The step of inputting the embedded vector into the encoder layer includes: The embedding vector is input into the residual network to obtain the residual result; The sum of the residual result and the embedding vector is calculated to obtain the first sum; The first sum value is input into the encoder layer.

8. The method according to claim 6, characterized in that, Before inputting the spliced ​​waveform into the unified recognition model to obtain the probability that the flying object is a drone, the method further includes: Obtain training waveform samples; The training waveform samples are input into the neural network model to obtain the output of the neural network; Based on the output results and the training waveform samples, calculate the overall loss function and the sparse loss function corresponding to the gating unit; The network parameters of the neural network model are adjusted according to the overall loss function and the sparse loss function to obtain the unified recognition model.

9. A device for identifying unmanned aerial vehicles (UAVs), used to perform the method as described in claim 1, characterized in that, The device, applied to a sensing base station, includes: The acquisition module is used to acquire the time-domain signal waveform, spherical waveform, and visual waveform of the flying object, respectively. The time-domain signal waveform is used to describe the motion characteristics of the flying object, the spherical waveform is used to describe the geometric characteristics of the flying object, and the visual waveform is used to describe the visual characteristics of the flying object. The splicing module is used to splice the time-domain signal waveform, the spherical waveform, and the visual waveform to obtain a spliced ​​waveform; The prediction module is used to input the spliced ​​waveform into a unified recognition model to obtain the probability that the flying object is a drone; The determination module is used to determine that the flying object is a drone when the probability is greater than a preset threshold.

10. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the method for drone identification as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the method for drone identification as described in any one of claims 1-8.

12. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the method for drone identification as described in any one of claims 1-8.

Images