Method and device for determining obstacle avoidance mechanism of unmanned aerial vehicle and electronic equipment

Abstract

The invention relates to the technical field of unmanned aerial vehicles, in particular to a method and device for determining an obstacle avoidance mechanism of an unmanned aerial vehicle and electronic equipment. The method comprises the following steps: acquiring a radio signal when the unmanned aerial vehicle flies; the method comprises the following steps: converting a radio signal into a spectrogram through short-time Fourier transform, extracting a time sequence feature representing a time dimension sequence of the spectrogram through a long-short-term memory network, extracting a frequency spectrum feature of the spectrogram through a convolutional neural network, and carrying out weighted fusion on the time sequence feature and the frequency spectrum feature according to a preset weight to form a fusion feature; and inputting the fusion feature to a detection model to obtain an unmanned aerial vehicle model, wherein the detection model is obtained by establishing a mapping relationship between the radio signal and the unmanned aerial vehicle model through machine learning. Based on the model of the unmanned aerial vehicle, determining an obstacle avoidance mechanism of the unmanned aerial vehicle from an obstacle avoidance database, the obstacle avoidance database including obstacle avoidance mechanisms of unmanned aerial vehicles of different models. According to the scheme, the unmanned aerial vehicle obstacle avoidance mechanism can be determined remotely.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to a method and device for determining an obstacle avoidance mechanism of an unmanned aerial vehicle and an electronic device. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology, unmanned aerial vehicles are increasingly widely used in civilian and military fields, but the resulting safety hazards (such as illegal intrusion and collision accidents) pose a serious threat to public safety. In the prior art, the regulatory authorities cannot learn about the obstacle avoidance mechanism of the intruding unmanned aerial vehicle in advance, and lack remote identification means for the obstacle avoidance mechanism of the unmanned aerial vehicle, resulting in a lack of basis for the decision to drive away or intercept. SUMMARY

[0003] Embodiments of the present application provide a method and device for determining an obstacle avoidance mechanism of an unmanned aerial vehicle and an electronic device, for realizing remote identification of the obstacle avoidance mechanism of the unmanned aerial vehicle.

[0004] In a first aspect, a method for determining an obstacle avoidance mechanism of an unmanned aerial vehicle is provided, the method comprising:

[0005] obtaining a radio signal when the unmanned aerial vehicle is flying, the radio signal being an electromagnetic wave emitted when the unmanned aerial vehicle communicates with a ground station or other equipment;

[0006] converting the radio signal into a frequency spectrum graph by short-time Fourier transform, extracting time sequence features representing a time dimension sequence of the frequency spectrum graph by a long short-term memory network, extracting frequency spectrum features of the frequency spectrum graph by a convolutional neural network, and weighting and fusing the time sequence features and the frequency spectrum features according to a preset weight to form fused features, the frequency spectrum features including spatial local features in the frequency spectrum graph;

[0007] inputting the fused features into a detection model to obtain a model of the unmanned aerial vehicle, the detection model being obtained by machine learning to establish a mapping relationship between the radio signal and the model of the unmanned aerial vehicle;

[0008] determining the obstacle avoidance mechanism of the unmanned aerial vehicle from an obstacle avoidance database based on the model of the unmanned aerial vehicle, the obstacle avoidance database containing obstacle avoidance mechanisms of multiple different models of unmanned aerial vehicles, the obstacle avoidance mechanism including a type of obstacle avoidance sensor, an algorithm strategy and a response rule used by the unmanned aerial vehicle.

[0009] In the above method, the radio signal is used to support remote monitoring at a distance of several kilometers due to its strong penetration and high concealment, and the obstacle avoidance mechanism of the unmanned aerial vehicle is determined at a distance. The radio signal is converted into a frequency spectrum graph by short-time Fourier transform, and the time sequence features are extracted by a long short-term memory network and the frequency spectrum features are extracted by a convolutional neural network, to realize multi-modal feature fusion. The time sequence dynamic features and the spatial local features are fused to improve the ability to distinguish different models of unmanned aerial vehicles in the same frequency band. The mapping database of the model and the obstacle avoidance strategy is established to associate the obstacle avoidance mechanism, which can improve the efficiency of determining the obstacle avoidance mechanism.

[0010] Optionally, the detection model is trained by the following method:

[0011] Obtain radio signals of multiple unmanned aerial vehicles in flight, and construct a mapping relationship between the radio signals and the unmanned aerial vehicle models;

[0012] The radio signals are processed by short-time Fourier transform to obtain a frequency spectrum graph, and a frequency spectrum graph feature in a time-frequency domain is extracted from the frequency spectrum graph, the frequency spectrum graph feature being used to reflect the change rule of the signal in time and frequency;

[0013] A long short-term memory network time sequence signal is used to process to obtain a time sequence feature, the time sequence signal being a time dimension sequence of the frequency spectrum graph;

[0014] A convolutional neural network is used to extract features from the frequency spectrum graph feature to obtain a frequency spectrum feature, the frequency spectrum feature including a spatial local feature in the frequency spectrum graph;

[0015] The time sequence feature and the frequency spectrum feature are weighted and fused until the loss function of the detection model meets a preset condition, and it is determined that the training of the detection model is completed.

[0016] In the above method, the time sequence feature and the frequency spectrum feature are weighted and fused, and the loss function is optimized to a preset threshold. It is ensured that the model fully captures the time-frequency domain correlation of the radio signal in the training process, and overfitting is avoided.

[0017] Optionally, the above method further comprises:

[0018] Obtaining a multi-angle image set of the unmanned aerial vehicle,

[0019] An image recognition model is used to match the image set with an image database to determine the unmanned aerial vehicle model, the image database containing image features of multiple models of unmanned aerial vehicles and model labels corresponding to the image features, and the image recognition model being obtained by training images of the image database using a convolutional neural network model.

[0020] In the above method, the multi-angle image set of the unmanned aerial vehicle is obtained, and the image database is matched to determine the unmanned aerial vehicle model. When the radio signal is disturbed, multiple ways of identifying the unmanned aerial vehicle can be provided, and the diversity of the unmanned aerial vehicle obstacle avoidance mode determination is increased.

[0021] Optionally, the image recognition model is trained by the following method:

[0022] The images in the image database are subjected to geometric transformation data enhancement to generate training samples containing multi-angle images;

[0023] The convolutional neural network model is used to perform image feature recognition on the training samples until the loss function of the convolutional neural network model reaches a preset condition in a verification set, so as to obtain an image recognition model, and the verification set is a subset of the training samples.

[0024] In the method, the multi-angle training samples are generated through geometric transformation data enhancement and combined with convolutional neural network training. The adaptability to the attitude change of the unmanned aerial vehicle can be enhanced, the single-image recognition accuracy can be improved, and the misjudgment caused by the perspective difference can be reduced.

[0025] Optionally, if the unmanned aerial vehicle model is unknown, the method further includes:

[0026] Obtaining images of the unmanned aerial vehicle from multiple angles;

[0027] Using an AI search engine to perform intelligent search to determine the unmanned aerial vehicle model of the unmanned aerial vehicle;

[0028] Adding the unmanned aerial vehicle model and the images of the unmanned aerial vehicle to an image database.

[0029] In the method, the AI search engine is used to intelligently search for images of unknown models and dynamically update the image database. The system is self-learned, and the response time for identifying new models is shortened.

[0030] Optionally, the method further includes:

[0031] Using a radar receiver to capture electromagnetic waves reflected by the unmanned aerial vehicle;

[0032] Extracting feature information from the electromagnetic waves, the feature information including physical characteristics, signal features, and motion states generated by the reflection of radar waves by the unmanned aerial vehicle;

[0033] Comparing the feature information with a radar feature database to identify the unmanned aerial vehicle model, the radar feature database containing a correspondence between multi-dimensional radar features of multiple models of unmanned aerial vehicles and the unmanned aerial vehicle models.

[0034] In the method, the reflected electromagnetic waves are captured by the radar receiver, the physical characteristics and motion state features are extracted, and the radar feature database is matched. A multi-modal perception unmanned aerial vehicle obstacle avoidance mode is constructed, and the recognition rate of the unmanned aerial vehicle obstacle avoidance rules is improved.

[0035] In a second aspect, an embodiment of the present application provides a device for determining an unmanned aerial vehicle obstacle avoidance mechanism, the device comprising:

[0036] A transceiver module is configured to obtain radio signals when the unmanned aerial vehicle is flying, the radio signals being electromagnetic waves emitted when the unmanned aerial vehicle communicates with a ground station or other equipment;

[0037] The processing module is configured to convert the radio signal into a spectrogram through a short-time Fourier transform, extract time sequence features representing a time dimension sequence of the spectrogram through a long short-term memory network, extract spectral features of the spectrogram through a convolutional neural network, and fuse the time sequence features and the spectral features according to a preset weight to form fused features, the spectral features including spatial local features in the spectrogram.

[0038] The processing module is further configured to input the radio signal into a detection model to obtain the model of the unmanned aerial vehicle, the detection model being obtained by mapping the radio signal and the model of the unmanned aerial vehicle through machine learning.

[0039] The determining module is configured to determine an obstacle avoidance mechanism of the unmanned aerial vehicle from an obstacle avoidance database based on the model of the unmanned aerial vehicle, the obstacle avoidance database containing obstacle avoidance mechanisms of unmanned aerial vehicles of different models, and the obstacle avoidance mechanism including a type of obstacle avoidance sensor, an algorithm strategy and a response rule used by the unmanned aerial vehicle.

[0040] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable by the processor, when the computer program is executed by the processor, the processor implements any of the methods in the first aspect.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a processor, any of the methods in the first aspect is implemented.

[0042] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, the computer program is executed by a processor to implement any of the methods in the first aspect.

[0043] The technical effects brought by any of the implementation manners of the second aspect to the fifth aspect can be referred to the technical effects brought by the corresponding implementation manners of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 An application scenario schematic flowchart of the identity verification method provided in the embodiment of the present application;

[0045] Figure 2 A flowchart of the unmanned aerial vehicle obstacle avoidance mechanism method provided in the embodiment of the present application;

[0046] Figure 3 A structural schematic diagram of the unmanned aerial vehicle obstacle avoidance mechanism device provided in the embodiment of the present application;

[0047] Figure 4 A structural schematic diagram of the control device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0048] Hereinafter, some terms in the embodiments of the present application are explained and described, so as to facilitate the understanding of the skilled in the art.

[0049] In order to make the purposes, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the skilled in the art without creative work fall within the scope of protection of the present application.

[0050] The application scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. The skilled in the art can know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0051] As shown in the application scenario schematic diagram of an optional unmanned aerial vehicle obstacle avoidance mechanism determination method of the present application, Figure 1 The application scenario schematic diagram of an optional unmanned aerial vehicle obstacle avoidance mechanism determination method of the present application includes a server 100, a terminal 101 and an unmanned aerial vehicle 103. The server 100 and the terminal 101 can be communicatively connected through a network to implement the fault diagnosis model training method of the present application.

[0052] The user can use the server 100 to interact with the terminal 101 through the network, such as receiving or sending messages, etc. Various client application programs can be installed on the terminal 101, such as program writing type applications, web browser applications, search type applications, etc.

[0053] It can be understood that in the embodiments of the present application, the server 100 can be implemented by an independent server or a server cluster composed of multiple servers. The terminal 101 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, desktop computers, etc.

[0054] As shown in the flowchart of an optional unmanned aerial vehicle obstacle avoidance mechanism determination method of the present application, Figure 2 The flowchart of an optional unmanned aerial vehicle obstacle avoidance mechanism determination method of the present application can include the following steps. The following is described taking the terminal as an example.

[0055] Step S201, obtaining a radio signal when the unmanned aerial vehicle is flying. The radio signal is an electromagnetic wave emitted when the unmanned aerial vehicle communicates with the ground station or other equipment.

[0056] For example, when the terminal detects the flight of the UAV, the radio signal of the UAV is acquired through a software defined radio (SDR) device.

[0057] In step S202, the radio signal is converted into a spectrogram through a short-time Fourier transform (STFT), a time sequence feature representing a time dimension sequence of the spectrogram is extracted through a long short-term memory (LSTM) network, a frequency spectrum feature of the spectrogram is extracted through a convolutional neural network (CNN), and the time sequence feature and the frequency spectrum feature are weighted and fused according to a preset weight to form a fusion feature, and the frequency spectrum feature includes a spatial local feature in the spectrogram.

[0058] For example, after obtaining the radio signal, the terminal can be preprocessed as follows: removing noise outside the frequency band and retaining the effective signal bandwidth (such as 20 MHz). The signal power is standardized to a uniform level (such as -70 dBm) to eliminate amplitude differences. The preprocessed time domain signal is converted into a time-frequency domain spectrogram through a short-time Fourier transform. The time-frequency domain spectrogram can reflect the energy distribution of the signal in time and frequency. Sequence data is constructed along the time dimension of the spectrogram and input into a double-layer LSTM network to output a time sequence feature vector. A two-dimensional convolutional neural network is used to extract spatial local features of the spectrogram. The spectrogram is scanned through a convolution kernel (such as 3x3) and combined with a dimension reduction pooling layer (2x2) to obtain a frequency spectrum vector. The time sequence feature extracted by the LSTM and the frequency spectrum feature extracted by the CNN are weighted and fused according to a preset weight (such as time sequence feature 0.4, frequency spectrum feature 0.6) to form a fusion feature.

[0059] In step S203, the radio signal is input into a detection model to obtain the UAV model.

[0060] The detection model is obtained by establishing a mapping relationship between the radio signal and the UAV model through machine learning.

[0061] The training method of the detection model is described as follows:

[0062] A plurality of radio signals of unmanned aerial vehicles in flight are acquired, and a mapping relationship between the radio signals and unmanned aerial vehicle models is constructed. A short-time Fourier transform (STFT) is used to process the radio signals to obtain a spectrogram, and a spectrogram feature of the spectrogram in a time-frequency domain is extracted, the spectrogram feature being used to reflect a change rule of the signal in time and frequency. A long short-term memory (LSTM) network is used to process a time sequence signal to obtain a time sequence feature, the time sequence signal being a time dimension sequence of the spectrogram. A convolutional neural network (CNN) is used to extract a frequency spectrum feature from the spectrogram feature, the frequency spectrum feature including a spatial local feature and a high-level abstract feature in the spectrogram. The time sequence feature and the frequency spectrum feature are weighted and fused until a loss function of a detection model meets a preset condition, and it is determined that the detection model is trained.

[0063] For example, the terminal acquires radio signals of unmanned aerial vehicles at different distances (such as 100 meters, 500 meters, etc.), angles (0°-360° range), and environmental scenes (such as different scenes in cities and suburbs) through an SDR device, thereby constructing a multi-source signal data set containing various scene information. Moreover, a label corresponding to the radio signals and unmanned aerial vehicle models is established, forming a mapping relationship of “radio signals-unmanned aerial vehicle models”, and providing accurate information for subsequent model training. The collected radio signals are processed using a short-time Fourier transform to extract a spectrogram feature of the radio signals in a time-frequency domain. The spectrogram feature is used to reflect a change rule of the signal in time and frequency. Then, a long short-term memory network is used to process a time sequence signal. The LSTM can capture long-term dependencies of the signal in the time sequence, and effectively extract time sequence dynamic features of the signal. Meanwhile, a convolutional neural network is used to extract a frequency spectrum feature from the spectrogram. The frequency spectrum feature includes a spatial local feature and a high-level abstract feature in the spectrogram. Finally, a fusion layer is used to weight and fuse the time sequence feature extracted by the LSTM and the frequency spectrum feature extracted by the CNN, so that the detection model can automatically adjust the fusion ratio according to the importance of different features, and obtain a comprehensive feature representation with time sequence information and spatial information. When the loss function of the detection model meets the preset condition, it is determined that the detection model is trained. For example, the loss function can use an ArcFace loss function, which increases the inter-class distance and reduces the intra-class distance, thereby enhancing the ability of the model to distinguish different unmanned aerial vehicle models.

[0064] Optionally, the terminal can also build a test set to verify the stability of the detection model at different time periods. The test set contains radio signals collected at different times of the day and in different seasons in the time dimension. In the spatial dimension, it covers data collected in different geographic locations to examine the adaptability of the model in different regions. In the device dimension, it covers models of unmanned aerial vehicles from commercial manufacturers such as DJI, Autel, and Parrot to ensure the recognition ability of the detection model for various types of unmanned aerial vehicles.

[0065] In addition, the terminal can also deploy a continuous learning mechanism to implement distributed model updating using a federated learning framework, enabling the detection model to continuously learn new data and continuously improve performance. At the same time, a confidence threshold (e.g., 0.85) is set, and when the confidence of the detection model's prediction result is less than the confidence threshold, an artificial review process is triggered to ensure the accuracy of the recognition result.

[0066] In an optional case, the terminal can also analyze the unmanned aerial vehicle model through Remote Identification (RemoteID) technology in the radio signal.

[0067] The Remote Identification technology transmits structured information including the unique serial number of the unmanned aerial vehicle, the operator's certificate number, the takeoff location coordinates, the flight path planning, and the real-time attitude data through the encrypted digital signal actively broadcast by the unmanned aerial vehicle.

[0068] For example, the system uses a software-defined radio (SDR) device (such as USRP N320) deployed in the monitoring area to capture 2.4 GHz / 5.8 GHz frequency band radio signals in real time. When the unmanned aerial vehicle enters the monitoring range, the SDR captures a 1090 MHz extension band signal with a signal power of -65 dBm and a duration of 120 ms. A protocol feature matching algorithm is used to detect RemoteID data packets that meet the Remote Identification and Tracking Standard Specification ASTM F3411-22a standard. The following features are identified: data packet start identifier (starting with 0x7E) JSON format metadata field (containing latitude, longitude, altitude, and unmanned aerial vehicle ID) cyclic redundancy check code (CRC-32). For example, the data packet header "7E 00 12 4B 2A..." is parsed, confirming compliance with the RemoteID v2.1 protocol specification, and the CRC check value is 0xC1A3F7B2. The parsed data packet is subjected to digital signature verification: the device manufacturer's signature is verified using Elliptic Curve Cryptography (ECDSA-384), confirming that the data has not been tampered with.

[0069] For registered device identification: extract the device unique identifier (UID) from the X.509 digital certificate. Perform a hash table query in the in-memory database to match the pre-registered device record. Determine the matching result as: model name: DJI Mavic 3 Pro manufacturer: SZ DJI Technology Co., Ltd. (Unified Social Credit Code: 9144030076196647XW) airworthiness certificate number: CAAC-UA-2025-001234 registered owner: Zhang San

[0070] For unregistered device identification: match the signal features in the radio signal with the model feature library. If the feature matching degree is greater than or equal to the preset threshold, the corresponding unmanned aerial vehicle model of the unmanned aerial vehicle is determined. The model feature library includes signal features of multiple unmanned aerial vehicle models. The signal features at least include modulation mode, frequency band occupation template, and frame synchronization header sequence. It can be understood that the threshold corresponding to different feature matching degrees can be different. For example, by matching the signal features in the radio signal with the model feature library, it is determined that the modulation mode matching degree is ≥ 90% (first threshold), and the frequency band occupation template similarity is ≥ 85% (second threshold), then it can be confirmed that the unmanned aerial vehicle is the unmanned aerial vehicle model corresponding to the features.

[0071] In this way, the RemoteID technology can complement the detection model based on signal feature learning, ensuring identification efficiency.

[0072] In another optional case, the terminal can also obtain a multi-angle image set of the unmanned aerial vehicle through the camera, and match the image set with the image database using an image recognition model to determine the unmanned aerial vehicle model. The image database contains image features of multiple unmanned aerial vehicle models and model labels corresponding to the image features. The image recognition model is obtained by training the images in the image database using a convolutional neural network model.

[0073] The training method of the image recognition model is described below:

[0074] The images in the image database are subjected to geometric transformation data enhancement to generate training samples containing multi-angle images. The convolutional neural network model is used to identify image features of the training samples until the loss function of the convolutional neural network model reaches the preset condition in the verification set, and the image recognition model is obtained. The verification set is a subset of the training samples.

[0075] The establishment method of the image database is described below:

[0076] In some embodiments, image feature recognition can be performed for known models of UAVs. For example, by collecting multi-angle image data of multiple known models of UAVs, including different perspectives such as front, side, top, bottom, etc., an image data set covering various scenarios (such as different lighting conditions, background environments) is constructed. A multi-angle image feature recognition of known models of UAVs is performed using a machine learning algorithm, such as a convolutional neural network (CNN). Since CNN can automatically learn high-level features in images, such as key features such as body outline, rotor structure, sensor layout, logo pattern, etc. The image recognition model can accurately extract and distinguish the features of different models of UAVs, thereby establishing a known image database. The image database contains image feature vectors of various models of UAVs and their corresponding model labels, providing a reliable basis for subsequent model recognition.

[0077] In other embodiments, if the model of the UAV is unknown, the terminal can obtain multiple-angle images of the UAV. An AI search engine is used for intelligent search to determine the model of the UAV. The model of the UAV and the images of the UAV are added to the image database.

[0078] For example, when encountering a UAV of an unknown model, multiple-angle images of the UAV are taken. An AI search engine is used for a full-network search. For example, the AI search engine can analyze the above-mentioned images, extract key features in the images such as shape, color, texture, etc., and generate corresponding feature descriptions. Then, the search engine can search for UAV model information matching the above-mentioned feature descriptions within the full network. By comparing the image, product description, technical parameters, etc. in the search results, the model of the UAV is identified online. Once the matching model information is found, the related features of the model are supplemented to the known image database to continuously improve the image database and improve the recognition ability of UAVs of unknown models.

[0079] In another optional case, the terminal can also capture electromagnetic waves reflected by the UAV through a radar receiver. Feature information in the electromagnetic waves is extracted, including physical characteristics, signal features, and motion states generated by the reflection of radar waves by the UAV. The feature information is compared with a radar feature database to identify the model of the UAV.

[0080] The radar feature database contains a correspondence between multi-dimensional radar features of multiple models of UAVs and the models of the UAVs.

[0081] For example, when the radar emits electromagnetic waves to irradiate the UAV, the UAV will reflect part of the electromagnetic waves back to the radar receiver. Different models of UAVs have different reflection characteristics of radar waves due to differences in material (such as carbon fiber, engineering plastic, metal alloy, etc.), shape (such as multi-rotor, fixed-wing, single-rotor, etc.), size (such as micro, small, medium, etc.), and surface coating, which mainly manifest in the intensity, frequency, phase, and other parameters of the reflected electromagnetic waves. By capturing the reflected electromagnetic waves with the radar receiver and accurately measuring and deeply analyzing the reflection values, the characteristic information of the UAV can be extracted. The characteristic information is compared with a pre-established radar reflection value characteristic database of UAV models to identify the UAV model. Physical characteristics (such as radar cross section RCS, external dimensions), signal modulation characteristics (such as micro-Doppler effect, polarization scattering matrix), and motion state (such as speed, trajectory, acceleration);

[0082] In this way, the UAV can be identified at a distance greater than 1 kilometer, providing important detection data for the identification of the obstacle avoidance mechanism. In the UAV obstacle avoidance scene, identifying the model of the UAV at a distance in advance helps the system quickly determine its flight characteristics, obstacle avoidance strategy, and potential risks, so as to make timely obstacle avoidance decisions and ensure the safe flight of the UAV in complex environments.

[0083] Step S204, determining the obstacle avoidance mechanism of the UAV from the obstacle avoidance database based on the UAV model.

[0084] The obstacle avoidance database contains obstacle avoidance mechanisms of multiple different models of UAVs, and the obstacle avoidance mechanism includes the type of obstacle avoidance sensor used by the UAV, the algorithm strategy, and the response rule.

[0085] For example, the obstacle avoidance database can include the UAV brand, the UAV model, whether infrared obstacle avoidance, whether ultrasonic obstacle avoidance, whether visual obstacle avoidance, and whether aurora obstacle avoidance. Among them, the model and the obstacle avoidance method can be collected through the network and set by the experience of those skilled in the art, so as to ensure the richness of the data. The obstacle avoidance mechanism includes that the sensor of the UAV is an ultrasonic sensor. The algorithm strategy is pure visual navigation based on optical flow. The response rule is to trigger a vibration alarm when an obstacle enters the field of view and forcibly reduce the throttle. After the collision, the last known coordinates are automatically recorded,

[0086] In the above method, the database supports millisecond-level query response through the application programming interface (API). When it is necessary to determine the obstacle avoidance method of a certain model of UAV, the corresponding API interface can be called to obtain the obstacle avoidance rules and behavior patterns of the UAV model from the database in a very short time, realizing efficient data query.

[0087] Optionally, the obstacle avoidance database can be updated according to a preset period. The automatic, efficient and safe updating of the obstacle avoidance database provides accurate data support for remote identification and driving away.

[0088] For example, a database update task is automatically performed at 2 a.m. every day to capture the latest information from the official websites of various manufacturers, industry reports and third-party data services (such as the Aliyun unmanned aerial vehicle model library).

[0089] Figure 3 A structure diagram of a determination apparatus of an unmanned aerial vehicle obstacle avoidance mechanism provided by an embodiment of the present application is shown in FIG. 1. Figure 3 As shown in the figure, the apparatus includes a transceiver module 301, a processing module 302 and a determination module 303.

[0090] The transceiver module 301 is configured to acquire a radio signal when the unmanned aerial vehicle is flying, the radio signal being an electromagnetic wave emitted when the unmanned aerial vehicle communicates with a ground station or other equipment;

[0091] The processing module 302 is configured to convert the radio signal into a frequency spectrum graph by short-time Fourier transform, extract time sequence features representing a time dimension sequence of the frequency spectrum graph by a long short-term memory network, extract frequency spectrum features of the frequency spectrum graph by a convolutional neural network, and form fused features by weighting and fusing the time sequence features and the frequency spectrum features according to a preset weight, the frequency spectrum features including spatial local features in the frequency spectrum graph.

[0092] The processing module 302 is further configured to input the radio signal into a detection model to obtain a model of the unmanned aerial vehicle, the detection model being obtained by mapping the radio signal and the model of the unmanned aerial vehicle through machine learning;

[0093] The determination module 303 is configured to determine an obstacle avoidance mechanism of the unmanned aerial vehicle from an obstacle avoidance database based on the model of the unmanned aerial vehicle, the obstacle avoidance database containing obstacle avoidance mechanisms of multiple different models of unmanned aerial vehicles, the obstacle avoidance mechanism including a type of obstacle avoidance sensor, an algorithm strategy and a response rule adopted by the unmanned aerial vehicle.

[0094] Optionally, the detection model is trained by the following method, and the processing module 302 is further configured to:

[0095] Acquire multiple radio signals when the unmanned aerial vehicles are flying, and build a mapping relationship between the radio signals and the models of the unmanned aerial vehicles;

[0096] The radio signals are processed by short-time Fourier transform to obtain frequency spectrum graphs, and frequency spectrum graph features in the time-frequency domain are extracted, the frequency spectrum graph features being used to reflect the change law of the signal in time and frequency;

[0097] The time sequence features are obtained by processing time sequence signals by a long short-term memory network, the time sequence signals being time dimension sequences of the frequency spectrum graphs;

[0098] The convolutional neural network is used to extract features of the spectrum graph to obtain spectrum features, and the spectrum features include spatial local features in the spectrum graph.

[0099] The time sequence features and the spectrum features are weighted and fused until the loss function of the detection model meets the preset condition, and it is determined that the training of the detection model is completed.

[0100] Optionally, the transceiver module 301 is further configured to: acquire a plurality of images of the unmanned aerial vehicle from different angles,

[0101] The processing module 302 is further configured to: match the image set with an image database by using an image recognition model to determine the unmanned aerial vehicle model, the image database contains image features of a plurality of unmanned aerial vehicle models and model labels corresponding to the image features, and the image recognition model is obtained by training images in the image database by using a convolutional neural network model.

[0102] Optionally, the image recognition model is trained in the following manner, and the processing module 302 is further configured to:

[0103] The images in the image database are subjected to geometric transformation data enhancement to generate training samples containing images from different angles;

[0104] The training samples are subjected to image feature recognition by using a convolutional neural network model until the loss function of the convolutional neural network model reaches a preset condition in a verification set, and the image recognition model is obtained, and the verification set is a subset of the training samples.

[0105] Optionally, if the unmanned aerial vehicle model is unknown, the transceiver module 301 is further configured to: acquire a plurality of images of the unmanned aerial vehicle from different angles;

[0106] The processing module 302 is further configured to: perform intelligent search by using an AI search engine to determine the unmanned aerial vehicle model of the unmanned aerial vehicle;

[0107] The unmanned aerial vehicle model and the image of the unmanned aerial vehicle are added to the image database.

[0108] Optionally, the transceiver module 301 is further configured to: capture electromagnetic waves reflected by the unmanned aerial vehicle by using a radar receiver;

[0109] The processing module 302 is further configured to: extract feature information in the electromagnetic waves, and the feature information includes physical characteristics, signal features and motion states generated by reflection of radar waves by the unmanned aerial vehicle;

[0110] The feature information is compared with a radar feature database to identify the unmanned aerial vehicle model, and the radar feature database contains a correspondence relationship between multi-dimensional radar features of a plurality of unmanned aerial vehicle models and the unmanned aerial vehicle models.

[0111] Figure 4 A structural schematic diagram of an electronic device provided by the embodiments of the present application is shown.

[0112] The at least one processor 401 and the memory 402 connected with the at least one processor 401 are not limited to the specific connection medium between the processor 401 and the memory 402 in the embodiments of the present application, Figure 4 The connection between the processor 401 and the memory 402 in the embodiments of the present application is taken as an example of connection through the bus 400. The bus 400 is used to connect the processor 401 and the memory 402 in the embodiments of the present application, Figure 4 The connection mode between other components is only schematically illustrated, and is not limited. The bus 400 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 In the embodiments of the present application, the processor 401 can also be called a controller, and the name is not limited.

[0113] In the embodiments of the present application, the memory 402 stores instructions executable by the at least one processor 401. The at least one processor 401 can execute the instructions stored in the memory 402 to perform the method for determining the unmanned aerial vehicle obstacle avoidance mechanism discussed above. The processor 401 can realize the functions of various modules in the apparatus shown in the drawings. Figure 3

[0114] The processor 401 is the control center of the apparatus, can connect each part of the whole control device through various interfaces and lines, and perform overall monitoring on the apparatus by running or executing the instructions stored in the memory 402 and calling the data stored in the memory 402, thereby processing data and realizing various functions of the apparatus.

[0115] In a possible design, the processor 401 can include one or more processing units. The processor 401 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the driver interface and the application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 can be implemented on the same chip, and in some embodiments, they can also be respectively implemented on independent chips.

[0116] ​The processor 401 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the method for determining an obstacle avoidance mechanism of a UAV disclosed in the embodiments of the present application can be directly embodied by a hardware processor for execution, or executed by a combination of hardware and software modules in the processor.

[0117] The memory 402 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 402 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory 402 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 402 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0118] By designing and programming the processor 401, the code corresponding to the method for determining an obstacle avoidance mechanism of a UAV in the foregoing embodiments can be fixed into the chip, so that the chip can execute the method for determining an obstacle avoidance mechanism of a UAV of the embodiments shown in the figure at runtime. Figure 1 How to design and program the processor 401 is a technology known to those skilled in the art, which will not be described here.

[0119] It should be noted that the above general electronic device provided by the embodiments of the present application can realize all method steps realized by the above method embodiments, and can achieve the same technical effects, and the same parts and beneficial effects in the embodiments will not be described here.

[0120] The embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are used for causing a computer to execute the method for determining the UAV obstacle avoidance mechanism in the above embodiment.

[0121] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of being implemented on one or more computer readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0122] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks. Figure 1 The functions specified in one or more flows and / or blocks.

[0123] These computer program instructions can also be stored in a computer readable memory capable of guiding the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks. Figure 1 The functions specified in one or more flows and / or blocks.

[0124] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks. Figure 1 The functions specified in one or more flows and / or blocks.

[0125] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for determining an obstacle avoidance mechanism for an unmanned aerial vehicle (UAV), characterized in that, The method includes: Acquire radio signals during the flight of the UAV, wherein the radio signals are electromagnetic waves emitted by the UAV when communicating with ground stations or other equipment; The radio signal is converted into a spectrum graph by short-time Fourier transform, the temporal features representing the time dimension sequence of the spectrum graph are extracted by long short-term memory network, the spectral features of the spectrum graph are extracted by convolutional neural network, and the temporal features and spectral features are weighted and fused according to preset weights to form fused features. The spectral features include spatial local features in the spectrum graph. The fused features are input into the detection model to obtain the drone model, which is obtained by establishing a mapping relationship between radio signals and drone models through machine learning; Based on the drone model, the obstacle avoidance mechanism of the drone is determined from the obstacle avoidance database. The obstacle avoidance database contains obstacle avoidance mechanisms for various drone models. The obstacle avoidance mechanism includes the type of obstacle avoidance sensor, algorithm strategy, and response rules used by the drone.

2. The method according to claim 1, characterized in that, The detection model is trained in the following manner: Acquire radio signals from multiple drones during flight and establish a mapping relationship between radio signals and drone models; The radio signal is processed using a short-time Fourier transform to obtain a spectrum diagram. The spectrum diagram features in the time-frequency domain are extracted from the spectrum diagram. These spectrum diagram features are used to reflect the variation patterns of the signal in time and frequency. Temporal features are obtained by processing temporal signals using a long short-term memory network, wherein the temporal signal is the time dimension sequence of the spectrogram. A convolutional neural network is used to extract spectral features from the spectrogram features, and the spectral features include spatial local features in the spectrogram. The time-series features and the spectral features are weighted and fused until the loss function of the detection model meets the preset conditions, at which point the detection model training is considered complete.

3. The method according to claim 1, characterized in that, The method further includes: Obtain a set of images of the drone from multiple angles. An image recognition model is used to match the image set with an image database to determine the drone model. The image database contains image features of various drone models and model labels corresponding to the image features. The image recognition model is obtained by training the images in the image database using a convolutional neural network model.

4. The method according to claim 3, characterized in that, The image recognition model is trained in the following way: Geometric transformations are performed on images in the image database to augment data and generate training samples containing images from multiple angles. The image recognition model is obtained by using a convolutional neural network model to perform image feature recognition on the training samples until the loss function of the convolutional neural network model reaches a preset condition on the validation set, where the validation set is a subset of the training samples.

5. The method according to claim 3 or 4, characterized in that, If the drone model is unknown, the method further includes: Acquire images of the drone from multiple angles; The drone model is determined by using an AI search engine for intelligent searching. Add the drone model and its image to the image database.

6. The method according to claim 1, characterized in that, The method further includes: The electromagnetic waves reflected by the UAV are captured by a radar receiver; Extract feature information from the electromagnetic waves, including the physical characteristics, signal characteristics, and motion state of the UAV reflecting radar waves; The feature information is compared with a radar feature database to identify the UAV model. The radar feature database contains the correspondence between multi-dimensional radar features of various UAV models and UAV models.

7. A device for determining an obstacle avoidance mechanism for an unmanned aerial vehicle (UAV), characterized in that, The device includes: The transceiver module is used to acquire radio signals during the flight of the UAV, which are electromagnetic waves emitted by the UAV when communicating with ground stations or other equipment; The processing module is used to convert the radio signal into a spectrum graph through short-time Fourier transform, extract the temporal features representing the time dimension sequence of the spectrum graph through a long short-term memory network, extract the spectral features of the spectrum graph through a convolutional neural network, and fuse the temporal features and spectral features according to preset weights to form a fused feature. The spectral features include spatial local features in the spectrum graph. The processing module is also used to input the radio signal into the detection model to obtain the drone model, wherein the detection model is obtained by establishing a mapping relationship between the radio signal and the drone model through machine learning; The determination module is used to determine the obstacle avoidance mechanism of the UAV from the obstacle avoidance database based on the UAV model. The obstacle avoidance database contains obstacle avoidance mechanisms for various UAV models, and the obstacle avoidance mechanism includes the obstacle avoidance sensor type, algorithm strategy and response rules adopted by the UAV.

8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program for causing the computer to perform the method of any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is invoked by a computer, it causes the computer to perform the method as described in any one of claims 1-6.

Images