Training method, recognition method and system of unmanned aerial vehicle behavior pattern recognition model

Abstract

The invention discloses a training method, a recognition method and a system of an unmanned aerial vehicle behavior pattern recognition model, and belongs to the technical field of unmanned aerial vehicle safety. Then, sensing data are extracted according to the flight parameters, a time interval of behavior mode execution is traversed in combination with an adaptive sliding window method, a time window of a key action corresponding to the behavior model is determined, and a key action data set is constructed according to time sequence interval data of the time window; and finally, training the behavior recognition mode by adopting the key action data set to obtain a trained behavior recognition model. According to the method, the crash risk of the unmanned aerial vehicle due to improper behavior mode execution can be remarkably reduced.

Description

Technical Field

[0001] The present invention relates to the field of drone safety technology, and in particular to a training method, an identification method and a system for a drone behavior pattern recognition model. Background Art

[0002] In recent years, drones represent an emerging technological frontier that is revolutionizing industries in the military, civilian, and commercial sectors. In today's fast-paced world, there is an increasing demand for drones that can operate in challenging environments such as confined spaces and dense forests. These drones are required to perform tasks such as inspections and emergency responses in confined urban areas, as well as monitor wildlife and assess the environment in dense forests. However, drones may deviate from their planned flights due to vulnerability to external attacks, inherent firmware errors, and other faults. These weaknesses affect the integrity of the drone's flight plan, leading to collisions in confined spaces and even serious damage such as personal safety risks. Therefore, it is crucial to maintain the accuracy of drone flights. The current methods used to ensure the accuracy of drone flights have certain limitations, and a variety of methods are involved for such identification, including combining program-level detection, post-task log detection, etc. However, program-level detection only focuses on the software level and ignores possible failures caused by hardware. In addition, due to the lack of real-time performance of post-log detection, unexpected deviations may occur during the flight of the drone, and potential safety risks may occur.

[0003] The flight trajectory of a drone is usually divided into two main parts: straight flight and turning flight. During straight flight, the drone usually remains stable and does not deviate significantly from the planned flight path. However, when turning, the actual flight trajectory may deviate from the planned path due to factors such as the dynamic characteristics of the drone, wind speed changes, and limitations on GPS accuracy. Although these deviations may be small in a single turn, these errors may gradually accumulate during the entire mission execution, resulting in a large gap between the planned path and the actual path, especially in autonomous flight mode. Therefore, in order to ensure the smooth execution of the drone mission, various turning behaviors are regarded as "critical actions" for identification and monitoring.

[0004] Since the "key actions" mentioned above mainly affect the attitude of the drone, including position information (longitude, latitude, altitude, etc.), environmental information (temperature, humidity) and other attitude-independent data do not change significantly during the execution of the behavior, the flight data related to the change of the drone attitude can be used as a data source for verifying the key behavior pattern. The Chinese invention patent "UAV Fault Prediction Method Based on Flight Data" with application publication number CN117369403 A proposes a fault prediction method based on real-time flight data of drones, which is used to monitor and predict the health status of drones based on health dynamic data, solves the timeliness and accuracy of drone fault diagnosis, and achieves the effect of reducing maintenance costs and improving operational safety. However, in this method, a fixed window is used to characterize the flight mode of the drone. On the one hand, there will be a time window that is too large or too small, resulting in poor recognition effect. On the other hand, due to the limited onboard computing resources of the drone, it may also bring great challenges to the computing resources of the drone, and even affect the normal flight of the drone. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a training method, an identification method and a system for a drone behavior pattern recognition model. By establishing a behavior pattern recognition library and collecting sensor data representing the drone flight mode in real time during the drone flight, the drone flight mode can be verified and identified for correctness.

[0006] The present invention is achieved through the following technical solutions: A method for training a drone behavior pattern recognition model comprises the following steps: Step 1: Obtain the flight parameters of the drone in different behavior modes; Step 2: Extract perception data based on flight parameters and traverse the time interval of behavior pattern execution in combination with the adaptive sliding window method to determine the time window of key actions corresponding to the behavior model, and construct a key action data set based on the time series interval data of the time window; Step 3: Use the key action data set to train the behavior recognition pattern to obtain a trained behavior recognition model.

[0007] Preferably, the behavior modes include a turning mode at a constant altitude, a fixed angle lifting mode of the drone, and a hovering mode of the drone at a constant altitude.

[0008] Preferably, the method for constructing the key action data set in step 2 is as follows: Obtain the flight data of the UAV in various behavior modes, and extract the perception data after preprocessing the flight data; Perform volatility analysis on the perception data to determine the sliding standard deviation, dynamically adjust the size of the time window based on the sliding standard deviation, iteratively update the window size based on the volatility level and information entropy in the current window until the entire time interval is traversed, and use the voting method to determine the time window for key actions; The time series interval data of the time window corresponding to the key action is subjected to dimensionality reduction processing to obtain the time series interval data after dimensionality reduction, and the key action data set is constructed according to the time series interval data of each key action.

[0009] Preferably, the preprocessing includes normalization and missing value processing.

[0010] Preferably, the sliding standard deviation is calculated as follows:

[0011] in, is the average value of the data in this time window.

[0012] Preferably, the time series interval data is reduced in dimension to extract feature data, and a key action data set is constructed based on the feature data.

[0013] Preferably, the PCA method is used to reduce the dimension of the time series interval data, and the autoencoder is used to extract features from the reduced-dimensional time series interval data.

[0014] Preferably, the method further comprises the following steps: A behavior pattern database is built based on the key action data set. The behavior patterns of the drone are identified based on the trained behavior recognition model. The identified behavior patterns are classified and matched with the corresponding drone behavior patterns in the behavior pattern database to verify the correctness and completeness of the key actions.

[0015] A training system for a drone behavior pattern recognition model, comprising: The acquisition module is used to obtain the flight parameters of the drone in different behavior modes; A data processing module is used to extract perception data according to flight parameters and traverse the time interval of behavior pattern execution in combination with an adaptive sliding window method, determine the time window of key actions corresponding to the behavior model, and construct a key action data set based on the time series interval data of the time window; The training module is used to train the behavior recognition pattern using the key action data set to obtain a trained behavior recognition model.

[0016] A method for identifying a behavior pattern of a drone, comprising: S100, obtaining real-time flight time series data of the UAV; S200: Inputting the flight timing data into the pre-trained behavior recognition model to obtain the current behavior pattern of the UAV.

[0017] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention provides a training method for a drone behavior pattern recognition model. The method aims at the problem that the existing drones face the crash of the drone or even the threat to personal safety due to the deviation of the behavior pattern execution when performing tasks. By obtaining the flight parameters of the drone in different behavior modes and extracting the perception data accordingly, the method can more accurately capture the characteristics of the drone in different behavior modes. This helps the behavior recognition model to more accurately identify the current behavior mode of the drone, thereby correcting the deviation in time and avoiding potential safety risks. Secondly, the time interval of the behavior pattern execution is traversed in combination with the adaptive sliding window method, and the size of the time window can be dynamically adjusted to adapt to the timing characteristics of different behavior patterns. This method not only improves the accuracy of key action recognition, but also enhances the adaptability of the model to different behavior patterns. By determining the time window of the key action corresponding to the behavior model and constructing the key action data set according to the time interval data of these time windows, the method focuses on the key actions that have the greatest impact on the drone behavior. This helps the model pay more attention to these key actions during the training process, thereby improving the sensitivity and response speed to the behavior pattern deviation. In addition, the behavior recognition pattern is trained using the key action data set, so that the model can converge to the optimal solution faster, improving the training efficiency. At the same time, because the data set is more focused and precise, the trained model has higher accuracy and robustness in identifying drone behavior patterns. By accurately identifying drone behavior patterns and correcting deviations in a timely manner, this method can significantly reduce the risk of drone crashes due to improper execution of behavior patterns. At the same time, for application scenarios that may involve personal safety (such as drone delivery, inspection, etc.), this method can also provide more reliable safety guarantees. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 This is a flow chart of feature extraction of key actions of the drone of the present invention; Figure 2 The figure is a flow chart of the UAV behavior pattern recognition method of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0022] See also Figure 1 and 2 ,A training method for a drone behavior pattern recognition model comprises the following steps: Step 1: Obtain the flight parameters of the drone in different behavior modes.

[0023] The behavior modes include turning mode at a constant altitude, drone lifting fixed angle mode, drone hovering at a constant altitude and other behavior modes.

[0024] Flight parameters include flight control parameters, position information, speed, attitude, sensor parameters and flight log.

[0025] Step 2: Extract perception data based on flight parameters and traverse the time interval of behavior pattern execution in combination with the adaptive sliding window method to determine the time window of key actions corresponding to the behavior model, and construct a key action data set based on the time series interval data of the time window.

[0026] S2.1. Repeat the execution of specific behavior patterns of the UAV on the UAV simulation platform, obtain the log data of the UAV in each behavior pattern, normalize and fill in missing values ​​on the log data, and extract perception data based on the processed log data.

[0027] S2.2. Perform volatility analysis on the perception data to determine the sliding standard deviation, dynamically adjust the size of the time window based on the sliding standard deviation, iteratively update the window size based on the volatility level and information entropy in the current window until the entire time interval is traversed, and use the voting method to determine the time window for key actions.

[0028] The calculation of the time series interval corresponding to the drone behavior pattern uses an adaptive sliding window method based on statistical characteristics to determine the time series interval for the execution of the drone behavior pattern. By analyzing the volatility of the drone's acceleration, angular velocity and other sensor data, the size of the time series interval is dynamically adjusted to capture the best characteristics of the behavior pattern. Time series interval W = [ ], where the specific timestamp of the drone behavior pattern execution is marked as t, and the specific value of t is obtained through Mavlink communication; It will be determined dynamically based on the volatility of the data. Specifically, the optimal time series interval size is determined through the following steps: 1. Calculate the sliding standard deviation: First, calculate the sliding standard deviation of the sensor data Xi (such as acceleration, angular velocity, etc.) to measure the volatility of the data within the time window. For each variable Xi (i∈[1,18]), the sliding standard deviation is calculated within the time window [ ] is represented by:

[0029] in, is the average value of the data in this time window.

[0030] 2. Adaptive time window adjustment: Dynamically adjust the size of the time window based on the result of the sliding standard deviation. Exceeding the preset threshold , indicating that the volatility in this time period is large and a larger time window is needed for observation; otherwise, a smaller time window is maintained. The adjustment rules are:

[0031] in, and These are the preset maximum and minimum window sizes respectively.

[0032] 3. Time period division and labeling: Divide each time point into high volatility and low volatility areas according to its volatility. Use a larger window in the high volatility area. , use smaller windows in low volatility areas , and label each time period. The label set Y contains labels for high volatility and low volatility.

[0033] 4. Iteratively update the window size: According to the volatility level and information entropy in the current window, repeat the process from steps (1) to (4) until the entire time interval W is traversed. Dynamically adjust the window size to ensure that each behavior pattern can be accurately captured.

[0034] Through this adaptive sliding window method, the size of the time window can be automatically adjusted according to the volatility of the data. This method can effectively adapt to the different motion states of the drone, thereby capturing the behavior pattern more accurately. So far, the time window determined by Xi is obtained. For the other 17 variables in the current action, operations (1) to (4) are applied to obtain the corresponding time windows. Since each variable may have a different time window size, the final time window W of the current key action is determined by voting.

[0035] S2.3. Perform dimensionality reduction processing on the time series interval data of the time window corresponding to the key action to obtain the time series interval data after dimensionality reduction, construct a key action data set based on the time series interval data of each key action, and at the same time construct a behavior pattern library of the UAV.

[0036] The time series data representing different actions are processed by dimensionality reduction, and a hybrid method based on dimensionality reduction and deep learning is used to unify the time series data of different behavior patterns. The following techniques are used to extract the behavior characteristics of drones: 1. Principal Component Analysis (PCA): First, the original high-dimensional sensor data is reduced in dimension using the PCA method to retain the main features of the data. The specific steps are: Z=XW Among them, X is the original data matrix, W is the PCA transformation matrix, and Z is the feature matrix after dimensionality reduction. PCA selects the first few principal components by maximizing the variance of the data, thereby reducing the feature dimension and retaining important information in the data.

[0037] 2. Autoencoder: Then use the deep learning-based autoencoder to further extract the high-level features of the time series interval data after dimensionality reduction. The autoencoder learns the structure of the data by compressing the data into a latent space and then restoring it. In the feature extraction of drone behavior data, the autoencoder structure is: Encoder part: maps input data X to a low-dimensional feature space Z:

[0038] Decoder part: reconstructs data from the low-dimensional feature space to obtain feature data of key actions.

[0039]

[0040] in, and They are the neural networks for encoder and decoder respectively.

[0041] 3. Construct a key action dataset based on the feature data corresponding to each key action.

[0042] The combination of PCA and autoencoder finally generates a low-dimensional feature vector of the drone's behavior pattern. This feature vector contains the main information of the action and unifies the feature representation of different action time windows. These feature vectors can be used for subsequent behavior pattern classification and recognition. Finally, the different behavior pattern data after feature extraction are combined to obtain the key action data set.

[0043] Step 3: Use the key action data set to train the behavior recognition pattern to obtain a trained behavior recognition model.

[0044] Extract features and labels from the historical behavior pattern recognition library, and use these data to train the supervised learning classifier. The label of each sample is the action pattern corresponding to the behavior (such as Ω+10, Ω-10, and Ω), and the feature is the sensor data after feature extraction.

[0045] In this embodiment, a support vector machine (SVM) algorithm is used for training so that it can recognize and classify different motion patterns.

[0046] Construct the validation set S: According to the current action fingerprint obtained in real time, the historical data closest to it is extracted to form a verification set S. The verification set S contains three behavior pattern data closest to the current action pattern: Ω+10, Ω−10, and Ω. These data, as samples, will be input into the trained classifier for classification together with the current real-time behavior pattern data.

[0047] Classification operation: The real-time behavior pattern data X is fed into the pre-trained classifier, and the classifier classifies it into a specific behavior pattern based on the input feature vector.

[0048] The classification results are , that is, the pattern label (such as Ω+10, Ω−10, or Ω) that the classifier considers the current behavior pattern to be closest to.

[0049] The classification process can be expressed as:

[0050] in, is a trained supervised learning model, X is the feature vector of the current action, For classification results Behavior pattern accuracy judgment: According to the output of the classifier , determine the accuracy and completeness of the current action: If the classification result If it is highly consistent with a label in the validation set S (such as Ω), the current action is considered accurate and complete, and the drone continues to fly along the current flight path.

[0051] If the classification result deviates from the expectation (for example, closer to Ω+10 or Ω−10), the current action is considered to be inaccurate or incomplete, and a warning is reported to the ground station for intervention.

[0052] If the classification results appear to be noise data or unrecognized patterns in some offline mode data, a warning is issued, indicating that the current behavior is unusual.

[0053] Iteration and Update As new real-time data is acquired, the classifier can be continuously updated and adjusted. New data can also be used to further train and optimize the performance of the classifier to improve classification accuracy and robustness of behavioral pattern recognition.

[0054] Step 4: Identify the behavior pattern of the drone based on the trained behavior recognition model.

[0055] Acquire real-time flight data of the drone. During the flight of the drone, execute a specific drone behavior pattern Ω, communicate with the sensor module used to obtain motion perception data in real time through Mavlink, and save the obtained data in CSV, including the current timestamp and perception data; operate the CSV file, then extract the features of the flight data, input the features into the trained behavior recognition model, and obtain the current behavior pattern of the drone.

[0056] Step 5: Verify the behavior pattern output by the behavior recognition model.

[0057] The obtained real-time behavior pattern data is classified and matched with the corresponding drone behavior patterns in the behavior pattern recognition library to verify the correctness and completeness of the action.

[0058] The behavior pattern data closest to the current action fingerprint in the behavior pattern recognition library are used to form the verification set S of the current action, which includes the offline mode data of Ω+10, Ω-10 and Ω. The obtained real-time behavior pattern data and the verification set S are subjected to supervised learning-based classification operations, and the accuracy and completeness of the current action are determined according to the classification results.

[0059] The present invention proposes a solution to the safety risks caused by the deviation of the behavior pattern execution of drones. By monitoring the drone sensor data in real time and comparing it with the pre-built behavior pattern library, a drone behavior pattern verification model is constructed. The model can timely detect and correct the behavior deviation of the drone, effectively reducing the risk of crash, and flexibly adjust the data extraction interval according to the different behavior patterns of the drone, ultimately forming a systematic solution integrating data collection, processing, calculation and behavior pattern verification, simplifying system deployment and improving convenience and efficiency in practical applications.

[0060] Based on the above-mentioned training method of a drone behavior pattern recognition model, the present application also provides a training system for a drone behavior pattern recognition model, which may include: The acquisition module is used to obtain the flight parameters of the drone in different behavior modes.

[0061] The data processing module is used to extract perception data based on flight parameters and traverse the time interval of behavior pattern execution in combination with the adaptive sliding window method, determine the time window of key actions corresponding to the behavior pattern, and construct a key action data set based on the time series interval data of the time window.

[0062] The training module is used to train the behavior recognition model using the key action data set to obtain a trained behavior recognition model.

[0063] This application also proposes a method for identifying drone behavior patterns, including: S100, obtaining real-time flight time series data of the UAV; S200, inputting the flight time series data into a pre-trained behavior recognition model to obtain the current behavior pattern of the UAV.

[0064] Correspondingly, the present application also proposes a drone behavior pattern recognition system, which may include: The acquisition module is used to obtain the real-time flight timing data of the drone; The recognition module is used to input the flight timing data into the pre-trained behavior recognition model to obtain the current behavior pattern of the drone.

[0065] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0066] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0067] An electronic device provided in an embodiment of the present application includes a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps of the drone behavior pattern recognition method described in any of the above embodiments are implemented.

[0068] Another electronic device provided in the embodiment of the present application may also include: an input port connected to the processor, used to transmit multimodal data collected by an external acquisition device to the processor; and a display unit connected to the processor, used to display the processing results of the processor to the outside world; a communication module connected to the processor, used to realize the communication between the electronic device and the outside world. The display unit can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module includes but is not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless connection (including wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE802.11s).

[0069] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the drone behavior pattern recognition method described in any of the above embodiments are implemented.

[0070] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0071] For the description of the relevant parts of the drone behavior pattern recognition system, electronic device, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the drone behavior pattern recognition method provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0072] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A training method for a drone behavior pattern recognition model, characterized in that: The following steps are involved: Step 1: Obtain the flight parameters of the drone in different behavior modes; Step 2: Extract perception data based on flight parameters and traverse the time interval of behavior pattern execution in combination with the adaptive sliding window method to determine the time window of key actions corresponding to the behavior model, and construct a key action data set based on the time series interval data of the time window; Step 3: Use the key action data set to train the behavior recognition pattern to obtain a trained behavior recognition model.

2. The method for training a drone behavior pattern recognition model according to claim 1, characterized in that: The behavior modes include a turning mode at a constant altitude, a fixed angle lifting mode of the drone, and a hovering mode of the drone at a constant altitude.

3. The method for training a drone behavior pattern recognition model according to claim 1, characterized in that: The method for constructing the key action dataset described in step 2 is as follows: Obtain the flight data of the UAV in various behavior modes, and extract the perception data after preprocessing the flight data; Perform volatility analysis on the perception data to determine the sliding standard deviation, dynamically adjust the size of the time window based on the sliding standard deviation, iteratively update the window size based on the volatility level and information entropy in the current window until the entire time interval is traversed, and use the voting method to determine the time window for key actions; The time series interval data of the time window corresponding to the key action is subjected to dimensionality reduction processing to obtain the time series interval data after dimensionality reduction, and the key action data set is constructed according to the time series interval data of each key action.

4. The method for training a drone behavior pattern recognition model according to claim 3, characterized in that: The preprocessing includes normalization and missing value processing.

5. The method for training a drone behavior pattern recognition model according to claim 3, characterized in that: The sliding standard deviation is calculated as follows: in, is the average value of the data in this time window.

6. The method for training a drone behavior pattern recognition model according to claim 3, characterized in that: After reducing the dimension of the time series interval data, feature data is extracted, and a key action data set is constructed based on the feature data.

7. The method for training a drone behavior pattern recognition model according to claim 6, characterized in that: The PCA method is used to reduce the dimension of the time series interval data, and the autoencoder is used to extract features from the reduced dimension time series interval data.

8. The method for training a drone behavior pattern recognition model according to claim 1, characterized in that: The following steps are also included: A behavior pattern database is built based on the key action data set. The behavior patterns of the drone are identified based on the trained behavior recognition model. The identified behavior patterns are classified and matched with the corresponding drone behavior patterns in the behavior pattern database to verify the correctness and completeness of the key actions.

9. A training system for a drone behavior pattern recognition model, characterized in that: include: The acquisition module is used to obtain the flight parameters of the drone in different behavior modes; A data processing module is used to extract perception data according to flight parameters and traverse the time interval of behavior pattern execution in combination with an adaptive sliding window method, determine the time window of key actions corresponding to the behavior model, and construct a key action data set based on the time series interval data of the time window; The training module is used to train the behavior recognition pattern using the key action data set to obtain a trained behavior recognition model.

10. A method for identifying drone behavior patterns, characterized in that: include: S100, obtaining real-time flight time series data of the UAV; S200, inputting the flight timing data into the pre-trained behavior recognition model described in any one of claims 1-8 to obtain the current behavior pattern of the UAV.

Images