Unmanned aerial vehicle trajectory feature anomaly detection method based on grid map and Inceptionv3

Abstract

The invention discloses an unmanned aerial vehicle trajectory feature anomaly detection method based on a grid map and Inceptionv3, and the method comprises the steps: segmenting segments, judging the trajectory of an unmanned aerial vehicle, carrying out the complementation of a data missing value of the unmanned aerial vehicle through employing a nearest neighbor method, converting the trajectory segments into the grid map instead of directly employing the trajectory data, and carrying out the detection of the trajectory features of the unmanned aerial vehicle. The sensitivity of the unmanned aerial vehicle trajectory anomaly detection model to the trajectory is improved. In order to solve the problem of normal and abnormal data matching, random abnormity is generated for normal data; in order to improve the robustness of the model, a data enhancement technology is used, and Inceptionv3 is utilized to train a trajectory detection model. The accuracy of the unmanned aerial vehicle trajectory anomaly detection model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) anomaly detection, and particularly to a UAV trajectory feature anomaly detection method based on a grid map and Inceptionv3. Background Art

[0002] As an important field of research for countries in the future, UAV technology has applications in medical treatment, agriculture, factories, search and rescue, etc. With the annual increase in the sales volume and output value of UAVs, the requirements for their safety and reliability are also increasing. UAVs usually operate in complex environments, and their disadvantages such as lack of rapid response, limited size and power consumption also pose challenges to safety and reliability. The accident rate of UAVs is higher than that of manned aircraft. The flight state information of UAVs consists of flight data, including parameters such as altitude, speed, and attitude angle, which comprehensively reflect the operating state of key components of UAVs and the operation information of operators.

[0003] With the continuous accumulation of historical flight data of UAVs, processing and analyzing their abnormal data information has become the main means to evaluate UAV flight quality, analyze accident causes, improve work efficiency, and improve design. Therefore, researching an effective UAV flight data anomaly detection method is a necessary way to improve the safety performance of UAVs.

[0004] Chinese patent document CN117193241A discloses a "UAV autonomous cruise trajectory compliance anomaly detection method based on statistical analysis". It includes calculating the track deviation: determining the distance between the actual track and the ideal track according to the flight state of the UAV; using a statistical method to obtain the distribution characteristics of the track deviation, setting maximum and minimum threshold values, and establishing a track deviation anomaly detection model based on the statistical method; inputting the new track deviation into the anomaly detection model, and using the determined threshold range for judgment. If the input track deviation exceeds this threshold range, the track is considered abnormal. The above technical solution lacks the screening of the causes of track anomalies and the judgment of UAV faults. Summary of the Invention

[0005] The present invention mainly solves the technical problem that the original technical solution lacks the screening of the causes of track anomalies and the judgment of UAV faults, and provides a UAV trajectory feature anomaly detection method based on a grid map and Inceptionv3. It converts the three-dimensional trajectory data of UAVs into a two-dimensional grid map, improving the visualization of UAV flight trajectories. It uses flight segment data for trajectory anomaly detection, reducing the detection overhead. At the same time, it generates an abnormal trajectory data set, and combines data augmentation to improve the accuracy of the model for trajectory classification. It screens and eliminates track anomalies induced by external factors. The model trained using Inceptionv3 has low overhead and high accuracy, and is more suitable for the scenario of UAV trajectory anomaly detection.

[0006] The above technical problems of the present invention are mainly solved by the following technical solutions: The present invention includes the following steps: S1 Obtain the normal flight data sequence and abnormal flight data sequence of the UAV; S2 Divide the UAV flight data sequence into trajectory segment data according to the time window T and store it in a set; S3 Convert the coordinate data of the trajectory segment and perform standardization processing, assign it to the raster map data and draw the raster map; S4 Adjust the raster map data volume and perform data enhancement; S5 Divide the raster map data set and perform model training; S6 Generate a raster map for the UAV flight data collected every time window T, input it into the trained model, and judge the flight trajectory according to the output result.

[0007] For the normal flight data sequence NormalLoc and abnormal flight data sequence AbnormalLoc of the UAV, after dividing them into trajectory segment data according to the time window T, store them in the UAV trajectory segment set: NormalUAVTrajs = {TrajSeg1, TrajSeg2, …, TrajSeg i , …, TrajSeg n-1 , TrajSeg n} AbnormalUAVTrajs = {TrajSeg n+1 , TrajSeg n+2 , …, TrajSeg n+j , …, TrajSeg n+m-1 , TrajSeg n+m} Among them, the trajectory segment TrajSeg i stores the coordinate data TrajSeg i = {Loc (i-1)T , Loc (i-1)T+1 , …, Loc (i-1)T+j , …, Loc iT-1 , Loc iT}.

[0008] Traverse the normal flight data sequence NormalLoc and abnormal flight data sequence AbnormalLoc of the UAV, and for the missing values of longitude, latitude, and altitude {Lo i , La m , H m , H m} of each trajectory segment TrajSeg i are obtained according to the following formula (nearest neighbor interpolation): (Lo m ,La m ,H m )=(Lo nn ,La nn ,H nn )。

[0009] Preferably, step S3 specifically includes: S3.1 Obtain the missing values of longitude, latitude, and altitude {Lo m ,La m ,H m} for each trajectory segment; S3.2 Convert the longitude and latitude coordinates of the first coordinate data under each trajectory segment into planar coordinates, and store them together with the altitude in the second coordinate data; S3.3 Standardize the second coordinate data under each trajectory segment and store it in the standardized UAV flight data sequence; S3.4 Traverse the standardized UAV flight data sequence and assign the trajectory segments to the UAV flight grid dataset; S3.5 Traverse the UAV flight grid dataset, and draw the data of each grid map on a grid map and store it in a folder.

[0010] Preferably, step S3.2 specifically includes traversing the normal flight data sequence NormalLoc and the abnormal flight data sequence AbnormalLoc of the UAV. For the coordinate data Loc i under each trajectory segment TrajSeg i , convert the longitude and latitude coordinates Lo i , La i to planar coordinates (x i , y i ), and store them together with the altitude H i in the coordinate data Loc i of the converted UAV flight data.

[0011] For the coordinate data Loc i under each trajectory segment TrajSeg i , convert the longitude and latitude coordinates Lo i , La i to planar coordinates (x i , y i ) according to the following formula (Mercator projection formula), and store them together with the altitude H iStore the trajectory segments TrajSeg of the normal flight data sequence NormalTranTrajs of the converted UAV and the abnormal flight data sequence AbnormalTranTrajs of the converted UAV i coordinate data Loc i in: x i = R * Lo radi where R represents the radius of the earth, and x i represents the abscissa after conversion of the i-th coordinate data, and y i represents the ordinate after conversion of the i-th coordinate data.

[0012] Preferably, in step S3.3, traverse the normal flight data sequence NormalTranTrajs of the converted UAV and the abnormal flight data sequence AbnormalTranTrajs of the converted UAV, and for each trajectory segment TrajSeg i coordinate data Loc i latitude, longitude and altitude {x i , y i , H i} are normalized to {Normx i , Normy i , NormH i} and stored in the coordinate data Loc i of the normalized UAV flight data sequence.

[0013] According to the following formula, the coordinate data Loc i of each trajectory segment TrajSeg i latitude, longitude and altitude {x i , y i , H i} are normalized to {Normx i , Normy i , NormH i} and stored in the coordinate data Loc i of the trajectory segment TrajSeg of the normal flight data sequence NormalNormTrajs of the normalized UAV and the abnormal flight data sequence AbnormalNormTrajs of the normalized UAV i in: Normx i = (x i - Minx i ) * Size / (Maxx i-Minx i ) Normy i =(y i -Miny i )*Size / (Maxy i -Miny i ) NormH i =(H i -MinH i )*Size / (MaxH i -MinH i ) where Size represents the side length of the raster map, Minx i , Maxx i represent the minimum and maximum values of the abscissa sequence {x1, x2,..., x i ,..., x i , x n-1 , x n} in the trajectory segment TrajSeg i , Miny i , Maxy i represent the minimum and maximum values of the ordinate sequence {y1, y2,..., y i ,..., y n-1 , y n} in the trajectory segment TrajSeg i , MinH i , MaxH i represent the minimum and maximum values of the height sequence {H1, H2,..., H i ,..., H n-1 , H n} in the trajectory segment TrajSeg

[0014] Preferably, in step S3.4, the normal flight data sequence NormalNormTrajs of the drone after normalization and the abnormal flight data sequence AbnormalNormTrajs of the drone after normalization are traversed, and the data of the trajectory segment TrajSeg i under the corresponding data sequences are assigned to the normal flight raster dataset NormalGridMap of the drone and the abnormal flight raster dataset AbnormalGridMap of the drone.

[0015] Traverse the normal flight data sequence NormalNormTrajs of the drone after normalization and the abnormal flight data sequence AbnormalNormTrajs of the drone after normalization, and the trajectory segment TrajSegi The data is assigned to the normal flight grid dataset NormalGridMap and the abnormal flight grid dataset AbnormalGridMap of the UAV using the following formula: NormalGridMap ={GridMap1,GridMap2,…,GridMap i ,…,GridMap n-1 ,GridMap n} AbnormalGridMap ={GridMap n ,GridMap n+1 ,…,GridMap j ,…,GridMap n+m-1 ,GridMap n+m} point i ={CellPosition i ,CellValue i} CellPosition i =(Normx i ,Normy i ) CellValue i =NormH i GridMap i ={point1,point2,…,point i ,…,point n-1 ,point n}。

[0016] Preferably, step S3.5 specifically includes traversing the normal flight grid dataset NormalGridMap and the abnormal flight grid dataset AbnormalGridMap of the UAV, and drawing the data of each grid map on a grid map. GridMap i ={point1,point2,…,point i ,…,point n-1 ,point n} The size of the grid map is Size, and the horizontal and vertical coordinates of the i-th grid point are CellPosition i =(Normx i ,Normyi ), the gray value of the i-th grid point is CellValue i ; The grid map drawn using the normal flight grid dataset NormalGridMap of the UAV is stored in the normal folder, and the grid map drawn using the abnormal flight grid dataset AbnormalGridMap of the UAV is stored in the abnormal folder.

[0017] Preferably, the step S1 specifically includes obtaining the normal flight data sequence of the UAV NormalUAVLoc = {Loc1, Loc2, …, Loc i , …, Loc n-1 , Loc n} The abnormal flight data sequence of the UAV AbnormalUAVLoc = {Loc n+1 , Loc n+2 , …, Loc j , …, Loc n+m-1 , Loc n+m} Among them, for the i-th UAV coordinate data Loc i = {Lo i , La i , H i}, Lo i represents the longitude of the i-th coordinate of the UAV, La i represents the latitude of the i-th coordinate of the UAV, and H i represents the altitude of the i-th coordinate of the UAV.

[0018] Preferably, the step S4 specifically includes that if the data volume of the abnormal trajectory grid map is less than that of the normal trajectory grid map, a new grid map is generated in the abnormal folder in the following manner: First, copy a normal trajectory grid map GridMap i from the folder normal, and then replace several random points point i of the normal trajectory grid map GridMap i with abnormally generated points point Δi generated by random offset until the data volume of the abnormal trajectory grid map is equal to that of the normal trajectory grid map.

[0019] If the data volume of the abnormal trajectory grid map is less than that of the normal trajectory grid map, a new grid map is generated in the abnormal folder in the following manner: First, copy a normal trajectory grid map GridMap i from the folder normal, and then the normal trajectory grid map GridMapi A random number of points i are replaced with abnormal points generated by random offsets Δi until the data volume of the abnormal trajectory raster map is equal to that of the normal trajectory raster map: x Δi = x + Δx y Δi = y + Δy CellPosition Δi = (x Δi , y Δi ) CellValue Δi = H Δi point Δi = {CellPosition Δi , CellValue Δi} where x Δi , y Δi , H Δi are randomly generated offsets, ω is the angular frequency, and φ is the phase offset.

[0020] Traverse all raster maps and perform data augmentation on the raster map GridMap i using the following formula to generate a new rotation-augmented raster map GridMapr i , a horizontally flipped augmented raster map GridMapfh i , and a vertically flipped augmented raster map GridMapfv i : x t = x - Size / 2 y t = y - Size / 2 x t1 = x t * cos(θ) - y t * sin(θ) y t1 = x t * sin(θ) - y t * cos(θ) x r = x t1 + Size / 2 y r = y t1 + Size / 2 x fh = Size - x yfh = y x fv = x y fv = Size - y GridMapr i = {x r , y r , H} GridMapfh i = {x fh , y fh , H} GridMapfv i = {x fv , y fv , H} Where θ is the rotation angle of the picture, taking values of 90° or 180°.

[0021] Preferably, step S5 specifically includes loading the pre-trained InceptionV3 model, freezing the weights of InceptionV3, constructing a custom classifier on top of InceptionV3, with the activation function being sigmoid, and dividing the GridMap i dataset into a training set, a test set, and a validation set and using the InceptionV3 model for training. Set the parameters: target_size = (Size, Size) batch_size = 32 num_epochs = 100 class mode = 'binary'.

[0022] Preferably, step S6 specifically includes, when using the trained InceptionV3 model, generating a GridMap of the UAV flight data collected every time window T i , and inputting the GridMap i into the trained InceptionV3 model for classification. If the output result is normal, it is a normal trajectory; if the output result is abnormal, it is an abnormal trajectory.

[0023] The beneficial effects of the present invention are as follows: The three-dimensional trajectory data of the drone is converted into a two-dimensional raster map. First, a drone flight data set is obtained, including longitude, latitude, and altitude within a continuous time period. It is segmented according to a specified time length and a trajectory raster map is generated according to the segments, which improves the visualization of the drone flight trajectory. The flight segment data is used for trajectory anomaly detection, reducing the detection overhead. The segments are segmented to judge the drone trajectory, and the nearest neighbor method is used to fill in the missing values of the drone data. At the same time, the trajectory segments are converted into raster maps instead of directly using the trajectory data, which improves the sensitivity of the drone trajectory anomaly detection model to the trajectory. To solve the problem of the ratio of normal and abnormal data, random anomalies are generated for normal data; to improve the robustness of the model, data augmentation techniques are used, and the Inceptionv3 is used to train the trajectory detection model. This improves the accuracy of the drone trajectory anomaly detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the technical solutions of the present invention will be further described in detail below through embodiments and in conjunction with the drawings. It should be understood that the specific embodiments described herein are only the best embodiments of the present invention, which are only used to explain the present invention and do not limit the protection scope of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0026] There is information indicating that with the continuous accumulation of the historical flight data of drones, processing and analyzing their abnormal data information has become the main means to evaluate the flight quality of drones, analyze the causes of accidents, improve work efficiency, and improve designs. Therefore, researching effective methods for detecting abnormal drone flight data is a necessary way to improve the safety performance of drones.

[0027] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but it can also have additional steps not included in the drawings; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0028] UAV data has the characteristics of high dimensionality, few abnormal labels, and streaming data. Usually, various sensors and measurement devices are used to obtain UAV data, record parameters such as flight speed, acceleration, heading angle, and pitch angle during the UAV flight, and send them to the on-board recording system for real-time storage in a specific data format. UAV data anomalies are divided into three categories: point anomalies, context anomalies, and set anomalies. UAV data anomaly algorithms include three major categories: knowledge-based methods, model-based methods, and data-driven methods. Trajectory, as the most intuitive and characteristic data feature of UAVs, can reasonably and efficiently identify UAV anomalies, preparing for subsequent UAV maintenance and repair. Abnormal trajectories usually refer to those records in the dataset that do not conform to the random error law. They are generated by different mechanisms and often exhibit characteristics that are inconsistent with the expected normal motion pattern or are significantly different from most normal trajectories.

[0029] With the development of deep learning, deep learning models based on time series such as recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) are widely used in the work of processing and analyzing time series data. UAV trajectory data has the characteristics of time continuity and high variability, making it suitable for using LSTM for feature extraction. By training three-dimensional data, LSTM can predict the future trajectory of the UAV, and by comparing the predicted trajectory with the actual trajectory, anomalies can be discovered. However, LSTM mainly focuses on the local features of sequence data and lacks a global grasp of the overall trajectory. Especially in cases where the environment changes greatly or the trajectory data is complex, it is prone to misjudging abnormal trajectories. To overcome this limitation, the present invention proposes a UAV trajectory feature anomaly detection algorithm based on a grid map and Inceptionv3. Inception, also known as GoogLeNet, is a network with a CNN architecture. InceptionV3 mainly proposed decomposed convolutions, factoring large convolutions into small convolutions and asymmetric convolutions to improve network flexibility. After the UAV path is drawn using a grid map in this algorithm, an image classification network is then used for anomaly detection, which can ensure the accuracy of trajectory anomaly detection while taking into account the above problems.

[0030] The technical solution of the present invention will be further specifically described below through embodiments in conjunction with the accompanying drawings.

[0031] Embodiment: A method for detecting UAV trajectory feature anomalies based on a grid map and Inceptionv3 in this embodiment, as Figure 1 shown, includes the following steps: S1 Obtain the UAV normal flight data sequence and the UAV abnormal flight data sequence. Specifically, obtain the UAV normal flight data sequence NormalUAVLoc = {Loc1, Loc2, …, Loc i,…,Loc n-1 ,Loc n} Unmanned Aerial Vehicle (UAV) Abnormal Flight Data Sequence AbnormalUAVLoc = {Loc n+1 ,Loc n+2 ,…,Loc j ,…,Loc n+m-1 ,Loc n+m} Among them, for the i-th UAV coordinate data Loc i = {Lo i ,La i ,H i}, Lo i represents the longitude of the i-th coordinate of the UAV, La i represents the latitude of the i-th coordinate of the UAV, and H i represents the altitude of the i-th coordinate of the UAV.

[0032] S2 divides the UAV flight data sequence into trajectory segment data according to the time window T and stores it in a set.

[0033] For the normal flight data sequence NormalLoc and the abnormal flight data sequence AbnormalLoc of the UAV, after dividing them into trajectory segment data according to the time window T, store them in the UAV trajectory segment set: NormalUAVTrajs = {TrajSeg1, TrajSeg2,…, TrajSeg i ,…, TrajSeg n-1 , TrajSeg n} AbnormalUAVTrajs = {TrajSeg n+1 , TrajSeg n+2 ,…, TrajSeg n+j ,…, TrajSeg n+m-1 , TrajSeg n+m} Among them, the trajectory segment TrajSeg i stores the coordinate data within this segment TrajSeg i = {Loc (i-1)T , Loc (i-1)T+1 ,…, Loc (i-1)T+j ,…, Loc iT-1 , Loc iT}.

[0034] S3 converts the coordinate data of the trajectory segments, normalizes it, assigns it to the raster map data, and draws the raster map. Specifically, it includes: S3.1 Obtain the longitude, latitude, and altitude missing values {Lo m , La m , H m} for each trajectory segment.

[0035] Traverse the normal flight data sequence NormalLoc and the abnormal flight data sequence AbnormalLoc of the drone. For each trajectory segment TrajSeg i 's longitude, latitude, and altitude missing values {Lo m , La m , H m} are obtained according to the following formula (nearest neighbor interpolation): (Lo m , La m , H m ) = (Lo nn , La nn , H nn ).

[0036] S3.2 Convert the longitude and latitude coordinates of the first coordinate data under each trajectory segment into plane coordinates, and store them together with the altitude in the second coordinate data. Specifically, traverse the normal flight data sequence NormalLoc and the abnormal flight data sequence AbnormalLoc of the drone. For each trajectory segment TrajSeg i 's coordinate data Loc i 's longitude and latitude coordinates Lo i , La i are converted into plane coordinates (x i , y i ) according to the following formula (Mercator projection formula), and are stored together with the altitude H i in the coordinate data Loc i of the trajectory segment TrajSeg i in the converted normal flight data sequence NormalTranTrajs and the converted abnormal flight data sequence AbnormalTranTrajs of the drone: x i = R * Lo radi where R represents the radius of the earth, x i represents the abscissa after conversion of the i-th coordinate data, and y iRepresents the ordinate after the transformation of the i-th coordinate data.

[0037] S3.3 Standardize the second coordinate data under each trajectory segment and store it in the standardized UAV flight data sequence. Traverse the normal flight data sequence NormalTranTrajs of the UAV after transformation and the abnormal flight data sequence AbnormalTranTrajs of the UAV after transformation. According to the following formula, standardize the coordinate data Loc i of each trajectory segment TrajSeg i of longitude, latitude and altitude {x i , y i , H i} to {Normx i , Normy i , NormH i} and store it in the coordinate data Loc i of the trajectory segment TrajSeg i in the normal flight data sequence NormalNormTrajs of the UAV after standardization and the abnormal flight data sequence AbnormalNormTrajs of the UAV after standardization: Normx i = (x i - Minx i ) * Size / (Maxx i - Minx i ) Normy i = (y i - Miny i ) * Size / (Maxy i - Miny i ) NormH i = (H i - MinH i ) * Size / (MaxH i - MinH i ) Among them, Size represents the side length of the grid map, Minx i , Maxx i respectively represent the minimum value and the maximum value of the abscissa sequence {x1, x2,..., x i ,..., x i ,..., x n-1 , x n} in the trajectory segment TrajSeg i , Miny i , Maxy iThe minimum and maximum values of the vertical ordinate sequence {y1, y2,..., y i ,..., y n-1 , y n}, MinH i , MaxH i respectively represent the minimum and maximum values of the height sequence {H1, H2,..., H i in the trajectory segment TrajSeg i ,..., H n-1 , H n}.

[0038] S3.4 Traverse the standardized UAV flight data sequence and assign the trajectory segments to the UAV flight grid dataset. Traverse the normal flight data sequence NormalNormTrajs of the UAV after standardization and the abnormal flight data sequence AbnormalNormTrajs of the UAV after standardization, and assign the trajectory segments TrajSeg i data to the UAV normal flight grid dataset NormalGridMap and the UAV abnormal flight grid dataset AbnormalGridMap using the following formula: NormalGridMap ={GridMap1, GridMap2,…, GridMap i ,…, GridMap n-1 , GridMap n} AbnormalGridMap ={GridMap n , GridMap n+1 ,…, GridMap j ,…, GridMap n+m-1 , GridMap n+m} point i ={CellPosition i , CellValue i} CellPosition i =(Normx i , Normy i ) CellValue i =NormH i GridMap i ={point1, point2,…, pointi ,…, point n-1 , point n}。

[0039] S3.5 Traverse the UAV flight grid dataset, and draw each grid map data on a grid map and save it in a folder. Specifically, traverse the normal flight grid dataset NormalGridMap and the abnormal flight grid dataset AbnormalGridMap of the UAV. Each grid map data is drawn on a grid map, GridMap i ={point1, point2, …, point i ,…, point n-1 , point n}} The size of the grid map is Size, and the horizontal and vertical coordinates of the i-th grid point are CellPosition i =(Normx i , Normy i ), and the gray value of the i-th grid point is CellValue i ; The grid map drawn using the normal flight grid dataset NormalGridMap of the UAV is saved in the normal folder, and the grid map drawn using the abnormal flight grid dataset AbnormalGridMap of the UAV is saved in the abnormal folder.

[0040] S4 Adjust the amount of grid map data and perform data augmentation. Specifically, if the amount of abnormal trajectory grid map data is less than the amount of normal trajectory grid map data, generate a new grid map in the abnormal folder in the following way: First, copy a normal trajectory grid map GridMap from the normal folder i , and then replace several random points point i of the normal trajectory grid map GridMap i with abnormally generated points point Δi generated by random offset until the amount of abnormal trajectory grid map data is equal to the amount of normal trajectory grid map data: x Δi =x + Δx y Δi =y + Δy CellPosition Δi =(x Δi , y Δi ) CellValue Δi =H Δi point Δi ={CellPosition Δi ,CellValue Δi} where x Δi , y Δi , H Δi are randomly generated offsets, ω is the angular frequency, and φ is the phase offset.

[0041] Traverse all raster maps and perform data augmentation on the raster map GridMap i using the following formula to generate a new rotation-augmented raster map GridMapr i , a horizontally flipped augmented raster map GridMapfh i , and a vertically flipped augmented raster map GridMapfv i : x t = x - Size / 2 y t = y - Size / 2 x t1 = x t * cos(θ) - y t * sin(θ) y t1 = x t * sin(θ) - y t * cos(θ) x r = x t1 + Size / 2 y r = y t1 + Size / 2 x fh = Size - x y fh = y x fv = x y fv = Size - y GridMapr i ={x r , y r , H} GridMapfh i ={x fh , y fh , H} GridMapfv i ={x fv , y fv , H} Among them, θ is the rotation angle of the picture, and the value is 90° or 180°.

[0042] S5 divides the raster map dataset and conducts model training. Specifically, it includes loading the pre-trained InceptionV3 model, freezing the weights of InceptionV3, constructing a custom classifier on top of InceptionV3, with the activation function being sigmoid, and dividing the raster map GridMap i dataset into a training set, a test set, and a validation set and using the InceptionV3 model for training. Set the parameters: target_size=(Size,Size) batch_size = 32 num_epochs = 100 class mode ='binary'.

[0043] S6 generates a raster map from the collected UAV flight data every time window T, inputs it into the trained model, and determines the flight trajectory based on the output result. Specifically, when using the trained InceptionV3 model, a raster map GridMap is generated from the collected UAV flight data every time window T i , and the raster map GridMap i is input into the trained InceptionV3 model for classification. If the output result is normal, it is a normal trajectory; if the output result is abnormal, it is an abnormal trajectory.

[0044] Embodiment Step 1: Obtain the normal flight data sequence of the UAV NormalUAVLoc={Loc1,Loc2,…,Loc i ,…,Loc n-1 ,Loc n} and the abnormal flight data sequence of the UAV AbnormalUAVLoc={Loc n+1 ,Loc n+2 ,…,Loc j ,…,Loc n+m-1 ,Loc n+m} Among them, for the i-th UAV coordinate data Loc i ={Lo i ,La i ,H i}, where Lo i represents the longitude of the i-th coordinate of the UAV, and Lai Denote the latitude of the i-th coordinate of the UAV as H i Denote the altitude of the i-th coordinate of the UAV as H

[0045] Step 2: For the normal flight data sequence NormalLoc and abnormal flight data sequence AbnormalLoc of the UAV, segment them into trajectory segment data according to the time window T and store them in the UAV trajectory segment set: NormalUAVTrajs = {TrajSeg1, TrajSeg2, …, TrajSegi, …, TrajSeg n-1 , TrajSeg n} AbnormalUAVTrajs = {TrajSeg n+1 , TrajSeg n+2 , …, TrajSeg n+j , …, TrajSeg n+m-1 , TrajSeg n+m} Among them, the trajectory segment TrajSeg i Stores the coordinate data within this segment TrajSeg i = {Loc (i-1)T , Loc (i-1)T+1 , …, Loc (i-1)T+j , …, Loc iT-1 , Loc iT}.

[0046] Step 3: Traverse the normal flight data sequence NormalLoc and abnormal flight data sequence AbnormalLoc of the UAV, and for each trajectory segment TrajSeg i , obtain the missing values of longitude, latitude, and altitude {Lo m , La m , H m} according to the following formula (nearest neighbor interpolation): (Lo m , La m , H m ) = (Lo nn , La nn , H nn ) Step 4: Traverse the normal flight data sequence NormalLoc and abnormal flight data sequence AbnormalLoc of the UAV, and for each trajectory segment TrajSeg i , the longitude and latitude coordinates Lo of the coordinate data Loc i under iti , La i is converted into planar coordinates (x i , y i ) according to the following formula (Mercator projection formula), and the altitude H i is stored in the trajectory segment TrajSeg of the normal flight data sequence NormalTranTrajs of the converted UAV and the abnormal flight data sequence AbnormalTranTrajs of the converted UAV i coordinate data Loc i in: x i = R * Lo radi where R represents the radius of the earth, x i represents the abscissa after conversion of the i-th coordinate data, and y i represents the ordinate after conversion of the i-th coordinate data.

[0047] Step 5: Traverse the normal flight data sequence NormalTranTrajs of the converted UAV and the abnormal flight data sequence AbnormalTranTrajs of the converted UAV. According to the following formula, the coordinate data Loc i of each trajectory segment TrajSeg i latitude, longitude and altitude {x i , y i , H i} is normalized to {Normx i , Normy i , NormH i} and stored in the coordinate data Loc i of the trajectory segment TrajSeg of the normal flight data sequence NormalNormTrajs of the normalized UAV and the abnormal flight data sequence AbnormalNormTrajs of the normalized UAV i in: Normx i = (x i - Minx i ) * Size / (Maxx i - Minx i ) Normy i = (y i - Miny i ) * Size / (Maxy i - Miny i ) NormH i =(H i -MinH i )*Size / (MaxH i -MinH i ) where Size represents the side length of the raster map, Minx i , Maxx i represent the minimum and maximum values of the abscissa sequence {x1, x2,..., x i ,..., x i ,..., x n-1 , x n} in the trajectory segment TrajSeg i , Miny i , Maxy i represent the minimum and maximum values of the ordinate sequence {y1, y2,..., y i ,..., y n-1 , y n} in the trajectory segment TrajSeg i , MinH i , MaxH i represent the minimum and maximum values of the height sequence {H1, H2,..., H i ,..., H n-1 , H n} in the trajectory segment TrajSeg

[0048] Step 6: Traverse the normal flight data sequence NormalNormTrajs of the drone after normalization and the abnormal flight data sequence AbnormalNormTrajs of the drone after normalization, and assign the data of the trajectory segment TrajSeg i under the corresponding data sequences to the normal flight raster dataset NormalGridMap and the abnormal flight raster dataset AbnormalGridMap of the drone using the following formula: NormalGridMap ={GridMap1, GridMap2, …, GridMap i , …, GridMap n-1 , GridMap n} AbnormalGridMap ={GridMap n , GridMap n+1 , …, GridMap j , …, GridMap n+m-1,GridMap n+m} point i ={CellPosition i ,CellValue i} CellPosition i =(Normx i ,Normy i ) CellValue i =NormH i GridMap i ={point1,point2,…,point i ,…,point n-1 ,point n}.

[0049] Step 7: Traverse the normal flight grid dataset NormalGridMap and the abnormal flight grid dataset AbnormalGridMap of the UAV. Each grid map data GridMap i ={point1,point2,…,point i ,…,point n-1 ,point n} is drawn on a grid map. The size of the grid map is Size. The horizontal and vertical coordinates of the i-th grid point are CellPosition i =(Normx i ,Normy i ), and the gray value of the i-th grid point is CellValue i . The grid map drawn using the normal flight grid dataset NormalGridMap of the UAV is stored in the normal folder, and the grid map drawn using the abnormal flight grid dataset AbnormalGridMap of the UAV is stored in the abnormal folder.

[0050] Step 8: If the data volume of the abnormal trajectory grid map is less than that of the normal trajectory grid map, generate a new grid map in the abnormal folder in the following way: First, copy a normal trajectory grid map GridMap i from the folder normal, and then replace several random points point i of the normal trajectory grid map GridMap i with abnormal points point Δi generated by random offset until the data volume of the abnormal trajectory grid map is equal to that of the normal trajectory grid map: x Δi = x + Δx y Δi = y + Δy CellPosition Δi = (x Δi , y Δi ) CellValue Δi = H Δi point Δi = {CellPosition Δi , CellValue Δi} Wherein, x Δi , y Δi , H Δi are randomly generated offsets, ω is the angular frequency, and φ is the phase offset.

[0051] Step 9: Traverse all raster maps, and use the following formula to perform data augmentation on the raster map GridMap i to generate a new rotation-augmented raster map GridMapr i , a horizontally flipped augmented raster map GridMapfh i , and a vertically flipped augmented raster map GridMapfv i : x t = x - Size / 2 y t = y - Size / 2 x t1 = x t * cos(θ) - y t * sin(θ) y t1 = x t * sin(θ) - y t * cos(θ) x r = x t1 + Size / 2 y r = y t1 + Size / 2 x fh = Size - x y fh = y x fv = x y fv = Size - y GridMapr i = {x r , y r , H} GridMapfh i = {x fh , y fh , H} GridMapfv i = {x fv , y fv , H} Where θ is the rotation angle of the picture, taking values of 90° or 180°.

[0052] Step 10: Load the pre-trained InceptionV3 model, freeze the weights of InceptionV3, build a custom classifier on top of InceptionV3, and the activation function is sigmoid. Divide the GridMap i dataset into a training set, a test set, and a validation set and use the InceptionV3 model for training. Set the parameters: target_size = (Size, Size) batch_size = 32 num_epochs = 100 class mode = 'binary'.

[0053] Step 11: When using the trained InceptionV3 model, every time window T, generate the GridMap i from the collected UAV flight data according to Steps 1 - 7, and input the GridMap i into the trained InceptionV3 model for classification: If the output result is normal, it is a normal trajectory; if the output result is abnormal, it is an abnormal trajectory.

[0054] An abnormal detection algorithm for UAV trajectory features based on grid map and Inceptionv3 proposed by the present invention. Compared with general UAV trajectory detection algorithms, the main advantages of this method are as follows: segmenting fragments to judge the UAV trajectory, using the nearest neighbor method to complement the missing values of UAV data, and converting the trajectory fragments into grid maps instead of directly using the trajectory data, which improves the sensitivity of the UAV trajectory abnormal detection model to the trajectory. To solve the problem of normal-abnormal data ratio, random anomalies are generated for normal data; to improve the robustness of the model, data augmentation technology is used to train the trajectory detection model using Inceptionv3. This improves the accuracy of the UAV trajectory abnormal detection model.

[0055] Specific description: I. Data preprocessing Before drawing the grid map, preprocess the UAV normal trajectory and UAV abnormal trajectory data respectively. Segment the UAV flight data according to the time window, and use the nearest neighbor interpolation method to fill the missing values of the UAV flight data. After completing the data filling, convert the longitude and latitude into plane coordinates using the Mercator projection formula, and standardize the coordinate and altitude data of the UAV, and store them in the corresponding data structure. That is, the above steps 1, 2, 3, 4, 5, 6, 7.

[0056] II. Constructing a grid map dataset After completing the data cleaning, supplement the abnormal flight data using the random interpolation method according to the UAV normal flight data. Use the data to draw the grid map and store it in the normal trajectory folder and the abnormal trajectory folder. Use the data augmentation methods of enhancement and flipping for all grid maps. That is, the above steps 8, 9.

[0057] III. Training a trajectory abnormal detection model Use the constructed UAV trajectory grid map as the input and the folder name where the grid map is located as the label, and use Inceptionv3 for training. That is, the above steps 10, 11.

[0058] When obtaining the UAV flight dataset, it is necessary to segment and interpolate the data. The UAV trajectory data belongs to three-dimensional data, including longitude, latitude and altitude. General trajectory abnormal detection models are applicable to two-dimensional coordinate data, and the constructed trajectory abnormal detection model is not applicable to UAV trajectory detection. When the UAV is cruising and encounters a turning angle, the trajectory is prone to drastic changes, and the false detection rate of the abnormal detection model based on trajectory prediction will increase. Moreover, the UAV trajectory anomaly is manifested as an anomaly within a period of time, and only capturing point anomalies lacks an overall grasp.

[0059] Using the UAV trajectory feature anomaly detection algorithm based on grid map and Inceptionv3 in the present invention, after obtaining the UAV flight data set, interpolation is performed using the nearest neighbor method. Then, the trajectory data is sliced according to the specified time slice and a trajectory grid map is generated. Finally, classification, data generation, and data augmentation are performed according to the UAV trajectory type to construct a UAV trajectory grid map data set, and the Inceptionv3 training model is used.

[0060] This UAV trajectory feature anomaly detection algorithm based on grid map and Inceptionv3 converts the UAV three-dimensional trajectory data into a two-dimensional grid map, improving the visualization of the UAV flight trajectory. Using flight segment data for trajectory anomaly detection reduces the detection overhead. Generating an abnormal trajectory data set and combining data augmentation improve the accuracy of the model for trajectory classification. At the same time, the model trained using Inceptionv3 has low overhead and high accuracy, and is more suitable for the scenario of UAV trajectory anomaly detection.

[0061] The specific embodiments described in this article are only illustrative of the spirit of the present invention. The above embodiments only represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention patent. It should be noted that those skilled in the art of the present invention can make various modifications or supplements or use similar methods to replace the described specific embodiments, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims. For those of ordinary skill in the art, without departing from the concept of the present invention, multiple deformations and improvements can also be made. Therefore, the protection scope of the present invention patent should be subject to the appended claims.

Claims

1. A method for detecting abnormalities in drone trajectory features based on grid graphs and Inceptionv3, characterized in that: The following steps are involved: S1 obtains the normal flight data sequence and abnormal flight data sequence of the UAV; S2 divides the UAV flight data sequence into trajectory segment data according to the time window T and stores them into a collection; S3 converts and standardizes the coordinate data of the trajectory segment, assigns it to the raster map data and draws the raster map; S4 adjusts the amount of raster image data and performs data enhancement; S5 divides the raster map dataset and performs model training; S6 generates a grid map of the collected UAV flight data every time window T, inputs it into the trained model, and determines the flight trajectory based on the output results.

2. According to claim 1, a method for detecting abnormalities in drone trajectory features based on a grid map and Inceptionv3 is characterized in that: The step S3 specifically includes: S3.1 Get the missing values ​​of latitude, longitude and altitude of each trajectory segment {Lo m ,La m ,H m }; S3.2 converting the latitude and longitude coordinates of the first coordinate data of each track segment into plane coordinates, and storing them together with the height into the second coordinate data; S3.3 normalizes the second coordinate data of each trajectory segment and stores them in a standardized UAV flight data sequence; S3.4 traverses the standardized UAV flight data sequence and assigns the trajectory segments to the UAV flight raster dataset; S3.5 traverses the drone flight raster dataset, and each raster map data is drawn into a raster map and saved in a folder.

3. According to claim 2, a method for detecting anomaly of drone trajectory features based on grid graph and Inceptionv3 is characterized in that: The step S3.2 specifically includes traversing the normal flight data sequence NormalLoc and the abnormal flight data sequence AbnormalLoc of the drone, and each trajectory segment TrajSeg i The coordinate data Loc i The latitude and longitude coordinates of Lo i , La i Convert to plane coordinates (x i ,y i ), and the height H i Store the coordinate data of the converted drone flight data Loc i middle.

4. According to claim 3, the method for detecting abnormality of drone trajectory features based on grid graph and Inceptionv3 is characterized in that: The step S3.3 traverses the normal flight data sequence NormalTranTrajs of the converted UAV and the abnormal flight data sequence AbnormalTranTrajs of the converted UAV, and converts each trajectory segment TrajSeg i Coordinate data Loc i The latitude and longitude height {x i ,y i ,H i } is standardized to {Normx i ,Normy i ,NormH i } and store the coordinate data Loc of the standardized UAV flight data sequence i middle.

5. According to claim 4, a method for detecting abnormalities in drone trajectory features based on grid graph and Inceptionv3 is characterized in that: The step S3.4 specifically includes traversing the normal flight data sequence NormalNormTrajs of the standardized UAV and the abnormal flight data sequence AbnormalNormTrajs of the standardized UAV, and converting the trajectory segment TrajSeg under the corresponding data sequence i The data is assigned to the UAV normal flight raster dataset NormalGridMap and the UAV abnormal flight raster dataset AbnormalGridMap.

6. The method for detecting anomaly of drone trajectory features based on grid graph and Inceptionv3 according to claim 5 is characterized in that: The step S3.5 specifically includes traversing the drone normal flight grid data set NormalGridMap and the drone abnormal flight grid data set AbnormalGridMap, and drawing each grid map data on a grid map. GridMap i ={point1,point2,…,point i ,…,point n-1 ,point n } The grid size is Size, and the horizontal and vertical coordinates of the i-th grid point are CellPosition i =(Normx i ,Normy i ), the grayscale of the i-th grid point is CellValue i ; The grid map drawn using the UAV normal flight grid dataset NormalGridMap is stored in the normal folder, and the grid map drawn using the UAV abnormal flight grid dataset AbnormalGridMap is stored in the abnormal folder.

7. The method for detecting anomaly of drone trajectory features based on grid graph and Inceptionv3 according to claim 2 is characterized in that: The step S1 specifically includes obtaining a normal flight data sequence of the drone. NormalUAVLoc={Loc1,Loc2,…,Loc i ,…,Place n-1 ,Place n } Abnormal drone flight data sequence AbnormalUAVLoc={Loc n+1 ,Place n+2 ,…,Place j ,…,Place n+m-1 ,Place n+m } Among them, for the i-th drone coordinate data Loc i = {Lo i ,La i ,H i }, Lo i Indicates the longitude of the i-th coordinate of the drone, La i represents the latitude of the i-th coordinate of the drone, H i Indicates the altitude of the i-th drone.

8. The method for detecting anomaly of drone trajectory features based on grid graph and Inceptionv3 according to claim 6 is characterized in that: The step S4 specifically includes, if the amount of data in the abnormal track grid map is less than that in the normal track grid map, generating a new grid map in the abnormal folder in the following manner: first, copy a normal track grid map GridMap from the normal folder i , and then the normal trajectory grid map GridMap i A number of random points i Replace the outlier point generated by random offset Δi , until the amount of abnormal trajectory raster map data is equal to the amount of normal trajectory raster map data.

9. The method for detecting anomaly of drone trajectory features based on grid graph and Inceptionv3 according to claim 6 or 8, characterized in that: The step S5 specifically includes loading the pre-trained InceptionV3 model, freezing the weights of InceptionV3, building a custom classifier on top of InceptionV3, using the activation function as sigmoid, and converting the grid map GridMap i The dataset is divided into training set, test set and validation set and trained using the InceptionV3 model.

10. The method for detecting anomaly of drone trajectory features based on grid graph and Inceptionv3 according to claim 9 is characterized in that: The step S6 specifically includes, when using the trained InceptionV3 model, generating a grid map GridMap from the collected drone flight data every time window T i , the grid map GridMap i Input into the trained InceptionV3 model for classification. If the output result is normal, it is a normal trajectory. If the output result is abnormal, it is an abnormal trajectory.

Images