Unmanned aerial vehicle flight anomaly identification and early warning method and system based on deep learning

Abstract

This invention discloses a method and system for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning. The key technical points include: uniformly acquiring UAV formation data and ideal formation data; preprocessing the data to construct formation deviation and distortion features, obtaining a formation feature vector; inputting the formation feature vector into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features; based on the flight anomaly identification results and probability values, using the anomaly candidate features to determine candidate anomaly objects, using the anomaly candidate frames to calculate the anomaly start frame and deviation growth rate of the candidate anomaly objects, locating the anomaly source, and generating warning information. This addresses the problems of insufficient ability to identify formation-level anomalies such as formation distortion and spacing imbalance, difficulty in locating fault sources, and poor real-time performance in existing methods, improving the accuracy and timeliness of formation flight anomaly detection, and reducing the collision risk caused by formation anomalies and spacing imbalance.

Description

Technical Field

[0001] This invention relates to the field of drone flight anomaly recognition technology, and more specifically to a drone flight anomaly recognition and early warning method and system based on deep learning. Background Technology

[0002] Unmanned aerial vehicle (UAV) swarm flight is increasingly used in fields such as inspection, surveying, and collaborative operations, but its flight safety and status monitoring still face many challenges. Existing methods mostly focus on detecting faults in the position, attitude, or sensors of individual UAVs, with less consideration for the relative constraints and spatial topology between UAVs within the swarm. This makes it difficult to effectively identify swarm-level faults such as formation distortion and abnormal spacing caused by the gradual propagation of individual UAV deviations. Furthermore, during dense swarm collaborative flight, if formation anomalies occur and are not identified in time, they can easily lead to a rapid imbalance in the relative spacing between UAVs, disrupting safe distances, increasing the risk of collisions, seriously threatening swarm flight safety, and even triggering cascading failures and overall mission failure.

[0003] Meanwhile, most existing methods only achieve anomaly identification and detection, lacking the ability to locate the source drone of the anomaly, making it difficult to distinguish between single-drone independent anomalies and formation propagation and diffusion anomalies. Furthermore, some deep learning-based solutions have a large number of model parameters and slow inference speed, making it difficult to meet the requirements of reliable, timely, and traceable anomaly early warning and decision support in actual formation operations. Therefore, existing technologies have shortcomings. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning. This method identifies UAV flight anomalies by using a temporal-spatial fusion deep learning model and locates the source of the anomaly by combining anomaly candidate frames and anomaly candidate features, thereby achieving anomaly identification and source tracing warning. It has the advantages of accurate identification and strong practicality.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A deep learning-based method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies includes:

[0007] Unified acquisition of UAV formation data and ideal formation data, preprocessing and constructing formation deviation and distortion features to obtain formation feature vectors;

[0008] The formation feature vector is input into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features;

[0009] Based on the flight anomaly identification results and probability values, candidate anomaly objects are determined using the anomaly candidate features, and the anomaly start frame and deviation growth rate of the candidate anomaly objects are calculated using the anomaly candidate frames to locate the anomaly source and generate early warning information.

[0010] The flight anomaly identification results are normal, position anomaly, and formation anomaly. The candidate anomaly objects are the set of UAVs to be inspected when the position is abnormal and the core candidate UAV units when the formation is abnormal.

[0011] As a further improvement of the present invention, the preprocessing includes:

[0012] Acquire drone formation data, including the three-dimensional position of each drone and the relative distance between any two drones;

[0013] Obtain ideal formation data, including the ideal formation position of each drone and the ideal formation distance between any two drones;

[0014] A timestamp synchronization mechanism is used to align the UAV formation data with the ideal formation data to form raw formation data in single-frame format;

[0015] The original formation data is interpolated, filtered, denoised, and standardized to obtain standardized formation data.

[0016] As a further improvement of the present invention, the step of constructing formation deviation and distortion features to obtain formation feature vectors includes:

[0017] Construct single-machine hierarchical features and calculate single-machine positional deviation features;

[0018] Construct inter-machine hierarchical features and inter-computer spacing deviation features;

[0019] Construct the overall hierarchical features of the formation, and calculate the formation center offset features and formation distortion features;

[0020] All the above features are concatenated in a preset order to form a fixed-dimensional array feature vector.

[0021] As a further improvement of the present invention, the formation feature vector is input into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features, including:

[0022] Continuous The frame array feature vectors form the input feature vector, which is then input into a pre-trained temporal deep learning model. The model includes an input layer, a temporal feature extraction layer, a spatial correlation fusion layer, an anomaly detection layer, and an output layer.

[0023] The input feature vector is input into the temporal feature extraction layer to extract temporal features frame by frame, resulting in a weighted temporal feature vector and anomaly candidate frames;

[0024] The input feature vector and the weighted temporal feature vector are input into the spatial correlation fusion layer for feature fusion to obtain a fused feature vector and anomaly candidate features;

[0025] The fused feature vector is input into the anomaly detection layer, and the probability values ​​of whether the formation is normal, abnormal in position, or abnormal in formation are output through multi-classification operation. The category with the highest probability value is taken as the recognition result.

[0026] The output layer outputs the flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features.

[0027] As a further improvement of the present invention, the input feature vector is input into the temporal feature extraction layer to extract temporal features frame by frame, resulting in a weighted temporal feature vector and anomaly candidate frames, including:

[0028] The input feature vector is input into a gated recurrent unit network to extract temporal features frame by frame, resulting in a temporal feature vector;

[0029] Attention operations are performed on the temporal feature vector to obtain attention weights for each frame, and then weighted and fused to obtain a weighted temporal feature vector;

[0030] Frames with attention weights greater than a preset anomaly threshold are independently selected and recorded as anomaly candidate frames. These anomaly candidate frames are only used for subsequent anomaly source localization and do not participate in weighted fusion.

[0031] As a further improvement of the present invention, the input feature vector and the weighted temporal feature vector are input to a spatial correlation fusion layer for feature fusion to obtain a fused feature vector and anomaly candidate features, including:

[0032] Spatial correlation features are extracted from the input feature vector and then concatenated and fused with the weighted temporal feature vector in the feature dimension to obtain a fused feature vector.

[0033] The fused feature vector is compressed in dimension through a fully connected layer to output a compressed feature vector.

[0034] In parallel statistical input feature vectors, each feature dimension is The weighted activation sum within the frame time window is sorted by the weighted activation sum, and the features corresponding to the top-ranked feature dimensions are recorded as anomalous candidate features.

[0035] As a further improvement to the present invention, locating the source of the anomaly based on the flight anomaly identification results and probability values ​​also includes:

[0036] When the flight anomaly identification result is normal, no positioning is performed;

[0037] When the flight anomaly identification result is a position anomaly or a formation anomaly, determine whether the probability value is greater than or equal to a preset confidence threshold.

[0038] If the probability value is less than the confidence threshold, it is determined to be an unreliable identification and no positioning is performed;

[0039] If the probability value is greater than or equal to the confidence threshold, then proceed to the process of locating the source of the anomaly.

[0040] As a further improvement of the present invention, when the flight anomaly identification result is a location anomaly and the process enters the location anomaly source process, the location anomaly source includes:

[0041] Based on the associated objects of the abnormal candidate features, the set of UAVs to be inspected is obtained;

[0042] Based on the abnormal candidate frames, the abnormal start time and deviation change trend are determined for each UAV in the UAV collection to be inspected, and the abnormal start frame and deviation growth rate are obtained.

[0043] Drones with a deviation growth rate greater than zero are retained as valid abnormal drones, and all drones that are ultimately retained are identified as all abnormal drones at that location.

[0044] As a further improvement of the present invention, when the flight anomaly identification result is a formation anomaly and the process enters the location anomaly source process, the location anomaly source includes:

[0045] The formation anomaly type is determined based on the associated objects of the anomaly candidate features, wherein the formation anomaly type includes inter-machine spacing anomaly and overall formation anomaly;

[0046] When the inter-machine spacing is determined to be abnormal, the core candidate UAV unit is determined based on the UAV pair in the abnormal candidate features. The source UAV is obtained by combining the abnormal start frame and deviation growth rate of the UAV in each unit.

[0047] When the formation is determined to be abnormal as a whole, the formation reference drone is selected as the source drone.

[0048] As a further improvement of the present invention, the step of determining the core candidate UAV unit based on the UAV pair in the abnormal candidate features, and obtaining the source UAV by combining the UAV's own abnormal start frame and deviation growth rate within each unit, includes:

[0049] An undirected graph is constructed based on the drone pairs in the abnormal candidate features, and the connected components in the graph are used as the core candidate drone units.

[0050] For each core candidate UAV unit, calculate the individual abnormal starting frame and deviation growth rate of each UAV within the unit:

[0051] If the unit contains only one pair of drones, the source drone is determined by combining the timing relationship between the drone's abnormal start frame and its own abnormal start frame.

[0052] If the unit contains multiple drone pairs, the drone with the earliest abnormal start frame is identified as the source drone. If the start frames are the same, the drone with the largest deviation growth rate is selected as the source drone.

[0053] A deep learning-based UAV flight anomaly identification and early warning system, used to implement the aforementioned deep learning-based UAV flight anomaly identification and early warning method, includes:

[0054] Data acquisition and processing module: used to uniformly acquire drone formation data and ideal formation data, and obtain standardized formation data through preprocessing;

[0055] Formation feature construction module: used to construct formation deviation and distortion features based on the standardized formation data, and obtain formation feature vector;

[0056] Flight anomaly identification module: used to input the formation feature vector into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features;

[0057] The positioning and early warning module is used to determine candidate anomaly objects based on the flight anomaly identification results and probability values, use the anomaly candidate features to determine the anomaly starting frame and deviation growth rate of the candidate anomaly objects, use the anomaly candidate frames to calculate the anomaly starting frame and deviation growth rate of the candidate anomaly objects, locate the source of the anomaly, and generate early warning information.

[0058] The beneficial effects of this invention are as follows: By standardizing UAV formation data and ideal formation data, a multi-level formation feature from single UAV to the whole is constructed. Then, a lightweight temporal deep learning model is used to identify abnormal states. Combined with abnormal candidate frames and abnormal candidate features, the source of the abnormality is located. This effectively solves the technical problems of insufficient ability to identify formation-level anomalies such as formation distortion and spacing imbalance in existing UAV flight anomaly identification and early warning methods, difficulty in locating the source of the fault, and poor real-time performance. It can improve the accuracy and timeliness of formation flight anomaly detection, reduce the collision risk caused by formation anomalies and spacing imbalance, and provide strong protection for the safe, stable, and reliable collaborative flight of UAV formations. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating the steps of the deep learning-based UAV flight anomaly identification and early warning method of the present invention.

[0060] Figure 2 This is a schematic diagram of the temporal deep learning model of the present invention;

[0061] Figure 3 This is a flowchart illustrating the steps of abnormal single-machine positioning when the location is abnormal according to the present invention.

[0062] Figure 4 This is a flowchart illustrating the steps for locating the source drone when its formation is abnormal, as described in this invention.

[0063] Figure 5 This is a schematic diagram of the structure of the deep learning-based UAV flight anomaly identification and early warning system of the present invention. Detailed Implementation

[0064] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof.

[0065] Identical parts are indicated by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "up," and "down" used in the following description refer to directions in the accompanying drawings, while the terms "bottom surface," "top surface," "inner," and "outer" refer to directions toward or away from the geometric center of a specific part, respectively.

[0066] The term "and / or" in the following text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0067] Example 1

[0068] like Figure 1 As shown, this embodiment of the invention provides a method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning, including:

[0069] Unified acquisition of UAV formation data and ideal formation data, preprocessing and constructing formation deviation and distortion features to obtain formation feature vectors;

[0070] The formation feature vector is input into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features;

[0071] Based on the flight anomaly identification results and probability values, candidate anomaly objects are determined using anomaly candidate features, and the anomaly start frame and deviation growth rate of the candidate anomaly objects are calculated using anomaly candidate frames to locate the anomaly source and generate early warning information.

[0072] The flight anomaly identification results are normal, position anomaly, and formation anomaly. The candidate anomaly objects are the set of UAVs to be inspected when the position is abnormal and the core candidate UAV units when the formation is abnormal.

[0073] Specifically, in drone formation flight scenarios, unified drone formation data and ideal formation data are collected. Through time synchronization, missing value imputation, filtering and noise reduction, and dimensional standardization, standardized formation data is obtained. Based on this data, multi-level features including single-drone position deviation, inter-drone spacing deviation, formation center offset, and formation distortion degree are constructed to form a fixed-dimensional formation feature vector. Subsequently, the formation feature vector of multiple consecutive frames is input into a lightweight temporal deep learning model. Temporal feature extraction, attention weighting, and spatial feature fusion are used to identify three types of states: normal, positional abnormal, and formation abnormal, and anomaly candidate frames and anomaly candidate features are output. Finally, based on the anomaly identification results and probability values, it is determined whether to initiate the localization process. The scope of the anomaly object is determined based on the anomaly candidate features, and the start time and trend of the anomaly are determined based on the anomaly candidate frames. For positional anomalies, abnormal single drones with continuous offset are identified. For formation anomalies, the inter-drone spacing anomalies and overall formation anomalies are distinguished and the source drone of the distortion is located. Finally, early warning information is output to achieve real-time identification, localization, and safety warning of drone formation flight anomalies.

[0074] This embodiment standardizes UAV formation data and ideal formation data to construct multi-level formation features from individual UAVs to the whole system. Then, it uses a lightweight temporal deep learning model to identify abnormal states and combines abnormal candidate frames and abnormal candidate features to locate the UAV at the source of the abnormality. This effectively solves the technical problems of existing UAV flight anomaly identification and early warning methods, such as insufficient ability to identify formation-level anomalies like formation distortion and spacing imbalance, difficulty in locating the source of the fault, and poor real-time performance. It can significantly improve the accuracy and timeliness of formation flight anomaly detection, effectively reduce the collision risk caused by formation anomalies and spacing imbalance, and provide strong protection for the safe, stable, and reliable collaborative flight of UAV formations.

[0075] Furthermore, this embodiment provides a preprocessing step, including:

[0076] Acquire drone formation data, including the three-dimensional position of each drone and the relative distance between any two drones;

[0077] Obtain ideal formation data, including the ideal formation position of each drone and the ideal formation distance between any two drones;

[0078] A timestamp synchronization mechanism is used to align the UAV formation data with the ideal formation data to form raw formation data in single-frame format;

[0079] The original formation data is interpolated, filtered, denoised, and standardized to obtain standardized formation data.

[0080] Specifically, unified data collection is performed on each drone in the drone formation. All data is acquired in real time through a standard communication interface, and data collection is carried out at preset intervals. As an example, this embodiment uses a data update frequency of 200Hz, corresponding to an acquisition interval of 5ms, which can meet the flight monitoring requirements of the drone formation. The drone formation data specifically includes the three-dimensional position of the drones, where the three-dimensional position includes latitude, longitude, and relative altitude. Based on the collected multi-dimensional data of each drone, the relative distance between any two drones is calculated in real time using three-dimensional Euclidean distance, reflecting the relative positional relationship between the drones within the formation.

[0081] The ideal formation data for drone swarms is obtained, including ideal formation positions and ideal formation spacing. The ideal formation position is defined as the ideal longitude, ideal latitude, and ideal relative altitude of each drone, while the ideal formation spacing is the ideal distance between any two drones. The data collection frequency is the same as the frequency of collecting multidimensional drone data.

[0082] All collected data are synchronized using a timestamp mechanism, with the flight control computer providing unified timing. This collectively constitutes the original formation data, which is continuously cached in a fixed format. The fixed format for a single frame is: [timestamp, formation ID, 3D position, relative spacing, ideal position, ideal spacing]. The 3D position, relative spacing, ideal position, and ideal spacing are all arranged in the order of each UAV's ID number, ensuring that the original formation data format for a single frame is consistent.

[0083] The collected raw formation data undergoes preprocessing to eliminate data noise, standardize data scale, and construct standardized time-series samples. Specifically: the raw formation data is reordered in ascending order of timestamps; the timestamp intervals are checked to ensure they match the preset acquisition intervals, and valid frames are retained; any missing data is checked and filled using linear interpolation to ensure the raw formation information is evenly spaced and without gaps. Then, the individual aircraft's 3D position and the relative spacing of the formation are filtered using a sliding window filter with a 5-point sliding mean, meaning the sliding window length is 5 and the window sliding step is 1 frame. If the current frame is the previous 2 frames or the next 2 frames (i.e., there are no corresponding frames), zero-padding is used to fill in the window data, ensuring the filter covers all frames while avoiding filtering distortion, suppressing high-frequency sensor jitter and impulse noise, and preserving abnormal flight state characteristics. To standardize the units of measurement, min-max standardization is used to map each data point to the [0,1] interval, where min and max are the lower and upper limits of the factory-calibrated range of the corresponding parameter acquisition equipment, respectively. The specific values ​​are determined based on the equipment model and specifications, and this embodiment does not impose any limitations on this. The resulting standardized formation data is then obtained.

[0084] This embodiment collects UAV formation data and ideal formation data in a unified manner, and adopts timestamp synchronization, linear imputation of missing values, moving average filtering and standardized processing procedures. This can effectively suppress sensor noise, eliminate data jitter and missing data, and ensure that the data format is uniform, the units are consistent and the time sequence is complete. It not only preserves the abrupt change characteristics of flight anomalies, but also improves the quality and stability of subsequent model input data. It significantly reduces the problem of decreased anomaly recognition accuracy caused by data noise, and provides a reliable data foundation for subsequent formation anomaly recognition and early warning.

[0085] Furthermore, this embodiment provides a step for constructing formation deviation and distortion features to obtain formation feature vectors, including:

[0086] Construct single-machine hierarchical features and calculate single-machine positional deviation features;

[0087] Construct inter-machine hierarchical features and inter-computer spacing deviation features;

[0088] Construct the overall hierarchical features of the formation, and calculate the formation center offset features and formation distortion features;

[0089] All the above features are concatenated in a preset order to form a fixed-dimensional array feature vector.

[0090] Specifically, based on standardized formation data, formation deviation and distortion characteristics are constructed.

[0091] The calculation of single-unit position deviation characteristics is performed for each drone in the formation, denoted as drone. , , To determine the total number of drones in the formation, extract the preprocessed data: drones The three-dimensional position ( , , Ideal location ( , , ),in, , , These are the longitude, latitude, and relative altitude of the drone. , , These represent the ideal longitude, ideal latitude, and ideal relative altitude of the drone. (Calculate the drone's...) The deviation of the three-dimensional position from the ideal position The formula for representing the overall deviation of a single unit in space is: , The larger the value, the more severe the deviation of the single drone from the ideal position. Calculate the horizontal position deviation of drone i. and vertical position deviation The formulas are as follows: , It is used to distinguish the deviation of a single machine in different directions, and to assist in subsequent anomaly detection.

[0092] The inter-drone spacing deviation characteristic calculation is performed for any two drones within the formation, denoted as drones. and drones , ,and Extracting preprocessed data: Drone and relative spacing Ideal spacing Absolute deviation of distance between computers The formula is: ;in The larger the value, the more serious the deviation of the distance between the two machines from the ideal distance, and the higher the risk of collision or formation distortion. (Relative deviation of distance between computers) This is used to characterize the relative degree of spacing deviation, and the formula is: ,in Calculate the average deviation of the overall formation spacing. The formula for representing the deviation of the inter-aircraft spacing of the entire formation from the overall horizontal level is: , The larger the value, the more chaotic the spacing between aircraft in the entire formation.

[0093] Formation center offset feature calculation: For all UAVs, calculate the formation geometric center, which includes the longitude center. Latitude center center of height Similarly, calculate the ideal geometric center. , , ,in, For the ideal longitude center, For the ideal latitude center, Assuming the ideal altitude center, then calculate the formation center offset. , The larger the value, the more severely the formation deviates from its ideal position. Calculate the horizontal offset of the formation center. and vertical offset The formulas are as follows: , It is used to distinguish the overall deviation of the formation in the horizontal and vertical directions.

[0094] The formation distortion characteristic is calculated by combining formation flight characteristics and setting a weighting coefficient for individual aircraft position deviation. Weighting coefficient for machine spacing deviation formation center offset weighting coefficient ,satisfy Example value =0.3、 =0.4、 =0.3, substitute each weighting coefficient into the calculation of formation distortion degree. The formula is: , The larger the value, the more severe the formation distortion. To further adapt to different formation scenarios, experts can analyze the formation distortion degree of each frame of data. A score ranging from 0 to 1 is assigned. A regression model is established based on the formula for calculating formation distortion. The weighted coefficients obtained from the regression model are permanently saved for direct use in actual monitoring without the need for online adjustment. This embodiment does not limit the specific regression method and coefficient values, but they should be determined offline based on expert scoring data.

[0095] The features corresponding to each frame of the formation's raw data include: the individual position deviation of each drone ( , , ), the inter-drone spacing deviation of each pair of drones ( , ), mean deviation of overall formation spacing ( ), formation center offset ( , , ), formation distortion ( All features are dimension-unified. The individual features of each drone and the relative features of each pair of drones are arranged in drone ID order to ensure a fixed feature dimension for each time-series sample. All the normalized features are then concatenated in the order of individual features, relative features, and overall features to form a formation feature vector with dimension []. ].

[0096] This embodiment constructs a multi-dimensional feature system at three levels: individual drone, inter-drone, and overall formation. This system includes features such as positional deviation, spacing deviation, formation center offset, and formation distortion. These features are then concatenated into a unified-dimensional formation feature vector according to fixed rules. This feature structure can characterize the spatial deviation and formation changes of UAV formations, fully extracting key information such as individual drone offset, abnormal inter-drone spacing, and overall formation distortion. This provides deep learning models with highly expressive and discriminative input features. Compared with existing methods, this approach effectively solves the problem of single-dimensional features that are difficult to fully represent the mechanisms of formation anomalies, significantly improving the accuracy of subsequent anomaly identification and the reliability of fault source localization.

[0097] Furthermore, this embodiment provides a step of inputting formation feature vectors into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features, including:

[0098] Continuous The frame array feature vectors form the input feature vector, which is then input into a pre-trained temporal deep learning model. The model includes an input layer, a temporal feature extraction layer, a spatial correlation fusion layer, an anomaly detection layer, and an output layer.

[0099] The input feature vector is fed into the temporal feature extraction layer to extract temporal features frame by frame, resulting in a weighted temporal feature vector and anomaly candidate frames.

[0100] The input feature vector and the weighted temporal feature vector are input into the spatial correlation fusion layer for feature fusion to obtain the fused feature vector and anomaly candidate features.

[0101] The fused feature vector is input into the anomaly detection layer, and the multi-classification operation outputs the probability value of whether the formation is normal, abnormal in position, or abnormal in formation. The category with the highest probability value is taken as the recognition result.

[0102] The output layer outputs the flight anomaly identification results, corresponding probability values, anomaly candidate frames, and anomaly candidate features.

[0103] Specifically, a lightweight temporal deep learning network is used, taking the formation feature vector as input, to identify abnormal states in formation flight. Specifically, it selects... Frame formation feature vectors are used to form the input feature vector. ,in, For the first Frame formation feature vectors .

[0104] like Figure 2 As shown, the temporal deep learning model includes an input layer, a temporal feature extraction layer, a spatial correlation fusion layer, an anomaly detection layer, and an output layer. The input layer receives the input feature vector and unifies it into a fixed [1, ...]. , ]Dimension, among which, The vector dimension of the single-frame formation feature vector is... .

[0105] This embodiment employs a lightweight temporal deep learning network and uses continuous multi-frame formation feature vectors as model input. It can fully utilize the temporal dependencies and dynamic changes of UAV formation flight to effectively capture temporal features such as gradual deviation and anomaly propagation. Compared with single-frame judgment, it can better identify early weak anomalies and trend anomalies. At the same time, the fixed-dimensional temporal input structure ensures the stability and efficiency of model calculation, improving the accuracy of anomaly state identification and anti-interference ability.

[0106] Furthermore, this embodiment provides a step of inputting an input feature vector into a temporal feature extraction layer to extract temporal features frame by frame, obtaining a weighted temporal feature vector and anomaly candidate frames, including:

[0107] The input feature vector is fed into a gated recurrent unit network to extract temporal features frame by frame, resulting in a temporal feature vector.

[0108] Attention operations are performed on the temporal feature vectors to obtain the attention weights for each frame, and then weighted and fused to obtain a weighted temporal feature vector;

[0109] Frames with attention weights greater than a preset anomaly threshold are independently selected and recorded as anomaly candidate frames. These anomaly candidate frames are only used for subsequent anomaly source localization and do not participate in weighted fusion.

[0110] The temporal feature extraction layer employs a single-layer lightweight gated recurrent unit (GRU) network, targeting the input feature vector. The design adapts to the temporal continuity of frames and the gradual / abrupt characteristics of deviations, replacing the traditional Long Short-Term Memory (LSTM) network to reduce the number of parameters and improve inference speed. The GRU network has 64 hidden layer neurons. This embodiment uses the classic GRU gating logic without modifying the calculation formula. Specifically: the reset gate captures abrupt changes in single-machine position deviation and inter-machine spacing deviation in the formation feature vector; the update gate filters and retains effective temporal information strongly correlated with anomaly detection; and candidate hidden states are extracted frame by frame. The temporal features within the frame timing window are finally output frame by frame in the hidden state. After GRU gating, the temporal feature vector is obtained from the input feature vector. Each of them It is a 64-dimensional feature vector. This achieves the transformation from input feature vectors to temporal feature vectors. Addressing the characteristic that formation anomalies often manifest as sudden increases in deviation over a few frames, [the following is implemented / implemented]: Attention operations are performed on the hidden states output by the GRU frame. , ,in, The attention weight vector is obtained during the pre-training phase. It is a natural exponential function. For the first Frame attention weights The final output is a weighted time series feature vector with dimensions [1, 64].

[0111] Specifically, here, abnormal candidate frames are filtered based on attention weights and a preset anomaly threshold, and only records are made. >Frames with abnormal thresholds Identify abnormal candidate frames. Attention weights. The larger the value, the higher the contribution of the corresponding frame's features to anomaly detection and the stronger the correlation; setting The anomaly threshold can filter out invalid frames with low contribution and retain only frames that play a key role in anomaly identification. Therefore, these frames can be used as anomaly candidate frames to provide core basis for locating the start time of the anomaly. At the same time, a large number of irrelevant frames are removed to avoid introducing additional computation and improve overall processing efficiency.

[0112] This embodiment employs a lightweight GRU network to extract features from multiple consecutive frames in a temporal sequence, and combines this with an attention mechanism to achieve weighted fusion of temporal features. This approach not only fully captures the gradual and abrupt changes in deviations during formation flight but also highlights the contribution of key anomalous frames through attention weights. Furthermore, the lightweight GRU structure contains only two gating units—a reset gate and an update gate—reducing the number of parameters compared to traditional LSTM and simplifying the gating logic, thereby improving single-batch inference speed and meeting the requirements for real-time detection of UAV formations. By using attention weights to label anomalous candidate frames, the key time periods of anomaly occurrence can be located, providing a basis for subsequent UAV localization and improving the reliability, timeliness, and interpretability of anomaly identification.

[0113] Furthermore, this embodiment provides a step of inputting an input feature vector and a weighted temporal feature vector into a spatial correlation fusion layer for feature fusion to obtain a fused feature vector and anomaly candidate features, including:

[0114] Spatial correlation features are extracted from the input feature vector and then concatenated and fused with the weighted temporal feature vector along the feature dimension to obtain the fused feature vector.

[0115] The fused feature vector is compressed in dimension through a fully connected layer to output a compressed feature vector.

[0116] In parallel statistical input feature vectors, each feature dimension is The weighted activation sum within the frame time window is sorted by the weighted activation sum, and the features corresponding to the top-ranked feature dimensions are recorded as anomalous candidate features.

[0117] Specifically, the spatial correlation fusion layer, taking into account the characteristic that the input vector contains features from three layers—individual, inter-drone, and overall formation—designs convolutional and concatenation fusion operations. The core parameters are a 3×3 convolutional kernel, 32 output channels, and a stride of 1. For the input feature vector, it extracts the spatial correlation features that couple between individual drone position deviations, inter-drone spacing deviations, and overall formation deviations. For example, when the individual position deviation of a drone increases, its inter-drone spacing deviation with adjacent drones usually changes synchronously. This linkage pattern can be captured and encoded into the output channel through the cross-feature dimension weights of the convolutional kernel. This yields the spatial feature vector. ,in, For the first Frame number Feature values ​​of each convolution output channel For activation function, For convolution kernel weights, For convolution bias, and Through pre-training and solidification Number the output channels of the convolution. , For the first Frame number dimensional original eigenvalues, The sliding stride is the temporal dimension of the convolutional kernel. , The input feature dimension number is used. The final output is a spatial feature vector with dimensions [1, ...].

[32] . The output of the temporal feature extraction layer Remodeled into [1,1,64] dimensions, with the spatial feature tensor [1,

[32] By concatenating the features along the feature dimensions, a fused feature vector is obtained, with dimensions [1, 32].

[96] , which combines temporal regularity and spatial correlation characteristics. The feature dimension is compressed through a fully connected layer, outputting a compressed feature vector. , dimension [1, ,32).

[0118] Specifically, feature contribution statistics are performed in parallel during convolution operations, iterating through all feature dimensions. Statistical analysis of this feature Intra-frame weighted activation summation ,according to Sort the features from largest to smallest, select the top 3 largest feature dimensions as high-contribution feature dimensions, and record their indices. The system identifies candidate features for anomalies. By summing the activation values ​​of each feature dimension, the dimensions that contribute most to the extraction of spatial correlation features can be quickly selected from the high-dimensional features. These dimensions directly correspond to core anomaly features such as single-unit position deviation and inter-unit spacing deviation, and therefore can be used as candidate features for anomalies, providing core evidence for subsequent location of the source UAV. Here, 3 is example data, and the specific data can be set by those skilled in the art as needed. This embodiment does not limit this.

[0119] This embodiment extracts spatial features by convolving the input feature vector through a spatial correlation fusion layer, and then concatenates these features with weighted temporal features to achieve a deep fusion of temporal variation patterns and spatial correlation features. This enables the characterization of abnormal relationships between individual drones, between drones, and in the overall formation within a drone formation. Simultaneously, by parallel computing the weighted activation sum of each feature dimension, it quickly filters out the abnormal candidate features that contribute the most to the anomaly. This not only enhances the model's ability to perceive anomalies in the spatial structure of the formation but also provides a direct and reliable basis for locating the drone that is the source of the anomaly, making anomaly identification more targeted and interpretable.

[0120] Furthermore, to address the requirement that the formation only needs to identify three states: normal, abnormal position, and abnormal formation, the anomaly detection layer employs multi-class classification operations. The compressed feature vector is passed through a global pooling layer to output a global feature vector. The dimension is [1, 32]. Then, a Softmax multi-class classification operation is performed to calculate the probability values ​​of each anomaly. The formula is: ,in, and All are anomaly category numbers. , These correspond to "normal / abnormal position / abnormal formation" respectively. , The first , Layer-based weighting , The first , Layer discrimination layer bias term, For the first The probability values ​​of each abnormal state are calculated. Then, the category with the highest probability value is selected as the recognition result.

[0121] The output layer outputs the recognition results and corresponding probability values, as well as the anomaly candidate features and anomaly candidate frames obtained through identification. The format of the anomaly candidate feature is [dimensional index, associated object, deviation value], where the associated object is directly obtained from the dimension index using a preset mapping rule, and the deviation value is the deviation of the anomaly candidate feature in the specified position. The mean value within the frame time window; the format of anomaly candidate frames is [frame number]. Based on the anomaly candidate features and anomaly candidate frame identification rules, there are 3 anomaly candidate features and multiple anomaly candidate frames, but no more than [number missing]. indivual.

[0122] For example, the preset mapping rule is: a fixed vector of single-frame formation feature vectors, with dimensions... Each dimension corresponds to one of three types of features in a preset order: single-machine position deviation, inter-machine spacing deviation, and overall formation deviation; among them, single-machine position deviation includes The three-dimensional positional deviation, horizontal positional deviation, and vertical positional deviation of each UAV; the inter-UAV spacing deviation includes the absolute deviation and relative deviation between any two UAVs; the overall formation deviation includes the mean of the overall formation spacing deviation, the formation center offset, and the formation distortion. Therefore, the dimension sequence interval... For the corresponding single-unit position deviation, dimension = 3 × (UAV ID - 1) + 1 / 2 / 3, which corresponds to the three-dimensional position deviation, horizontal position deviation, and vertical position deviation of the UAV ID, where " / " represents "or"; dimension sequence range. Corresponding to the inter-machine spacing deviation, dimension = Corresponding drones and drones Absolute deviation of machine spacing, relative deviation of machine spacing; dimension number range The corresponding overall deviation of the formation is as follows: the average deviation of the overall spacing of the formation, the offset of the formation center, and the degree of formation distortion.

[0123] For example, the above model pre-training process includes collecting flight data for three types of conditions: normal formation flight, abnormal position, and abnormal formation. Formation feature vectors are generated according to the aforementioned steps. Training, validation, and test sets are constructed in a 7:2:1 ratio to ensure the dataset covers different formation sizes and different anomaly types. Weighted cross-entropy loss is used as the loss function for training, where the weight for abnormal formation is 1.2, the weight for abnormal position is 1.0, and the weight for normal formation is 0.8. A preset anomaly threshold is an empirically set value used to filter anomaly candidate frames to assist in subsequent localization, without affecting the model's discrimination of anomaly categories. Therefore, this threshold can be within the range […]. The threshold can be adjusted within [0.5] according to actual needs. A lower threshold can be used if more anomaly candidate frames are needed, and a higher threshold can be used if more accurate anomaly candidate frames are needed. This implementation does not impose a single limitation on this. The Adam optimizer is used with a learning rate of 0.001, 50 training epochs, and a batch size of 32. The early stopping strategy is to stop the training if the accuracy on the validation set does not improve for three consecutive epochs. The weight coefficients and bias terms obtained during training are then embedded into the model. The test set validation results must meet the following requirements: anomaly recognition accuracy ≥ 95%, false positive rate ≤ 3%, false negative rate ≤ 2%, and inference latency ≤ 10ms, where the inference latency is adapted to a 200Hz sampling rate. The above data are example data used for training, and this embodiment does not impose limitations on them.

[0124] It should be noted that the range of values ​​is [ [0.5] did not arise out of thin air; for If the sum of the attention weights for each frame is 1, then the average attention weight for each frame is... Under normal operating conditions, there is no significant deviation between frames, and the attention weight distribution is relatively uniform. The attention weight of most frames fluctuates slightly around the average value, with only a few frames slightly higher than the average. However, under abnormal operating conditions, abnormal frames with a sudden increase in deviation will receive attention weights much higher than the average value. Therefore, with... Using a lower threshold, suspected abnormal frames with weights higher than the average level can be effectively filtered out, avoiding the missed detection of weak anomalies; using 0.5 as the upper threshold can retain high-confidence abnormal frames with significantly higher weights, filter redundant noise frames, and form a system adapted to different... Adjustable filtering range.

[0125] Furthermore, when constructing training samples, the samples are labeled according to the following rules to clearly distinguish between normal, positional anomaly, and formation anomaly: (1) If the absolute deviation of the distance between any two UAVs is greater than or equal to the distance anomaly threshold, or the formation distortion degree is greater than or equal to the distortion threshold, then it is marked as a formation anomaly; (2) If, without satisfying the formation anomaly condition, the three-dimensional position deviation of at least one UAV is greater than or equal to the position anomaly threshold, then it is marked as a position anomaly; (3) If the three-dimensional position deviation, the absolute deviation of the distance between UAVs, and the formation distortion degree of all UAVs are less than the corresponding threshold, then it is marked as normal. The above thresholds and rules are only used to provide unified labels for training samples to supervise the deep learning model in learning anomaly features, and do not participate in the actual reasoning process of the model. The deep learning model achieves early identification and trend judgment that traditional threshold rules cannot accomplish by learning temporal changes, coupling correlations, and early weak anomaly features. The two are complementary in function and different in level, and together constitute a complete anomaly identification system.

[0126] For example, the position anomaly threshold, spacing anomaly threshold, and distortion threshold used in the training phase are all classification thresholds used for sample labeling. The position anomaly threshold can be set to 0.3 to 1.0 meters; the spacing anomaly threshold is used to determine whether the inter-machine spacing deviation constitutes a formation anomaly sample, and an exemplary value can be set to 0.5 to 2.0 meters; the distortion threshold can be set to 0.2 to 0.5. Those skilled in the art can adaptively select values ​​within the above ranges according to the UAV formation size, flight scenario, and control accuracy.

[0127] It should be noted that formation anomaly assessment takes precedence over position anomaly assessment. This is to avoid misjudging anomalies that have already spread and disrupted the formation structure as position anomalies, and to ensure accurate differentiation of anomaly levels and risk degrees. When a drone deviates from its position, judging only the status of a single drone would identify it as a position anomaly. However, in formation flight, a single drone's deviation may cause adjacent drones to passively adjust their positions to maintain a safe distance, leading to disordered inter-drone spacing and formation distortion. This can result in a transmitted, diffuse, and high-risk formation-level anomaly. If it is still treated as a position anomaly, only correcting the anomalous drone without locating the source of the fault cannot stop the transmission and spread of the anomaly within the formation, and it is difficult to eliminate formation distortion and collision risks.

[0128] This embodiment constructs a temporal deep learning model, using continuous multi-frame formation feature vectors as input. It fully utilizes temporal variation patterns and spatial correlations to identify three types of states: normal, abnormal position, and abnormal formation. During the training phase, a hierarchical labeling rule is adopted to prioritize the differentiation of high-risk formation anomalies, effectively avoiding the transmission-based misjudgment caused by single-drone offsets. This improves the accuracy and robustness of anomaly identification and achieves joint output of the anomaly occurrence time, anomaly feature dimensions, and anomaly category. This provides a reliable basis for subsequent source drone positioning. Furthermore, the model is lightweight and has a fast inference speed, which can meet the needs of real-time anomaly identification and early warning for drone formations.

[0129] Furthermore, the step of locating the source of the anomaly based on the flight anomaly identification results and probability values ​​in this embodiment also includes:

[0130] If the flight anomaly identification result is normal, no localization is performed;

[0131] When the flight anomaly identification result is a position anomaly or a formation anomaly, it is determined whether the probability value is greater than the preset confidence threshold.

[0132] If the probability value is less than the confidence threshold, it is judged as unreliable identification and no location is performed;

[0133] If the probability value is greater than or equal to the confidence threshold, then proceed to the process of locating the source of the anomaly.

[0134] Specifically, based on the obtained flight anomaly identification results, if the identification result is normal, the following operations are not performed, and the process returns directly to continue real-time detection; if the identification result is a position anomaly or formation anomaly, the probability value corresponding to the identification result is determined to be greater than or equal to a preset confidence threshold. When the probability value is less than the confidence threshold, it is judged as an unreliable identification, and no source localization is performed. Only a low-confidence anomaly prompt is output, including the flight anomaly identification result and the probability value; when the probability value is greater than or equal to the confidence threshold, the process of locating the source of the anomaly is initiated. For example, the confidence threshold can be set to 0.8, and the specific value can be set by those skilled in the art according to the identification accuracy. This embodiment does not limit this.

[0135] This embodiment first filters based on the probability value of the identification results, prioritizing the elimination of unreliable abnormal identification results to avoid positioning deviations and misjudgments caused by low-confidence identification. Then, it executes corresponding positioning logic for positional anomalies and formation anomalies respectively, distinguishing between independent anomalies and transmissive anomalies, ensuring the accuracy and reliability of source positioning, and improving the practicality and operability of UAV formation anomaly identification and early warning.

[0136] Furthermore, this embodiment provides a step for locating the source of the anomaly when the flight anomaly identification result is a location anomaly and the process enters the location anomaly source process, including:

[0137] Based on the associated objects of the abnormal candidate features, the set of UAVs to be inspected is obtained;

[0138] Based on the abnormal candidate frames, the abnormal start time and deviation change trend are determined for each UAV in the UAV collection to be inspected, and the abnormal start frame and deviation growth rate are obtained.

[0139] Drones with a deviation growth rate greater than zero are retained as valid abnormal drones, and all drones that are ultimately retained are identified as all abnormal drones at that location.

[0140] like Figure 3 As shown, when the identification result is a positional anomaly, it indicates that one or more drones in the formation have independently deviated from their own positions. There is no abnormal transmission between drones, and it does not cause distortion of the inter-drone spacing or disruption of the formation structure. This belongs to a multi-point independent anomaly. It is only necessary to identify all the anomalous individual drones, without determining a single source. Specific steps:

[0141] Based on the associated objects of the abnormal candidate features, extract the drone IDs of the associated objects that are single drones to form a set of drones to be inspected. If no single drone-type associated objects are found, two single drones can be optionally separated from the associated objects of the drone pair; if there are still no valid features, it is determined that there is no clear abnormal single drone, the localization process is terminated, and a prompt message indicating that the location is abnormal but the abnormal single drone cannot be located is output.

[0142] Next, based on the abnormal candidate frames, a temporal deviation analysis is performed. For each UAV in the UAV pool to be inspected, the abnormal start time is determined and the deviation change trend is calculated: the three-dimensional position deviation of the UAV in the abnormal candidate frames is extracted. This three-dimensional position deviation is directly obtained from the formation feature vector, which comprehensively reflects the overall deviation of the UAV from its ideal position in the three dimensions of longitude, latitude, and altitude, and characterizes the abnormal position state of the individual UAV. The frame sequence numbers of the abnormal candidate frames are traversed to find the earliest frame in the abnormal candidate frames where the three-dimensional position deviation value is greater than the first abnormal threshold, and it is recorded as the abnormal start frame of the UAV. The temporal sequence deviation sequence formed by the individual position deviation of the UAV from the abnormal start frame to the last frame of the abnormal candidate frames is taken, and the least squares method is used to perform linear fitting between the frame sequence number and the individual position deviation value to obtain the deviation growth rate. If the deviation growth rate is greater than 0, it means that the position deviation of the UAV increases with time, which meets the characteristics of an abnormal individual UAV, and it is retained in the UAV pool to be inspected. If the deviation growth rate is less than or equal to 0, it means that although the UAV has an instantaneous deviation, there is no continuous deterioration trend, and it is not considered an abnormal individual UAV, so it is removed from the UAV pool to be inspected.

[0143] All drones in the pooled database under inspection are identified as anomalous drones at this location, and their drone IDs, the initial frame of the anomaly, and the rate of deviation increase are output. If the anomalous drone result set is empty, a message indicating that a drone's location is anomalous but there is no further deterioration is output.

[0144] In this embodiment, the UAV set to be inspected is quickly located by filtering the associated objects of the abnormal candidate features in the position anomaly identification process. Then, the three-dimensional position deviation is extracted from the formation feature vector as the core judgment basis. Combined with the abnormal candidate frames, the abnormal starting frame is determined and the deviation growth rate is calculated. This effectively distinguishes between real anomalies that are continuously deteriorating and instantaneous fluctuation deviations, eliminates interference items with no deterioration trend, and identifies all abnormal single UAVs with independent position offsets that are continuously deteriorating. This improves the accuracy, reliability and interpretability of UAV positioning under position anomalies.

[0145] Furthermore, this embodiment provides a step for locating the source of an anomaly when the flight anomaly identification result is a formation anomaly and the process of locating the source of the anomaly is initiated, including:

[0146] The formation anomaly type is determined based on the associated objects of the anomaly candidate features. The formation anomaly types include inter-machine spacing anomalies and overall formation anomalies.

[0147] When the inter-machine spacing is determined to be abnormal, the core candidate UAV unit is determined based on the UAV pair in the abnormal candidate features. The source UAV is obtained by combining the abnormal start frame and deviation growth rate of the UAV in each unit.

[0148] When the formation is determined to be abnormal as a whole, the formation reference drone is selected as the source drone.

[0149] like Figure 4 As shown, when the identification result is a formation anomaly, the anomaly originates from a certain initial source drone and spreads through the relative relationships between drones, causing distortion of the formation spacing or overall deflection. This is a propagation-type formation-level anomaly, requiring the location of the source drone based on the identified anomaly points. Specific steps are as follows:

[0150] Based on the associated objects of the abnormal candidate features, the formation anomaly type is determined: when the associated objects include pairs of drones, it is determined to be an abnormal inter-drone spacing; when the associated objects only exist as a whole formation, it is determined to be an abnormal overall formation.

[0151] Furthermore, this embodiment provides a step of determining core candidate drone units based on drone pairs in the abnormal candidate features, and obtaining the source drone by combining the abnormal start frame and deviation growth rate of the drones within each unit, including:

[0152] An undirected graph is constructed based on the drone pairs in the abnormal candidate features, and the connected components in the graph are used as the core candidate drone units.

[0153] For each core candidate UAV unit, calculate the individual abnormal starting frame and deviation growth rate of each UAV within the unit:

[0154] If the unit contains only one pair of drones, the source drone is determined by combining the timing relationship between the drone's abnormal start frame and its own abnormal start frame.

[0155] If the unit contains multiple drone pairs, the drone with the earliest abnormal start frame is identified as the source drone. If the start frames are the same, the drone with the largest deviation growth rate is selected as the source drone.

[0156] Specifically, such as Figure 4 As shown in the source localization of the anomaly in the inter-drone spacing, the IDs of the two drones corresponding to all drone pairs in the anomaly candidate features are extracted. An undirected graph is constructed with drones as nodes and drone pairs as edges. Connected components in the undirected graph are identified. A connected component is a set of all nodes connected through shared drones. Each connected component serves as a core candidate drone unit, representing an association group that may have anomaly propagation. There may be one or more core candidate drone units.

[0157] The process of identifying connected components in an undirected graph is as follows: Traverse all nodes in the undirected graph, perform a depth-first search or breadth-first search on unmarked nodes, and filter out all nodes that are directly or indirectly connected to the unmarked node to form a connected component and mark it; repeat the above steps until all nodes in the undirected graph are marked, and finally obtain all connected components in the undirected graph, with each connected component serving as a core candidate UAV unit.

[0158] Each core candidate drone unit is processed independently:

[0159] If the core candidate drone unit contains only one pair of drones, then one-sided localization is performed:

[0160] For the pair of UAVs, the absolute deviation of the inter-UAV spacing within the candidate frames of anomalies is extracted. This absolute deviation is directly obtained from the formation feature vector and represents the degree of deviation between the actual relative spacing and the ideal spacing between the UAV pairs. It can intuitively reflect whether the safe distance between the UAVs has been violated and is a core indicator for judging the anomaly of the formation spacing and the risk of collision. The frame numbers of the candidate frames of anomalies are traversed, and the earliest frame that satisfies the condition that the absolute deviation of the inter-UAV spacing is greater than the second anomaly threshold is found and recorded as the starting frame of the anomaly for the UAV pair. .

[0161] For each drone in the drone pair and The three-dimensional position deviation of the UAV in the abnormal candidate frames is extracted respectively; the frame numbers of the abnormal candidate frames are traversed, and the earliest frame of each frame that satisfies the condition that the three-dimensional position deviation value is greater than the first abnormal threshold is found and recorded as its own abnormal start frame. and Given a three-dimensional positional deviation sequence from the initial frame of an anomaly to the last frame of an anomaly candidate, a least squares method is used to linearly fit the frame number and the deviation value to obtain the respective deviation growth rate. and .

[0162] Based on the drone's abnormal start frame , self-abnormal start frame and Deviation growth rate and To determine the source drone, the specific priority rules are as follows:

[0163] like and If drone i is the source;

[0164] like and If so, then drone j is the source;

[0165] like and If the starting frame of the error is earlier, the drone with the earlier error start frame is selected; if the starting frames are the same, the drone with the greater deviation growth rate is selected.

[0166] like and If the distance is abnormal, it is determined to be caused by external disturbance, and there is no clear source drone.

[0167] Once a specific source drone is located, all other drones within that core candidate drone unit, excluding the source drone, are marked as affected drones. The ID of each affected drone, its own abnormal start frame, and deviation growth rate are recorded and included in the list of affected drones to characterize the scope of the abnormal transmission caused by the source drone and the affected objects.

[0168] If the core candidate UAV unit contains multiple UAV pairs, i.e., there is a multilateral combination of shared UAVs, for example, UAV pair (1,2) and UAV pair (2,3) share UAV 2, or UAV pair (1,2), (1,3), and (1,4) share UAV 1, then perform multilateral positioning:

[0169] For each UAV in the core candidate UAV unit, extract its own abnormal start frame and deviation growth rate. This calculation method is the same as the single-UAV localization steps described above, and will not be repeated here.

[0170] The drone with the earliest self-abnormal start frame is identified as the source drone of the core candidate drone unit. If multiple drones have the same and earliest self-abnormal start frame, the drone with the largest deviation growth rate is selected as the source drone. All drones in this unit except the source drone are marked as affected drones, and their IDs, corresponding self-abnormal start frames, and deviation growth rates are recorded and added to the list of affected drones. For example, in the core candidate drone unit composed of drone pairs (1, 2) and (2, 3), if drone 2 has the earliest self-abnormal start frame, then drone 2 is the source drone, and drones 1 and 3 are affected drones; if drone 1 has the earliest self-abnormal start frame, then 1 is the source drone, and drones 2 and 3 are affected drones.

[0171] If there are multiple core candidate UAV units, and these core candidate UAV units are not connected to each other, then the above-mentioned single-pair positioning or multiple-pair positioning is performed independently on each core UAV unit to obtain their respective source UAVs.

[0172] For example, the first anomaly threshold is a judgment threshold set for three-dimensional position deviation, used to determine whether the UAV has experienced a real and valid positional shift; the second anomaly threshold is a judgment threshold set for the absolute deviation of the inter-UAV spacing, used to determine whether there is a real and valid spacing anomaly between the two UAVs. The aforementioned first and second anomaly thresholds can be preset according to the formation flight scenario, control accuracy, and safety requirements. As an example, for a typical multi-rotor UAV formation cruise scenario, the first anomaly threshold can be set to 0.5 to 1.5 meters, and the second anomaly threshold can be set to 1.0 to 3.0 meters. Those skilled in the art can adaptively select specific values ​​within the above range according to the UAV model, formation density, and flight safety level requirements. It should be noted that compared to the position anomaly thresholds and spacing anomaly thresholds set during the training phase, the thresholds here are usually set higher and more stringent to ensure the accuracy and reliability of the source positioning results.

[0173] Furthermore, such as Figure 4 As shown in the overall formation anomaly source localization, the specific steps are as follows: If a preset formation reference UAV exists, it is directly used as the core candidate UAV; if no preset formation reference UAV exists, the UAV closest to the geometric center of the formation is selected as the formation reference UAV. The formation reference UAV is a UAV used as a reference for the position and attitude of the entire formation during UAV formation flight missions. It is usually the lead aircraft, leader aircraft, or preset master UAV in the formation, and its position and status serve as the control reference for other UAVs in the formation. Using this formation reference UAV as the source UAV, the following steps are performed: extract the 3D position deviation value of the formation reference UAV in the anomaly candidate frames; traverse the frame numbers of the anomaly candidate frames, find the earliest frame where the 3D position deviation value is greater than the first anomaly threshold, and record it as the anomaly start frame of the formation reference UAV; take the position deviation sequence from the anomaly start frame to the last frame of the anomaly candidate frames, perform linear fitting on the frame number and deviation value to obtain the deviation growth rate.

[0174] The final warning report includes the anomaly category: position anomaly / formation anomaly; for position anomalies, all individual drone IDs, the starting frame of the anomaly, and the deviation growth rate; for formation anomalies, all source drone IDs, the starting frame of the anomaly, the deviation growth rate, and a list of affected drones.

[0175] This embodiment addresses formation anomaly scenarios by first differentiating between inter-drone spacing anomalies and overall formation anomalies based on the associated objects of anomaly candidate features. Then, it extracts the absolute deviation of inter-drone spacing and the three-dimensional position deviation from the formation feature vector as the basis for judgment. By using the anomaly start frame, its own anomaly start frame, and the deviation growth rate, it prioritizes and identifies the source drone that first experiences anomalies and whose deviation continues to worsen, and also identifies the affected drones. Simultaneously, for overall formation anomalies, it uses the formation reference drone or the drone closest to the geometric center for localization, effectively solving the problem of tracing the source of transmitted formation anomalies. This significantly improves the accuracy, logic, and interpretability of source localization under formation distortion, providing a reliable basis for formation safety management and anomaly handling.

[0176] Example 2

[0177] like Figure 5 As shown, this embodiment provides a deep learning-based UAV flight anomaly identification and early warning system to implement the aforementioned deep learning-based UAV flight anomaly identification and early warning method, including:

[0178] Data acquisition and processing module: used to uniformly acquire drone formation data and ideal formation data, and obtain standardized formation data through preprocessing;

[0179] Formation feature construction module: used to construct formation deviation and distortion features based on standardized formation data, and obtain formation feature vector;

[0180] Flight anomaly identification module: This module is used to input formation feature vectors into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features.

[0181] The positioning and early warning module is used to determine candidate anomaly objects based on flight anomaly identification results and probability values, use anomaly candidate features to determine the anomaly start frame and deviation growth rate of the candidate anomaly objects, locate the source of the anomaly, and generate early warning information.

[0182] Specifically, the modules establish communication connections through standardized data interfaces to achieve real-time data transmission, command interaction, and result feedback, forming a complete closed loop for UAV flight anomaly identification and early warning. The data acquisition and processing module, serving as the core of the system's data input, collects UAV formation data and ideal formation data through a communication interface. After time synchronization, noise reduction, interpolation, and standardization, it outputs the standardized formation data to the formation feature construction module. Based on the standardized formation data, the formation feature construction module calculates the deviation and distortion features of individual UAVs, between UAVs, and the overall formation, concatenates them to obtain a fixed-dimensional formation feature vector, and sends it to the flight anomaly identification module and the positioning warning module. The flight anomaly identification module inputs multiple consecutive frames of formation feature vectors into a pre-trained temporal deep learning model to complete the fusion of temporal and spatial features and anomaly classification, outputting the flight anomaly identification result, probability value, anomaly candidate frames, and anomaly candidate features to the positioning warning module. The positioning warning module determines whether to initiate the positioning process based on the anomaly type and probability value: if it is in a normal state, it returns to continue detection; if it is a positional or formation anomaly, it executes the positioning process based on the anomaly candidate frames and anomaly candidate features, ultimately generating and outputting warning information, thus achieving anomaly identification and warning. The specific functional implementation of each module is described in the relevant content of Embodiment 1, and will not be repeated in this embodiment.

[0183] This embodiment constructs a complete closed loop for UAV flight anomaly identification and early warning through clear division of labor and efficient collaboration among various modules. It achieves fully automated processing from data acquisition, preprocessing, feature construction, anomaly identification to source location and early warning output. It solves the technical problems of existing methods that separate anomaly identification from source location and untimely early warning. It can capture anomalies of individual UAVs, between UAVs, and as a whole in the formation, locate the source and development trend of anomalies, and output effective early warning information, providing full-process and all-round technical support for the safe and stable collaborative flight of UAV formations.

[0184] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0186] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0187] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A deep learning-based method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies, applied to UAV formation flight, characterized in that, include: Unified acquisition of UAV formation data and ideal formation data, preprocessing and constructing formation deviation and distortion features to obtain formation feature vectors; The formation feature vector is input into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features; Based on the flight anomaly identification results and probability values, candidate anomaly objects are determined using the anomaly candidate features, and the anomaly start frame and deviation growth rate of the candidate anomaly objects are calculated using the anomaly candidate frames to locate the anomaly source and generate early warning information. The flight anomaly identification results are normal, position anomaly, and formation anomaly. The candidate anomaly objects are the set of UAVs to be inspected when the position is abnormal and the core candidate UAV units when the formation is abnormal.

2. The method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 1, characterized in that, The preprocessing includes: Acquire drone formation data, including the three-dimensional position of each drone and the relative distance between any two drones; Obtain ideal formation data, including the ideal formation position of each drone and the ideal formation distance between any two drones; A timestamp synchronization mechanism is used to align the UAV formation data with the ideal formation data to form raw formation data in single-frame format; The original formation data is interpolated, filtered, denoised, and standardized to obtain standardized formation data.

3. The method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 1, characterized in that, The process of constructing formation deviation and distortion features to obtain formation feature vectors includes: Construct single-machine hierarchical features and calculate single-machine positional deviation features; Construct inter-machine hierarchical features and inter-computer spacing deviation features; Construct the overall hierarchical features of the formation, and calculate the formation center offset features and formation distortion features; All the above features are concatenated in a preset order to form a fixed-dimensional array feature vector.

4. The method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 1, characterized in that, The formation feature vector is input into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features, including: Continuous The frame array feature vectors form the input feature vector, which is then input into a pre-trained temporal deep learning model. The model includes an input layer, a temporal feature extraction layer, a spatial correlation fusion layer, an anomaly detection layer, and an output layer. The input feature vector is input into the temporal feature extraction layer to extract temporal features frame by frame, resulting in a weighted temporal feature vector and anomaly candidate frames; The input feature vector and the weighted temporal feature vector are input into the spatial correlation fusion layer for feature fusion to obtain the fused feature vector and anomaly candidate features; The fused feature vector is input into the anomaly detection layer, and the probability values ​​of whether the formation is normal, abnormal in position, or abnormal in formation are output through multi-classification operation. The category with the highest probability value is taken as the recognition result. The output layer outputs the flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features.

5. The method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 4, characterized in that, The input feature vector is input into the temporal feature extraction layer to extract temporal features frame by frame, resulting in a weighted temporal feature vector and anomaly candidate frames, including: The input feature vector is input into a gated recurrent unit network to extract temporal features frame by frame, resulting in a temporal feature vector; Attention operations are performed on the temporal feature vector to obtain attention weights for each frame, and then weighted and fused to obtain a weighted temporal feature vector; Frames with attention weights greater than a preset anomaly threshold are independently selected and recorded as anomaly candidate frames. These anomaly candidate frames are only used for subsequent anomaly source localization and do not participate in weighted fusion.

6. The method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 4, characterized in that, The input feature vector and the weighted temporal feature vector are input into a spatial correlation fusion layer for feature fusion to obtain a fused feature vector and anomaly candidate features, including: Spatial correlation features are extracted from the input feature vector and then concatenated and fused with the weighted temporal feature vector in the feature dimension to obtain a fused feature vector. The fused feature vector is compressed in dimension through a fully connected layer to output a compressed feature vector. In parallel statistical input feature vectors, each feature dimension is The weighted activation sum within the frame time window is sorted by weighted activation sum, and the features corresponding to the top-ranked feature dimensions are recorded as anomalous candidate features.

7. The method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 1, characterized in that, Based on the flight anomaly identification results and probability values, locating the source of the anomaly also includes: When the flight anomaly identification result is normal, no positioning is performed; When the flight anomaly identification result is a position anomaly or a formation anomaly, determine whether the probability value is greater than or equal to a preset confidence threshold. If the probability value is less than the confidence threshold, it is determined to be an unreliable identification and no positioning is performed; If the probability value is greater than or equal to the confidence threshold, then proceed to the process of locating the source of the anomaly.

8. The method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 7, characterized in that, When the flight anomaly identification result is a location anomaly and the process enters the location anomaly source process, the location anomaly source includes: Based on the associated objects of the abnormal candidate features, the set of UAVs to be inspected is obtained; Based on the abnormal candidate frames, the abnormal start time and deviation change trend are determined for each UAV in the UAV collection to be inspected, and the abnormal start frame and deviation growth rate are obtained. Drones with a deviation growth rate greater than zero are retained as valid abnormal drones, and all drones that are ultimately retained are identified as all abnormal drones at that location.

9. A method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 7, characterized in that, When the flight anomaly identification result is a formation anomaly and the process enters the location anomaly source process, the location anomaly source includes: The formation anomaly type is determined based on the associated objects of the anomaly candidate features, wherein the formation anomaly type includes inter-machine spacing anomaly and overall formation anomaly; When the inter-machine spacing is determined to be abnormal, the core candidate UAV unit is determined based on the UAV pair in the abnormal candidate features. The source UAV is obtained by combining the abnormal start frame and deviation growth rate of the UAV in each unit. When the formation is determined to be abnormal as a whole, the formation reference drone is selected as the source drone.

10. A method for identifying and warning of unmanned aerial vehicle (UAV) flight anomalies based on deep learning according to claim 9, characterized in that, The process of determining core candidate UAV units based on UAV pairs in the abnormal candidate features, and combining the abnormal start frame and deviation growth rate of the UAVs within each unit to obtain the source UAV, includes: An undirected graph is constructed based on the drone pairs in the abnormal candidate features, and the connected components in the graph are used as the core candidate drone units. For each core candidate UAV unit, calculate the individual abnormal starting frame and deviation growth rate of each UAV within the unit: If the unit contains only one pair of drones, the source drone is determined by combining the timing relationship between the drone's abnormal start frame and its own abnormal start frame. If the unit contains multiple drone pairs, the drone with the earliest abnormal start frame is identified as the source drone. If the start frames are the same, the drone with the largest deviation growth rate is selected as the source drone.

11. A deep learning-based UAV flight anomaly identification and early warning system, used to implement the deep learning-based UAV flight anomaly identification and early warning method as described in any one of claims 1-10, characterized in that, include: Data acquisition and processing module: used to uniformly acquire drone formation data and ideal formation data, and obtain standardized formation data through preprocessing; Formation feature construction module: used to construct formation deviation and distortion features based on the standardized formation data, and obtain formation feature vector; Flight anomaly identification module: used to input the formation feature vector into a pre-trained temporal deep learning model to obtain flight anomaly identification results, probability values, anomaly candidate frames, and anomaly candidate features; The positioning and early warning module is used to determine candidate anomaly objects based on the flight anomaly identification results and probability values, use the anomaly candidate features to determine the anomaly starting frame and deviation growth rate of the candidate anomaly objects, use the anomaly candidate frames to calculate the anomaly starting frame and deviation growth rate of the candidate anomaly objects, locate the source of the anomaly, and generate early warning information.

Images