Multi-dimensional data fusion unmanned aerial vehicle task monitoring method, device, equipment and medium

Abstract

This invention provides a method, apparatus, device, and medium for monitoring unmanned aerial vehicle (UAV) missions using multi-dimensional data fusion. The method includes: standardizing and preprocessing N-dimensional source data to generate N-dimensional standard data; fusing the N-dimensional source data to generate global fusion features; and inputting the global fusion features into a master prediction model to generate a predicted task output, which includes task completion time, remaining energy consumption, task completion probability, and risk level. This solution uses multi-dimensional source data for task prediction, effectively improving prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of drone monitoring technology, and in particular to a method, apparatus, equipment and medium for drone mission monitoring based on multi-dimensional data fusion. Background Technology

[0002] In the execution of drone missions in scenarios such as inspection, surveying, and logistics, accurate prediction of their trajectory, energy consumption, mission completion status, and environmental risks is the core foundation for ensuring the safe and efficient progress of missions. Currently, traditional drone mission monitoring solutions are mostly based on single-dimensional trajectory prediction, focusing only on extrapolating the drone's spatial position to achieve preliminary monitoring of the mission status. However, drone mission execution is a complex process influenced by the coupling of multiple dimensions of factors, and single-dimensional prediction will lead to large prediction errors. Summary of the Invention

[0003] This invention provides a method, apparatus, device, and medium for monitoring unmanned aerial vehicle (UAV) missions using multi-dimensional data fusion, in order to solve the aforementioned technical problems.

[0004] This invention provides a multi-dimensional data fusion method for monitoring unmanned aerial vehicle (UAV) missions, the method comprising: The N-dimensional source data is standardized and preprocessed to generate N-dimensional standard data. The N-dimensional source data includes location data, energy data, environmental data, task data, payload data, and time data. The location data, energy data, environmental data, task data, payload data, and time data are fused together to generate a global fusion feature; The global fusion features are input into the main prediction model to generate a prediction task output. The prediction task output includes task completion time, remaining energy consumption, task completion probability, and risk level. The task completion time is the predicted completion time of the matching task in the task data. The remaining energy consumption is the predicted energy consumption required to complete the matching task. The task completion probability is the predicted probability of completing the matching task. The risk level is the degree of risk of completing the matching task.

[0005] In one embodiment of the present invention, the location data, energy data, environmental data, task data, payload data, and time data are fused to generate a global fusion feature, including: The standard data collected from multiple sources in the same dimension are first fused to generate fused data; The fusion data from different dimensions are weighted and fused to generate the global fusion feature.

[0006] In one embodiment of the present invention, after inputting the global fusion features into the main prediction model and generating the prediction task output, the method further includes: Based on the historical error time series, the output of the prediction task is corrected to generate a compensated prediction value; Based on the compensated predicted value, the urgency of the matching task is quantified to generate an adjustment urgency index. Based on the aforementioned adjustment urgency index, the adjustment urgency level of the matching task is determined, and the task adjustment is carried out in a tiered manner.

[0007] In one embodiment of the present invention, the output of the prediction task is corrected based on the historical error time series to generate a compensated prediction value, including: The historical error time series and the output of the prediction task are input into the error prediction model, and the error prediction value is output; the historical error time series is generated based on the output of the prediction task and the actual execution status in the historical execution tasks. Based on the error prediction value, the output of the prediction task is corrected in real time.

[0008] In one embodiment of the present invention, based on the compensated predicted value, the urgency of the matching task is quantified to generate an adjustment urgency index, including: Based on the planned completion time and the compensated predicted value of the task completion time in the task data, the time deviation rate is determined; Based on the current actual remaining energy value in the energy data and the compensated predicted value of the remaining energy consumption, the energy margin ratio is determined; The current task completion probability is determined based on the compensated predicted value of the task completion probability. Based on the compensated predicted value of the risk level, the final output risk level is determined; Based on preset time thresholds, energy thresholds, probability thresholds, and risk thresholds, the time deviation rate, energy margin ratio, current task completion probability, and final output risk level are mapped to a matching fuzzy set to determine the adjustment urgency index.

[0009] In one embodiment of the present invention, based on the adjustment urgency index, the adjustment urgency level of the matching task is determined, and the task is adjusted in stages, including: The urgency levels of the adjustments are ranked from low to high as follows: mild adjustment, moderate adjustment, severe adjustment, and emergency adjustment. If it is a minor adjustment, reduce the cruise speed to reduce system energy consumption and reduce the sampling density in non-critical task areas to shorten task execution time; If the adjustment is moderate, the static priority in the task data is dynamically corrected to determine the dynamic priority of each task node; the dynamic priority is compared with a preset threshold, and task nodes that are lower than the preset threshold are identified as non-critical task nodes and deleted. In the case of a severe adjustment, task node resources are redistributed across platforms, and some task nodes of the overloaded platform are dynamically scheduled to the redundant collaborative platform. The optimal resource allocation of global tasks is achieved through multi-machine collaboration and multi-platform collaboration. If an emergency adjustment is needed, the mission will be aborted, and the aircraft will either make an emergency landing at the nearest airport or return to base to continue its mission.

[0010] In one embodiment of the present invention, N-dimensional source data is standardized and preprocessed to generate N-dimensional standard data, including: The N-dimensional source data is cleaned to remove outliers and fill in missing values ​​to generate the first intermediate data. Align the first intermediate data with different sampling frequencies to a unified time base to generate the second intermediate data; Based on the second intermediate data, feature data is extracted, including real-time energy consumption rate, task progress deviation rate, environmental degradation index, communication quality attenuation coefficient, and equipment performance attenuation coefficient. The feature data is normalized, and time windows are constructed. Temporal features within each time window are extracted to generate the standard data.

[0011] This invention also provides a multi-dimensional data fusion-based drone mission monitoring device, comprising: The data standardization processing module is used to perform standardization preprocessing on N-dimensional source data to generate N-dimensional standard data. The N-dimensional source data includes location data, energy data, environmental data, task data, payload data, and time data. The data fusion module is used to fuse the location data, energy data, environmental data, task data, payload data, and time data to generate global fusion features; The prediction task output module is used to input the global fusion features into the main prediction model and generate prediction task output. The prediction task output includes task completion time, remaining energy consumption, task completion probability, and risk level. The task completion time is the predicted completion time of the matching task in the task data. The remaining energy consumption is the predicted energy consumption required to complete the matching task. The task completion probability is the predicted probability of completing the matching task. The risk level is the degree of risk of completing the matching task.

[0012] The present invention also provides an electronic device, the electronic device comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, enable the electronic device to implement the multi-dimensional data fusion method for monitoring drone missions.

[0013] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer processor, causes the computer to perform the aforementioned multi-dimensional data fusion method for monitoring unmanned aerial vehicle (UAV) missions.

[0014] The beneficial effects of this invention are as follows: This invention proposes a multi-dimensional data fusion method for monitoring UAV missions. By collecting multi-dimensional source data, it increases the accuracy of subsequent mission prediction outputs. Since the multi-dimensional source data may be collected from different sensors, standardized preprocessing is required to facilitate subsequent data fusion. The collected data includes dimensions such as location, energy, environment, mission, payload, and time. By comprehensively considering the impact of different dimensional source data on the output of the main prediction model, the accuracy of the prediction results is effectively increased. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0016] In the attached diagram: Figure 1 This is a flowchart of a multi-dimensional data fusion method for monitoring drone missions provided in one embodiment of the present invention.

[0017] Figure 2 This is a block diagram of a multi-dimensional data fusion drone mission monitoring device provided in one embodiment of the present invention. Detailed Implementation

[0018] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0021] Please see Figure 1 , Figure 1 A multi-dimensional data fusion method for monitoring UAV missions, provided in an embodiment of the present invention, specifically includes: Step S1: Real-time acquisition of N-dimensional source data.

[0022] Obtain the original feature vector set F of the N-dimensional source data, denoted as: F={f1,f2,...,f N} in, f i Let N be the vector of the i-th feature dimension; N is the total number of feature dimensions.

[0023] Specifically, through multiple types of sensor interfaces and the task management system interface, N-dimensional source data of the entire task execution cycle are collected in real time and synchronously. All dimensional source data are marked with timestamps, and the sampling frequency can be configured according to task requirements.

[0024] For example, the source data considers six dimensions: location data, energy data, environmental data, mission data, payload data, and time data. Location data is determined using GPS or BeiDou coordinates and includes flight path, flight or driving speed, heading angle, and distance to the target point. Energy data includes remaining battery or fuel, real-time energy consumption rate, charging or recharging status, and state of health (SOH). Environmental data includes weather conditions, wind speed and direction, ambient temperature, obstacle distribution, communication signal strength, and electromagnetic interference intensity. Mission data includes overall mission completion, key milestone achievement status, number of completed mission nodes, number of remaining mission nodes, preset static priorities of each mission node, and dependencies between mission nodes. Payload data includes payload weight, mission payload operating status, sensor operating status, and equipment health. Time data includes planned mission execution time, actual execution time, node delay time, and total mission time window.

[0025] Step S2 involves standardizing and preprocessing the collected N-dimensional source data and extracting its features.

[0026] In this embodiment, the execution of step S2 is to provide standardized input for subsequent N-dimensional source data fusion and prediction, specifically including steps S210 to S250.

[0027] Step S210, Data Cleaning Outliers in the source data are removed using the 3σ criterion, missing values ​​are filled using linear interpolation or Kalman filtering, and noisy data is smoothed using the moving average method to obtain the first intermediate data to ensure data validity.

[0028] Specifically, the 3σ criterion states that in normally distributed data, almost all normal data fall within the range of the mean ± 3 times the standard deviation, and those outside this range can be identified as outliers.

[0029] For example, if we denote the mean as x and the standard deviation as σ, then the normal data range y is denoted as: x 3σ≤y≤x+3σ. Values ​​exceeding the normal range of y are considered outliers.

[0030] Step S220, Time Alignment Specifically, linear interpolation is used to align the first intermediate data with different sampling frequencies to a unified time base, generating the second intermediate data and ensuring the time synchronization of all data.

[0031] Step S230, Feature Extraction Specifically, derived features (i.e. feature data) are calculated based on the second intermediate data, including core derived features such as real-time energy consumption rate, task progress deviation rate, environmental degradation index, communication quality attenuation coefficient, and equipment performance attenuation coefficient.

[0032] For example, the formula for calculating the real-time energy consumption rate P(t) is shown below: in, for Remaining energy at any given moment.

[0033] for Remaining energy at any given moment.

[0034] This represents the time difference between adjacent sampling.

[0035] For example, the formula for calculating the task schedule deviation rate δη(t) is shown below:

[0036] in, Assess the completion rate of planned tasks; This represents the actual task completion rate.

[0037] For example, the environmental degradation index I env The calculation formula is as follows:

[0038] in, Real-time wind speed; Maximum permissible wind speed; Ambient temperature; To achieve the optimal operating temperature; Electromagnetic interference intensity; Maximum permissible interference; , , These are weighting coefficients, and the sum of the three is 1. Maximum permissible operating temperature; This is the minimum permissible operating temperature.

[0039] For example, the communication quality attenuation coefficient γ comm The calculation formula is as follows:

[0040] in, Real-time signal strength; This represents the upper limit of signal strength. This represents the lower limit of signal strength.

[0041] For example, drone equipment includes batteries, motors, communication devices, and sensor devices. The performance degradation of the equipment is affected by factors such as natural aging, ambient temperature, and workload, as specifically shown below:

[0042] in, This is the equipment performance degradation coefficient, with a value range of [0,1]. The larger the value, the more severe the equipment performance degradation. , , These are the weighting coefficients for natural aging, ambient temperature, and workload, respectively, and the sum of the three is 1, which can be adaptively adjusted according to the task scenario; tuse This refers to the cumulative usage time of the equipment. t life The design lifespan of the equipment; T dev The real-time operating temperature of the equipment; T opt To achieve the optimal operating temperature for the equipment; P load For the real-time workload of the equipment; P rated This is the rated load of the equipment.

[0043] Step S240, Data normalization processing Specifically, the Min-Max normalization method (i.e. deviation standardization) is used to uniformly map feature data of different dimensions to the [0,1] interval, thereby eliminating the impact of dimension differences on the main prediction model.

[0044] Step S250, construct the time window Specifically, a sliding time window of length T is constructed, and temporal features within each time window are extracted. Here, T is the time period, which can be set according to requirements.

[0045] Step S3, N-dimensional data fusion Specifically, a three-level fusion strategy of data layer, feature layer and decision layer is adopted to achieve deep fusion of N-dimensional heterogeneous data, including steps S310 to S330.

[0046] Step S310, Data Layer Fusion For multi-source sensor data of the same type, Kalman filtering algorithm is used for fusion, such as the fusion of position data from multiple GNSS (Global Navigation Satellite System) modules and the fusion of energy data from multiple electrical sensors, in order to reduce the measurement error of a single sensor and improve the reliability of the basic data.

[0047] Step S320, Feature layer fusion An attention mechanism is used to adaptively weight and fuse feature vectors of different dimensions. The core calculation method is as follows: The feature vector of the i-th dimension is denoted as f. i The attention weight α is calculated using a learnable weight matrix. i The calculation formula is as follows:

[0048]

[0049] in, The attention score for the i-th feature dimension; The attention score for the j-th feature dimension; α i Let be the attention weight corresponding to the i-th dimension feature, and the sum of all weights is 1; W q and W k The weight matrix is ​​a learnable matrix. For W q transpose; b is the bias term (bias vector).

[0050] N is the total number of feature dimensions.

[0051] tanh( ) is the hyperbolic tangent activation function.

[0052] exp( ) is the natural exponential function.

[0053] Based on attention weight α i Perform feature weighted fusion to obtain the final fused feature vector. , is represented as:

[0054] This mechanism can automatically amplify the weights of features that significantly impact task progress, suppress interference from irrelevant features, and solve the problem of weight allocation for N-dimensional data.

[0055] Step S330, Decision-making level integration By integrating the intermediate features output from the time series prediction sub-model, energy consumption prediction sub-model, and risk assessment sub-model, the fused feature vector is... A second optimization is performed to generate the final global fusion feature F. global , and input into the main prediction model.

[0056] Specifically, during the decision-level fusion, the fused feature vector obtained from the feature-level fusion is first... Intermediate features F output by the time series prediction sub-model time The intermediate features F output by the energy consumption prediction sub-model energy Intermediate features F output by the risk assessment sub-model risk Dimension alignment and normalization are performed; then, the fusion weights corresponding to each intermediate feature are calculated through fully connected mapping or attention-gated networks; finally, the intermediate features are weighted and concatenated or weighted and summed according to their weights, and secondary feature optimization is performed through residual connections, normalization layers, and nonlinear activation functions to obtain the final global fusion feature F. globalThis global fusion feature simultaneously preserves the fusion representation of the original multidimensional data, the temporal evolution trend, the energy consumption change trend, and the risk status information, thus serving as the input to the main prediction model and improving the prediction accuracy of subsequent task completion time, remaining energy consumption, task completion probability, and risk level.

[0057] Step S4: Perform multi-task progress prediction Based on the global fusion feature, an improved Bi-LSTM network (bidirectional long short-term memory network) is used to construct a multi-task joint master prediction model, which outputs four core task progress indicators.

[0058] For example, the main structure of the master prediction model includes: The encoder consists of two stacked bidirectional long short-term memory networks (Bi-LSTM), with 128 hidden units per layer, which can simultaneously capture forward and backward dependencies of time series and extract deep temporal features. The attention layer includes a four-head self-attention mechanism that adaptively focuses on the key time steps and feature dimensions that have the greatest impact on the prediction results, thereby strengthening the feature representation of key information. The residual connection module adds residual connections after each Bi-LSTM layer, directly mapping input features to the output, alleviating the gradient vanishing problem in deep networks and improving model training stability. Dropout layer (i.e., random dropout layer) sets the Dropout rate and randomly deactivates some neurons to prevent the model from overfitting; The decoder and output layer consist of two fully connected layers. Following these two fully connected layers are four output headers corresponding to different tasks, representing the outputs of four prediction tasks. These four prediction task outputs include four core metrics: Time to Completion (TTC), Remaining Energy Consumption (REC), Task Completion Probability (TCP), and Risk Level (RL). The task completion time is trained using a weighted multi-task loss function to focus on the accuracy of task completion time predictions and reduce the probability of inaccurate time predictions due to interference from multiple tasks. The total loss function is Loss. total The definition is as follows: Loss total =λ1*Loss TTC +λ2*Loss REC +λ3*Loss TCP +λ4*Loss RL in: Loss TTC Loss REC The mean squared error (MSE) loss corresponds to different regression tasks in the main prediction model. Loss TCP This is the binary cross-entropy loss, corresponding to the binary classification task of the main prediction model; Loss RL This is the multi-class cross-entropy loss, corresponding to the risk level classification task of the main prediction model; λ1, λ2, λ3, and λ4 are the weight coefficients for each corresponding subtask, used to balance the training priority of different subtasks, and can be adaptively adjusted according to the actual task scenario.

[0059] Step S5: Perform adaptive error compensation on the prediction results of the main prediction model.

[0060] To further improve prediction accuracy, an independent error compensation mechanism is designed to correct the prediction results of the main prediction model in real time. The specific execution process includes: Step S510: Construct historical error time series The deviation between the predicted value (i.e., the historical output of the prediction task) and the actual execution value (i.e. the actual execution status) at each time step is stored to form a historical error time series, which provides a data basis for subsequent error prediction.

[0061] Step S520: Construct the error prediction model A three-layer lightweight fully connected neural network is used to construct an error prediction model. The model takes historical error time series as input and outputs the estimated error value of the current prediction result. The specific principle is as follows: In actual task execution, factors such as sudden environmental changes, load variations, communication jitter, and equipment drift can cause real-time prediction deviations. These deviations exhibit temporal continuity and local regularity. Therefore, by learning the changing trends of historical prediction errors, the prediction error at the current moment can be estimated in real time, and this error can be compensated into the original prediction result, making the corrected output closer to the true value.

[0062] Step S530: Correct the core prediction results Based on the error prediction value obtained in step S520, the core prediction results such as TTC and REC output by the main prediction model are corrected in real time to obtain the final compensated prediction value y.

[0063] For example, the compensated task completion time is denoted as The remaining energy consumption after compensation is denoted as The probability of task completion after compensation is denoted as y. TCP The risk level after compensation is denoted as y. RL .

[0064] Step S540, incrementally update model parameters Upon completion of each task node, the parameters of the error compensation model are updated using the latest collected error data through incremental learning. This enables continuous optimization of compensation accuracy and ensures the predictive stability of the model during long-term operation.

[0065] Step S6: Adjust the trigger judgment and urgency quantification Based on the compensated predicted value, a multi-condition coupled adjustment trigger judgment is performed, specifically including: Step S610: Preset trigger threshold Trigger thresholds include time thresholds, energy thresholds, probability thresholds, and risk thresholds. The time threshold is the upper limit to which the task completion time exceeds the planned time; the energy threshold is the product of predicted remaining energy consumption and the current remaining energy and safety factor; the probability threshold is when the task completion probability is less than the minimum safety threshold; and the risk threshold is when the risk level reaches level 3 or higher (out of 5 levels, with higher levels indicating greater risk).

[0066] Step S620, Adaptive trigger threshold optimization Based on historical adjustment effect data, reinforcement learning algorithms are used to dynamically optimize each trigger threshold, achieving an optimal balance between task success rate and adjustment frequency, avoiding over-adjustment or under-adjustment.

[0067] Step S630, Adjust the urgency quantification Using a fuzzy logic algorithm, the results of multiple condition triggers are combined to quantify and calculate the adjustment urgency index (AUI). The AUI value ranges from [0,1], and the higher the value, the stronger the urgency of the adjustment.

[0068] For example, the adjustment urgency level includes mild adjustment, moderate adjustment, severe adjustment, and emergency adjustment. A mild adjustment occurs when the adjustment urgency index falls within the first range. A moderate adjustment occurs when the index falls within the second range. A severe adjustment occurs when the index falls within the third range, and an emergency adjustment occurs when the index falls within the fourth range. Different adjustment strategies are applied to different urgency levels. The first, second, third, and fourth ranges are non-overlapping value intervals that increase sequentially and are all within the range [0,1]. The specific boundaries can be configured according to the task scenario.

[0069] For example, the process of quantifying the Adjusted Urgency Index (AUI) includes: Step S6301: Determine the time deviation rate ΔT For example, the formula for calculating the time deviation rate ΔT is shown below:

[0070] in, The standard completion time is the initial planned time for the task, and the preset planned duration (i.e., the planned execution time of the task) is the time when the task is issued.

[0071] Step S6302: Determine the energy margin ratio ΔE.

[0072] For example, the formula for calculating the energy margin ratio ΔE is shown below:

[0073] in, This represents the current real-time remaining energy value.

[0074] β is a preset safety factor, which can be in the range of (0,1] and can be configured according to the task scenario.

[0075] Step S6303: Determine the task completion probability P under the current state. TCP .

[0076] For example, the task completion probability P in the current state TCP The calculation formula is as follows: P TCP =y TCP

[0077] Step S6304: Determine the final output risk level R. RL .

[0078] For example, R RL =argmax(y RL ) Where, argmax( ) is an operation function that takes the index corresponding to the maximum value and is used to extract the risk level with the highest probability from the probability vector.

[0079] Step S6305: Construct a membership function based on time threshold, energy threshold, probability threshold, and risk threshold, and integrate the time deviation rate ΔT, energy margin ratio ΔE, and task completion probability P. TCP Risk level R RL The rules are mapped to the corresponding fuzzy sets. A fuzzy rule base with four inputs and one output is constructed. The activation intensity of each rule is calculated using the Mamdani inference algorithm, and the fuzzy output membership set μAUI(x) of AUI is synthesized.

[0080] Step S6306: Determine the adjustment of the urgency index (AUI). The centroid method is used for deblurring, and the adjustment urgency index AUI in the range of [0,1] is calculated.

[0081]

[0082] in, x m Let m be the m-th discrete point in the fuzzy universe of AUI; n1 is the total number of points in the discrete universe; For the fuzzy output membership set μAUI(x) at discrete point x m The value at that location.

[0083] Step S7: Perform dynamic task adjustment and optimization in stages. Based on the Adjustment Urgency Index (AUI), a corresponding tiered adjustment strategy is matched, and a pre-set fast tabu search algorithm is used to optimize the solution. The specific implementation includes: Step S710: Implement a mild adjustment strategy. When the AUI is within the first range, the task status is considered to be slightly fluctuating. Only lightweight optimization adjustments to the task execution parameters are needed, without changing the overall task planning framework. Specific adjustments include reducing the cruising speed to decrease system energy consumption and reducing the sampling density in non-critical task areas to shorten task execution time. Through these parameter-level fine-tunings, a balance between task execution efficiency and resource consumption is achieved without disrupting the original task chain.

[0084] Non-critical mission areas are those that, relative to core mission objectives, critical paths, and security boundaries, have no decisive impact on the final completion of missions, milestones, or core data collection, and have no mandatory security or compliance requirements.

[0085] Step S720: Implement a moderate adjustment strategy. When AUI is in the second range, the task status is determined to be moderately abnormal or high-risk, requiring task-level replanning and path optimization. Specific adjustments include: reordering task nodes by priority and deleting non-critical task nodes; or using the A* (A-star) heuristic path planning algorithm to optimize the path, starting from the current task node and ending at the target node. During path optimization, the mission area is first discretized into a grid map or waypoint map structure based on the current mission area map, obstacle distribution, no-fly zones, weather conditions, communication quality, electromagnetic interference intensity, and the UAV's remaining energy. Each grid or waypoint serves as a search node, and adjacent passable nodes form candidate edges. A comprehensive cost function is constructed for each candidate edge, comprehensively considering path distance, estimated flight time, estimated energy consumption, environmental risk, communication risk, and obstacle avoidance cost. For example, the comprehensive cost function Cost can be expressed as: Cost=wD *D+w T *T+w E *E+w Renv *Renv+w Rcomm *Rcomm+w Robstacle Robstacle in, D is the path distance; T represents the estimated execution time; E represents the projected energy consumption; Renv represents environmental risk; Rcomm poses a communication risk; Robstacle is a penalty for obstructions or no-fly zones; w D w T w E w Renv w Rcomm w Robstacle The weights for each item can be adaptively configured according to the task scenario. D +w T +w E +w Renv +w Rcomm +w Robstacle =1.

[0086] It should be understood that each cost item is normalized before weighting to ensure that the dimensions of each cost item are consistent.

[0087] The A* algorithm starts at the current task node and ends at the target task node. It uses the comprehensive cost (Cost) as the actual cost g(n) and the estimated distance or flight time from the current position to the target node as the heuristic function h(n). It calculates the total cost f(n) of node n, f(n) = g(n) + h(n), and prioritizes expanding nodes with the smallest total cost f(n). Nodes located in no-fly zones, obstacle areas, communication-unreachable areas, or where energy consumption is insufficient are directly removed from the candidate nodes. After the search is complete, the obtained paths are smoothed and their energy consumption and safety are verified. If a path meets the requirements of remaining energy, risk threshold, and task time window, it is issued as a new task execution path; if no feasible path meets the constraints, a higher-level heavy adjustment or emergency adjustment strategy is triggered.

[0088] For example, the priority reordering of task nodes includes: Based on the predicted output of each node in step S4, and combined with the current remaining resources and environmental status, the initial preset static priority is dynamically corrected, and the dynamic priority of each node is recalculated. Then, based on the dynamic priority, non-critical task nodes with priorities lower than the preset threshold are filtered and deleted, and the task execution sequence is updated.

[0089] Specifically, assuming the total number of task nodes to be executed is M, the dynamic priority calculation formula for the k-th task node is as follows: P dynamic,k =ω1*P static,k +ω2*G k +ω3 / (C k +ε)+ω4*D k in, P dynamic,k The dynamic priority of the k-th task node; P static,k The preset static priority for the k-th task node; G k The contribution of the k-th node to the task objective is normalized to [0,1]. C k The total execution cost of the k-th node is calculated by comprehensively considering time, energy consumption, and risk level. ε is a preset positive small constant used to avoid the total execution cost C. k Taking zero results in a zero denominator and suppresses abnormal numerical amplification; the value of ε is determined according to C. k The dimensions and range of values ​​are preset; D k The value represents the dependency strength of the k-th node, with higher values ​​for critical preceding nodes. ω1, ω2, ω3, ω4 are weighting coefficients that satisfy ω1+ω2+ω3+ω4=1.

[0090] Finally, the nodes are reordered from highest to lowest dynamic priority. A higher dynamic priority value indicates higher priority, and a lower value indicates lower priority.

[0091] Step S730: Execute the severe adjustment strategy. When the AUI is in the third range, the task status is determined to be severely abnormal or resource-scarce, requiring platform-level resource reallocation and collaborative scheduling. Specific adjustments include: cross-platform reallocation of task node resources; dynamically scheduling some task nodes from overloaded platforms to redundant collaborative platforms; and achieving optimal resource allocation for global tasks through multi-machine and multi-platform collaboration to avoid task failures caused by single-node overload and ensure the overall reliability of global task execution.

[0092] Step S740: Execute the emergency adjustment strategy. When the AUI is in the fourth range, the mission status is determined to be critically dangerous, requiring immediate activation of emergency response strategies and execution of the highest safety priority emergency operations. Specific adjustments include: immediately executing safety operations such as mission return to base, emergency landing at the nearest airport, or mission abort, prioritizing the operational safety of the platform and payload, suspending non-essential missions until the risk is eliminated, and preventing equipment damage or safety accidents caused by the spread of danger.

[0093] Furthermore, for the fast tabu search algorithm, the tabu list length is dynamically adjusted based on the size of the optimization problem and the convergence of solutions during the iteration process: when the convergence speed of solutions is fast during the algorithm iteration, the tabu list length is appropriately shortened to improve search efficiency; when the algorithm gets stuck in a local optimum, the tabu list length is extended to expand the search range, effectively avoiding the algorithm getting stuck in a local optimum and balancing search efficiency and global optimization capability. A fast candidate solution screening rule is designed. After generating candidate solutions in each iteration, pre-set fast evaluation criteria (such as task execution time, resource load rate, risk index, and other core indicators) are used to preemptively eliminate obviously degraded candidate solutions, significantly reducing the computational load of a single iteration and improving the algorithm's running efficiency. The maximum number of iterations is dynamically set based on the AUI (Task Adjustment Priority Index) value: in normal task scenarios, a sufficient number of iterations is set to ensure optimization accuracy; in emergency scenarios, the maximum number of iterations is limited to ≤50, ensuring optimization effect while meeting the real-time decision-making requirements and adapting to the rapid handling needs of emergency tasks.

[0094] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0095] Figure 2 This is a block diagram of a multi-dimensional data fusion drone mission monitoring device, as illustrated in an exemplary embodiment of this application.

[0096] like Figure 2 As shown, this exemplary multi-dimensional data fusion drone mission monitoring device includes: The data standardization processing module 210 is used to perform standardization preprocessing on N-dimensional source data to generate N-dimensional standard data. The N-dimensional source data includes location data, energy data, environmental data, task data, payload data, and time data. The data fusion module 220 is used to fuse the location data, energy data, environmental data, task data, payload data and time data to generate global fusion features; The prediction task output module 230 is used to input the global fusion features into the main prediction model to generate prediction task output. The prediction task output includes task completion time, remaining energy consumption, task completion probability, and risk level. The task completion time is the predicted completion time of the matching task in the task data. The remaining energy consumption is the predicted energy consumption required to complete the matching task. The task completion probability is the predicted probability of completing the matching task. The risk level is the degree of risk of completing the matching task.

[0097] It should be noted that the multi-dimensional data fusion UAV mission monitoring device and the multi-dimensional data fusion UAV mission monitoring method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the multi-dimensional data fusion UAV mission monitoring device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0098] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, enable the electronic device to implement the multi-dimensional data fusion drone mission monitoring method provided in the above embodiments.

[0099] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the multi-dimensional data fusion UAV mission monitoring method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0100] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the multi-dimensional data fusion UAV mission monitoring method provided in the various embodiments above.

[0101] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for monitoring unmanned aerial vehicle (UAV) missions using multi-dimensional data fusion, characterized in that, include: The N-dimensional source data is standardized and preprocessed to generate N-dimensional standard data. The N-dimensional source data includes location data, energy data, environmental data, task data, payload data, and time data. The location data, energy data, environmental data, task data, payload data, and time data are fused together to generate a global fusion feature; The global fusion features are input into the main prediction model to generate a prediction task output. The prediction task output includes task completion time, remaining energy consumption, task completion probability, and risk level. The task completion time is the predicted completion time of the matching task in the task data. The remaining energy consumption is the predicted energy consumption required to complete the matching task. The task completion probability is the predicted probability of completing the matching task. The risk level is the degree of risk of completing the matching task.

2. The UAV mission monitoring method based on multi-dimensional data fusion according to claim 1, characterized in that, The location data, energy data, environmental data, mission data, payload data, and time data are fused to generate a global fusion feature, including: The standard data collected from multiple sources in the same dimension are first fused to generate fused data; The fusion data from different dimensions are weighted and fused to generate the global fusion feature.

3. The UAV mission monitoring method based on multi-dimensional data fusion according to claim 1, characterized in that, After inputting the global fusion features into the main prediction model and generating the prediction task output, the method further includes: Based on the historical error time series, the output of the prediction task is corrected to generate a compensated prediction value; Based on the compensated predicted value, the urgency of the matching task is quantified to generate an adjustment urgency index. Based on the aforementioned adjustment urgency index, the adjustment urgency level of the matching task is determined, and the task adjustment is carried out in a tiered manner.

4. The UAV mission monitoring method based on multi-dimensional data fusion according to claim 3, characterized in that, Based on the historical error time series, the output of the prediction task is corrected to generate a compensated prediction value, including: The historical error time series and the output of the prediction task are input into the error prediction model, and the error prediction value is output; the historical error time series is generated based on the output of the prediction task and the actual execution status in the historical execution tasks. Based on the error prediction value, the output of the prediction task is corrected in real time.

5. The UAV mission monitoring method based on multi-dimensional data fusion according to claim 3, characterized in that, Based on the compensated predicted value, the urgency of the matching task is quantified to generate an adjustment urgency index, including: Based on the planned completion time and the compensated predicted value of the task completion time in the task data, the time deviation rate is determined; Based on the current actual remaining energy value in the energy data and the compensated predicted value of the remaining energy consumption, the energy margin ratio is determined; The current task completion probability is determined based on the compensated predicted value of the task completion probability. Based on the compensated predicted value of the risk level, the final output risk level is determined; Based on preset time thresholds, energy thresholds, probability thresholds, and risk thresholds, the time deviation rate, energy margin ratio, current task completion probability, and final output risk level are mapped to a matching fuzzy set to determine the adjustment urgency index.

6. The UAV mission monitoring method based on multi-dimensional data fusion according to claim 3, characterized in that, Based on the aforementioned adjustment urgency index, the adjustment urgency level of the matching tasks is determined, and task adjustments are performed in a tiered manner, including: The urgency levels of the adjustments are ranked from low to high as follows: mild adjustment, moderate adjustment, severe adjustment, and emergency adjustment. If it is a minor adjustment, reduce the cruise speed to reduce system energy consumption and reduce the sampling density in non-critical task areas to shorten task execution time; If the adjustment is moderate, the static priority in the task data is dynamically corrected to determine the dynamic priority of each task node; the dynamic priority is compared with a preset threshold, and task nodes that are lower than the preset threshold are identified as non-critical task nodes and deleted. In the case of a severe adjustment, task node resources are redistributed across platforms, and some task nodes of the overloaded platform are dynamically scheduled to the redundant collaborative platform. The optimal resource allocation of global tasks is achieved through multi-machine collaboration and multi-platform collaboration. If an emergency adjustment is needed, the mission will be aborted, and the aircraft will either make an emergency landing at the nearest airport or return to base to continue its mission.

7. The UAV mission monitoring method based on multi-dimensional data fusion according to claim 1, characterized in that, The N-dimensional source data is standardized and preprocessed to generate N-dimensional standard data, including: The N-dimensional source data is cleaned to remove outliers and fill in missing values ​​to generate the first intermediate data. Align the first intermediate data with different sampling frequencies to a unified time base to generate the second intermediate data; Based on the second intermediate data, feature data is extracted, including real-time energy consumption rate, task progress deviation rate, environmental degradation index, communication quality attenuation coefficient, and equipment performance attenuation coefficient. The feature data is normalized, and time windows are constructed. Temporal features within each time window are extracted to generate the standard data.

8. A multi-dimensional data fusion unmanned aerial vehicle (UAV) mission monitoring device, characterized in that, include: The data standardization processing module is used to perform standardization preprocessing on N-dimensional source data to generate N-dimensional standard data. The N-dimensional source data includes location data, energy data, environmental data, task data, payload data, and time data. The data fusion module is used to fuse the location data, energy data, environmental data, task data, payload data, and time data to generate global fusion features; The prediction task output module is used to input the global fusion features into the main prediction model and generate prediction task output. The prediction task output includes task completion time, remaining energy consumption, task completion probability, and risk level. The task completion time is the predicted completion time of the matching task in the task data. The remaining energy consumption is the predicted energy consumption required to complete the matching task. The task completion probability is the predicted probability of completing the matching task. The risk level is the degree of risk of completing the matching task.

9. A device, characterized in that, include: One or more processors and memory, The memory stores a computer program that, when executed by the one or more processors, causes the device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by one or more processors, causes the device to perform the method as described in any one of claims 1-7.

Images