Intelligent equipment and unmanned aerial vehicle cooperative navigation method based on deep learning

Abstract

The invention discloses an intelligent equipment and unmanned aerial vehicle cooperative navigation method based on deep learning, which comprises the following steps: acquiring spatial information and equipment state information of a target task area, and constructing a task structure chart; inputting to the improved KAN model, and outputting task structure vectors corresponding to the nodes; executing a task allocation strategy; equipment state information is collected in real time, subtask splitting score calculation is executed, and a plurality of subtask structure diagrams are generated; executing a task rescheduling strategy, and distributing the task rescheduling strategy to idle intelligent equipment or unmanned aerial vehicles to take over and execute; updating a task allocation mapping relation; and if all the node tasks in the sub-task graph are completed, marking the sub-task graph as completed task blocks until all the node tasks in the task structure graph are completed. According to the method, the improved KAN model is combined with the structural features and the spatial features, the task division and scheduling process is dynamically optimized, the task execution efficiency and the resource utilization rate are improved, and the method is suitable for multi-device collaborative navigation tasks in complex spatial scenes.

Description

Technical Field

[0001] This invention relates to the field of intelligent navigation technology, and in particular to a method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning. Background Technology

[0002] With the development of multi-agent systems and the widespread application of intelligent devices and drones in emergency response, environmental inspection, and complex task execution, graph-based cooperative navigation and task allocation technologies have gradually become a research hotspot. Existing cooperative navigation methods mainly rely on static rule settings or traditional graph optimization algorithms to divide and schedule task regions, lacking the ability to dynamically perceive and deeply model the states of heterogeneous devices and the spatial structure of tasks. This results in task scheduling outcomes that are difficult to adapt to the actual needs of complex environments with frequent changes in task states and significant differences in device capabilities. Some studies have introduced graph neural networks or attention mechanisms for task graph modeling, but these are mostly limited to fixed-structure graphs or local optimization models, lacking joint modeling methods for task node attributes and device states, resulting in insufficient model generalization and dynamic response capabilities.

[0003] Furthermore, most existing task rescheduling mechanisms employ static rules or a single priority-based strategy. When subtasks malfunction or equipment deteriorates, they cannot promptly perform dynamic splitting and efficient reallocation of subtask structures, impacting overall task completion efficiency and system stability. The methods for determining task completion status are also rather crude, often relying solely on task marker signals or simple location judgments, lacking a fusion mechanism of multi-dimensional task execution indicators. This leads to frequent delays or misjudgments in task completion determination.

[0004] Therefore, how to provide a method for collaborative navigation between intelligent devices and drones based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a deep learning-based collaborative navigation method for intelligent devices and unmanned aerial vehicles. This invention fully utilizes the modeling capabilities of graph neural networks and the improved KAN model's ability to jointly represent task structure and device state. It details the entire modeling process from task structure graph construction, task structure vector generation, initial task allocation, structure split determination, task rescheduling to state synchronization and completion marking. It has the advantages of strong task structure expression capability, fast dynamic task rescheduling response, high task execution accuracy, and strong collaborative navigation stability.

[0006] A method for collaborative navigation between a deep learning-based intelligent device and a drone according to an embodiment of the present invention includes the following steps: Step 1: Collect spatial information and device status information of the target task area to construct a task structure diagram; the nodes of the task structure diagram represent individual task points to be completed, and the edges represent the spatial adjacency relationships between task points. Step 2: Input the task structure graph and device status information into the improved KAN model, perform graph embedding and structure encoding, and output the task structure vector corresponding to the node; Step 3: Based on the task structure vector and device status information, execute the task allocation strategy, generate multiple sub-task graphs, and allocate them to smart devices or drones; Step 4: Collect device status information in real time and perform subtask splitting score calculation; if the splitting score is greater than the preset score threshold, perform structural splitting on the corresponding subtask graph to generate multiple subtask structure graphs. Step 5: Execute the task rescheduling strategy based on the subtask structure diagram, and assign the subtask structure diagram to idle smart devices or drones for execution; Step Six: Update the task allocation mapping relationship and synchronize the task status information of the corresponding nodes in the task structure diagram; Step 7: If all node tasks in the subtask graph are completed, mark the subtask graph as a completed task block until all node tasks in the task structure graph are completed.

[0007] Preferably, step one specifically comprises: Collect spatial information of the target task area, including two-dimensional or three-dimensional spatial coordinate information of task points, regional boundary data, and geographic or topological relationship information between task points; Collect device status information, including the spatial location, battery level, task execution capability parameters, and device identification information of each smart device or drone; A task structure diagram is constructed based on the spatial information and device status information. The nodes of the task structure diagram are the task points to be executed, the edges are the spatial adjacency relationships between the task points, and the node attributes include task coordinates, task priority, task cost, and task density index.

[0008] Preferably, step two specifically includes: The graph structure information, node attribute information, and device status information of the task structure graph are input into the improved KAN model; the graph structure information includes the adjacency matrix or edge weight representation between task points; the node attribute information includes task coordinates, task priority, task cost, and task density. The improved KAN model includes a structural channel function and a spatial channel function; the structural channel function receives graph structure information and performs structural feature modeling; the spatial channel function receives node attribute information and device status information and performs spatial feature modeling. By jointly modeling with structural channel functions and spatial channel functions, the task structure vector corresponding to each node in the task structure graph is output.

[0009] Preferably, the improved KAN model is as follows: Extract the graph structure information and node attribute information from the task structure graph; The graph structure information is input into the structure channel function, which performs structural feature encoding between task points and outputs a structure channel feature representation. The structure channel function includes a graph attention mechanism module and a graph structure feature aggregation module. The graph attention mechanism module calculates attention coefficients based on the adjacency matrix and structural correlation between task points, and the attention coefficients are normalized using the Softmax function. The graph structure feature aggregation module performs a weighted summation operation on the feature vectors of adjacent task points based on the normalized attention coefficients and outputs a structure channel feature representation. Node attribute information and device status information are input into the spatial channel function to perform joint modeling of node spatial attributes and device status, and output spatial channel feature representation. The spatial channel function includes a node status fusion module and a spatial feature extraction module. The node status fusion module performs feature concatenation on node attribute information and device status information. The spatial feature extraction module uses a two-layer perceptron structure for nonlinear mapping. Each layer of the perceptron structure includes a linear transformation layer and a ReLU activation function. The structural channel feature representation and the spatial channel feature representation are concatenated along the feature dimension to generate a fused feature representation; The fused feature representation is input into a three-layer feedforward neural network. The first two layers use the ReLU activation function, and the last layer outputs a fixed-dimensional task structure vector, thus obtaining the task structure vector corresponding to each node in the task structure graph.

[0010] Preferably, step three specifically includes: Extract the task structure vector and device status information corresponding to each node in the task structure graph; the device status information includes device spatial location, power level, task execution capability parameters, and task load information. Calculate the adaptation score between the task structure vector and the status information of each device; the adaptation score is generated by a weighted combination of spatial distance factor, power adaptation factor, task load factor and execution capability factor; Construct a scoring matrix; the rows of the scoring matrix represent task points, the columns represent devices, and each element in the matrix represents the matching score between the corresponding task point and the device; The graph partitioning operation is performed based on the scoring matrix, and the task structure graph is divided into multiple sub-task graphs using a load-balanced clustering partitioning method. Each subtask graph is assigned to the device or drone with the highest fitting score in the scoring matrix to build the initial task assignment mapping relationship.

[0011] Preferably, step four specifically includes: Collect device status information of intelligent devices or drones executing subtasks; the device status information includes device spatial location, battery level, task execution progress, task path offset value, and number of task failures; Input the task structure vector corresponding to each task node in the subtask graph and the device status information of the corresponding device into the subtask splitting scoring model, and perform splitting score calculation. The subtask splitting scoring model constructs a scoring function based on the path connection relationship between task nodes, the gradient of task density change, the task delay ratio and the task completion rate. The scoring function is linearly combined with multiple weighting factors to output a splitting score value. The split score is compared with the split score threshold. If the split score is greater than the split score threshold, the corresponding subtask graph is split. The structural splitting operation includes performing region partitioning based on the spatial adjacency relationship between task nodes and the difference in task density in the task structure graph. This divides the task nodes into multiple structurally independent subgraph regions, and each structurally independent subgraph region constitutes a subtask structure graph.

[0012] Preferably, step five specifically includes: Extract the task node information and task structure vector corresponding to each subtask structure diagram, and extract the device status information of all smart devices or drones. Construct a rescheduling scoring function. The rescheduling scoring function performs scoring calculations based on task node density, the probability of task execution failure of the device from which the structure splits originate, the idle degree of the device and the task path reconstruction cost, and outputs an adaptation score value. Based on the task rescheduling strategy, the subtask structure diagram and device status information are input into the rescheduling scoring function to calculate the compatibility score between each subtask structure diagram and all smart devices or drones. Construct a task redistribution matrix. The rows of the task redistribution matrix represent the subtask structure diagram, the columns represent idle or low-load smart devices or drones, and the matrix elements are the adaptation score values. Select the smart device or drone with the highest adaptation score as the scheduling object of the subtask structure diagram, and assign the subtask structure diagram to the smart device or drone to perform task takeover operations.

[0013] Preferably, step six specifically includes: Extract the subtask structure diagram and corresponding takeover device identification information after the task rescheduling strategy is executed; Update the task assignment mapping relationship between all task nodes and takeover devices in the subtask structure diagram of the task structure diagram, and replace the original assigned device identifier with the takeover device identifier. Extract real-time task execution information from various smart devices or drones, including completed task node identifiers, task execution time, power consumption, and actual path offset. Update the task status information of the corresponding task node in the task structure diagram according to the task execution information. The task status information includes the task completion status, execution duration, power consumption and task trajectory offset. Construct a task status synchronization table, store the updated task status information in the task status synchronization table, and bind it in real time with the node attributes in the task structure graph.

[0014] Preferably, step seven specifically comprises: The task status information of all task nodes in each subtask graph is statistically analyzed to determine whether the task status is completed. The criteria for determining whether the task status is completed are that the task execution duration is not less than the estimated time and the task trajectory offset is within the tolerance range. When all task nodes are in a completed state, the subtask graph completion check is triggered, and the task flag field is updated by identifying the completed task blocks in the subtask graph. Set the task status field of all task nodes in the completed task block to the completed status in the task structure diagram. Record the identifiers of completed task blocks and their respective device identifiers, and update the task completion progress table in the task structure diagram. The task completion progress table includes the total number of task nodes, the number of completed task nodes, and the number of task nodes to be completed. If all nodes in the task structure graph are in the "completed" state, then update the overall task status of the task structure graph to "completed".

[0015] Preferably, it also includes: building a task monitoring module, specifically: The task monitoring module includes a task status monitoring unit, a device status monitoring unit, and a task anomaly handling unit. The task status monitoring unit periodically acquires the task status information of each task node in the task structure diagram to monitor the task completion progress and changes in the task node status. The equipment status monitoring unit continuously collects the spatial location, power level, operating status and communication connection status of smart devices or drones, and analyzes the equipment operation trend and resource bottlenecks. The task exception handling unit determines whether there are any task failures, equipment malfunctions, or communication interruptions based on the information provided by the task status monitoring unit and the equipment status monitoring unit. If an exception event is detected, an exception task list is constructed, recording the exception type, scope of impact, and corresponding task node identifier. The abnormal task nodes are remarked as pending assignment, and a rescheduling process based on the task structure diagram and device status information is triggered to rebuild the adaptation score and perform scheduling and assignment operations.

[0016] The beneficial effects of this invention are: This invention constructs a task structure graph and a task structure vector representation model, and introduces an improved KAN model to achieve graph embedding and spatial structure encoding. It addresses the problems of complex distribution of multi-source task data, heterogeneous equipment capabilities, and dynamic changes in execution status. It adopts a task allocation strategy based on task structure vectors and equipment status information, a sub-task structure splitting judgment mechanism, and a task rescheduling process. It outputs task structure vectors by combining structural channel functions and spatial channel functions. In the task scheduling stage, it constructs a multi-dimensional adaptation scoring function, and uses the scoring matrix and rescheduling scoring function as the core for initial task allocation and task rescheduling control. During task execution, the granularity of subtask structure diagram division and execution carrier are dynamically adjusted through real-time subtask splitting scoring and structure diagram re-division processing, thereby improving the robustness and adaptability of task allocation. During the task monitoring phase, an anomaly detection and scheduling recovery mechanism is constructed by combining the task status synchronization and task completion flag update mechanism with the task status monitoring unit and the equipment status monitoring unit, thereby realizing the takeover of abnormal tasks and closed-loop control of task execution status during the task scheduling process. Ultimately, a deep learning-based collaborative navigation method for intelligent devices and drones was realized, enabling dynamic modeling and closed-loop optimization of the entire process of task division, allocation, execution, and status feedback in complex task environments. This significantly improved the accuracy of task structure representation, scheduling efficiency, and task execution stability. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a deep learning-based collaborative navigation method for intelligent devices and drones proposed in this invention; Figure 2 This is a schematic diagram of the improved KAN model structure proposed in this invention; Figure 3 This is a block diagram of the task monitoring module proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1-3 A deep learning-based collaborative navigation method for intelligent devices and drones includes the following steps: Step 1: Collect spatial information and device status information of the target task area to construct a task structure diagram; the nodes of the task structure diagram represent individual task points to be completed, and the edges represent the spatial adjacency relationships between task points. Step 2: Input the task structure graph and device status information into the improved KAN model, perform graph embedding and structure encoding, and output the task structure vector corresponding to the node; Step 3: Based on the task structure vector and device status information, execute the task allocation strategy, generate multiple sub-task graphs, and allocate them to smart devices or drones; Step 4: Collect device status information in real time and perform subtask splitting score calculation; if the splitting score is greater than the preset score threshold, perform structural splitting on the corresponding subtask graph to generate multiple subtask structure graphs. Step 5: Execute the task rescheduling strategy based on the subtask structure diagram, and assign the subtask structure diagram to idle smart devices or drones for execution; Step Six: Update the task allocation mapping relationship and synchronize the task status information of the corresponding nodes in the task structure diagram; Step 7: If all node tasks in the subtask graph are completed, mark the subtask graph as a completed task block until all node tasks in the task structure graph are completed.

[0021] This implementation constructs a task structure graph and introduces an improved KAN model to jointly model the graph structure information and node attribute information between task points, generating task structure vectors with spatial and structural semantics, thus improving the accuracy and depth of task representation. A task scoring matrix is ​​constructed by combining device status information, and a load balancing strategy is used for subtask graph partitioning and task assignment, achieving reasonable scheduling of multi-device resources. A splitting scoring function is used to structurally split the subtask graph during execution, effectively addressing dynamic scenarios such as uneven task density and sudden changes in device load, enhancing the adaptability of the task system. During task execution, task rescheduling is performed based on real-time status feedback information, and a task mapping update mechanism and a status synchronization mechanism are constructed to improve the closed-loop management capability of task execution. Finally, accurate tracking and management of the overall task status are achieved through task completion status statistics and progress updates. This implementation provides a deep learning-based intelligent device and UAV collaborative navigation method, which has advantages such as strong robustness, high adaptability, and high execution efficiency in task representation, scheduling allocation, and status control, and is suitable for intelligent application scenarios where multiple devices collaboratively execute complex tasks.

[0022] In this embodiment, step one specifically includes: Collect spatial information of the target task area, including obtaining two-dimensional or three-dimensional spatial coordinate information using high-precision surveying equipment or remote sensing data, extracting regional boundary data through lidar, panoramic images or UAV images, extracting spatial adjacency relationships, topological connectivity and obstacle distribution information between task points through geographic information system, and constructing a spatial connectivity description matrix for subsequent graph structure edge information generation; Collect device status information of smart devices and drones, including the initial spatial location, battery percentage, available running time, task execution capability parameters and real-time device identification number of each device, and encapsulate all device status information in a structured manner to form a unified device status input table; When constructing the task structure diagram, task points are used as nodes in the diagram. Node attributes include task coordinates, task priority, task cost, and task density index. Task priority is obtained by weighting the task urgency and the task density of the area. Task cost is calculated based on the equipment energy consumption model and path length estimation function. Task density index is obtained by fitting the number density of task points in a unit area with the distribution of cluster centers. The edges in the task structure graph are constructed by combining the K-nearest neighbor connection strategy with geographical topological constraints. The edge weights are calculated based on the Euclidean distance between task points and the obstacle interference factor, forming a spatial adjacency matrix and an edge weight matrix. The connectivity of the constructed task structure graph is verified through a structural consistency check mechanism to ensure that all task nodes belong to the connected subgraphs in the graph, and isolated nodes or redundant edge structures are removed to form the standard task structure graph format that is finally input into the modeling module.

[0023] In this embodiment, step two specifically includes: Extract graph structure information from the task structure graph, including extracting the adjacency matrix to indicate whether task points are directly connected, and the edge weight matrix to represent the spatial distance or geographical resistance coefficient of the connecting edges. Process the adjacency matrix and edge weight matrix using graph normalization methods to generate a standardized graph structure input tensor. Extract node attribute information, including concatenating the task coordinates, task priority, task cost, and task density information of each task point into a unified node feature vector to form a node attribute matrix. Extract device status information, including device spatial location, power level, task execution capability parameters, and task load information, and uniformly construct a device status representation vector. The graph structure information is input into the structure channel function, which includes a graph attention mechanism module and a graph structure feature aggregation module. The graph attention mechanism module calculates the attention weights between task points using an attention weight calculation function. The input to the weight function is the adjacency matrix and the node feature vectors, and the output is a normalized structure correlation coefficient. The normalization process is obtained by fitting the graph structure with the Softmax function. The graph structure feature aggregation module performs a weighted summation operation on the neighboring node feature vectors of each task point according to the attention coefficients, and outputs a structure channel feature representation. The dimension of the structure channel feature representation is consistent with the original node feature vectors. Node attribute information and device status information are input into a spatial channel function, which includes a node status fusion module and a spatial feature extraction module. The node status fusion module concatenates node attribute information and device status information in the channel dimension to form a fusion vector. The spatial feature extraction module adopts a two-layer perceptron structure, which includes a linear transformation layer and a ReLU activation function, to extract joint semantic features of device and task nodes and output a spatial channel feature representation. The structural channel feature representation and the spatial channel feature representation are concatenated along the feature dimension to generate a fused feature representation. The fused feature representation is then input into a three-layer feedforward neural network. The first two layers of the neural network use the ReLU activation function for nonlinear mapping, and the last layer outputs a vector representation with a fixed dimension. The output is the task structure vector corresponding to each node in the task structure graph. The task structure vector serves as a structured input representation for task allocation, task scheduling, and task state inference.

[0024] This implementation uses an improved KAN model to jointly model the spatial topology and device status in the task structure graph, enhancing the context awareness of task node representation and improving the adaptability and discriminativeness of the task structure vector in scenarios such as task allocation, subtask partitioning, and rescheduling.

[0025] In this embodiment, the improved KAN model is specifically as follows: The graph structure information includes an adjacency matrix, an edge weight matrix, and a task point spatial location matrix. The adjacency matrix is ​​generated through spatial adjacency rules, the edge weight matrix is ​​constructed from the spatial distance between task points or the task dependency strength, and the task point spatial location matrix records task coordinate information and is standardized. The node attribute information includes task priority, task cost, and task density. The task feature tensor is constructed through normalized encoding and used as a unified input feature representation. The graph structure information is input into the structure channel function. The graph attention mechanism module generates attention coefficients through a trainable attention weight function. The weight function is obtained by fitting the feature similarity and connection relationship between node pairs. The attention coefficients are normalized within the same node's neighbor set using the Softmax function. The structure feature aggregation module performs a weighted summation on the features of all adjacent task points based on the normalized attention coefficients to obtain the aggregated structure context representation. The dimension of the structure channel feature representation is consistent with the dimension of the node features, maintaining the stability of the embedding space. The formula for calculating the attention coefficient in the graph attention mechanism module: ; in, Represents a node For adjacent nodes Attention coefficient; and Represents nodes in the task structure graph With nodes The initial structural feature vector; The linear transformation weight matrix represents the structural channel function and is used to unify the feature dimensions; This represents the trainable weight vector in the attention mechanism; LeakyReLU represents the vector concatenation operation. This represents a ReLU activation function with leakage. This indicates that for all adjacent nodes in the set... Normalized attention weights.

[0026] Formula for graph structure feature aggregation module: ; in, Represents a node Structural channel characteristics representation; Represents nodes in the task structure graph The set of adjacent nodes; Represents a node For adjacent nodes Attention coefficient; Indicates adjacent nodes The eigenvectors after linear transformation; This represents a non-linear activation function, which can be either ReLU or ELU.

[0027] Node attribute information and device status information are input into the spatial channel function. The node status fusion module uses a feature concatenation operation to connect the task attribute vector of each task node with the status codes of all available devices. The device status codes include spatial location, power level, task execution capability, and task load ratio parameters. The concatenated vector is input into the spatial feature extraction module. The spatial feature extraction module consists of a two-layer perceptron structure. Each perceptron structure includes a linear transformation layer and a ReLU activation function. It extracts fused semantic features through nonlinear mapping and outputs a spatial channel feature representation. Node state fusion module splicing formula: ; in, Represents a node The fused input feature vector; Represents nodes in the task structure graph The node attribute vector includes task coordinates, task density, task cost, and task priority; Indicates the assignment of equipment The device state vector includes spatial location, power level, task execution capability, and task load; This indicates a vector concatenation operation.

[0028] The formula for the two-layer perceptron in the spatial feature extraction module is as follows: ; ; in, This represents the intermediate representation vector output by the first-layer perceptron; This represents the spatial channel characteristics of the output of the spatial channel function. and This represents the linear weight matrix of the two-layer perceptron; and This represents the bias term of the two-layer perceptron; This represents the activation function, used for nonlinear mapping.

[0029] The structural channel feature representation and the spatial channel feature representation are concatenated along the feature dimension to construct a fused feature tensor. The fused feature tensor is input into a three-layer feedforward neural network. The first two layers are linear transformation layers followed by ReLU activation functions. The output layer is a linear mapping function used to generate a fixed-dimensional task structure vector. The task structure vector and the task node correspond one-to-one to form a node embedding matrix.

[0030] In this embodiment, step three specifically includes: Extract the task structure vector corresponding to each task node in the task structure diagram; extract the device status information of all smart devices or drones, including device spatial location, power level, task execution capability parameters, and task load information; input the task structure vector and device status information into the task adaptation scoring function to perform adaptation score calculation. The scoring function is a weighted linear combination of spatial distance factor, power adaptation factor, task load factor, and execution capability factor, and the weight coefficients are obtained by fitting historical task completion rate and average energy consumption; output the adaptation score value between each task node and each device; construct a task adaptation score matrix, where rows represent task nodes, columns represent smart devices or drones, and matrix elements are the adaptation score values ​​between the corresponding task and device. A clustering graph partitioning operation is performed on the task adaptation scoring matrix. The graph partitioning operation adopts a clustering strategy based on minimizing device load deviation. The goal is to maximize the matching degree between the total load of the task nodes in each sub-task graph and the task capacity that the assigned device can bear. The load matching degree is obtained by fitting the difference between the aggregated task structure vector and the remaining execution capacity of the device. The task structure graph is divided into multiple load-balanced sub-task graphs. Based on the task adaptation score matrix, each subtask graph is assigned to the device or drone with the highest adaptation score; an initial task allocation mapping relationship is generated, and a mapping table between task nodes and assigned devices is established.

[0031] In this embodiment, step four specifically includes: Collect device status information of smart devices or drones that are executing subtasks; the device status information includes device spatial location, battery level, task execution progress, task path offset value and number of task failures; construct the task execution trajectory based on the task execution time series record, and the task execution trajectory is obtained by fitting the device position change with the task time window; Extract the task structure vectors corresponding to all task nodes in the subtask graph; input the task structure vectors and device status information into the subtask splitting scoring function and perform splitting score calculation; the input of the splitting scoring function is the set of task structure vectors and the set of task execution indicators of the corresponding device, and the output is the task stability discrimination score. The subtask splitting scoring function is a multi-factor weighted combination function constructed based on the path connectivity between task nodes, the gradient of task density change, the task delay ratio, and the task completion rate. The path connectivity is determined based on the edge weight information of the task structure graph. The gradient of task density change is obtained by fitting the density difference between adjacent nodes in the subtask graph. The task delay ratio is calculated by the ratio between the actual time consumed by the task node and the estimated time. The task completion rate is calculated by the ratio of the number of successful tasks to the total number of tasks. The above factors are weighted and combined to form the splitting score, and the weight parameters are obtained by training with historical scheduling data. The split score output value is compared with the split score threshold; if the split score is greater than the preset score threshold, it is determined that the subtask graph task scheduling stability is insufficient and a structural split operation needs to be performed. The structural splitting operation includes extracting the spatial coordinates and task density information of the task nodes in the sub-task graph; performing region partitioning based on the spatial adjacency and density differences between nodes; using a density gradient segmentation algorithm to generate boundary sets by fitting the distance matrix and density matrix between task nodes, dividing the graph into multiple structurally independent sub-graph regions; each sub-graph region constitutes a sub-task structure graph; the sub-task structure graphs are spatially non-overlapping and structurally independent with no intersection.

[0032] In this embodiment, step five specifically includes: Extract the task structure vectors and task coordinate information corresponding to all task nodes in each subtask structure graph; construct a feature set for the subtask structure graph, which includes task density distribution, structural complexity index, and historical scheduling failure count; the structural complexity index is obtained by fitting the graph diameter, average degree, and node clustering coefficient of the task structure graph. Collect device status information from all smart devices or drones; the device status information includes spatial location, power level, task execution capability parameters, task load ratio, and number of task takeovers; construct a device operating status vector, which is obtained by fitting the device's historical task execution records with an idle time window; A task rescheduling scoring function is constructed. The input of the scoring function is the feature set of the subtask structure graph and the device operating state vector, and the output is the scheduling adaptation score. The scoring function combines multiple factors based on task node density, failure probability of the original device, idle level of the target device, and path reconstruction cost. Task node density is calculated by the number of task nodes per unit area. Task execution failure probability is obtained by fitting the failure record ratio of the original device before structural splitting. Device idle level is obtained by weighted combination of device task load ratio and idle duration. Task path reconstruction cost is obtained by fitting the average path length from the subtask structure graph to the target device and the task trajectory complexity. The weighted combination of each factor forms the scheduling score value, and the weight parameters are obtained through multiple rounds of simulation optimization training. Input the subtask structure diagram and the running state vector of all devices into the task rescheduling scoring function to calculate the adaptation score between the entire subtask structure diagram and all candidate devices; construct the task reassignment matrix, where the rows of the task reassignment matrix represent the subtask structure diagram, the columns represent candidate devices, and each element in the matrix is ​​the corresponding adaptation score value. Select the device with the highest score in each row as the scheduling object of the subtask structure diagram and record the scheduling mapping table; assign the subtask structure diagram to the corresponding smart device or drone to take over and execute the task operation.

[0033] This implementation method can efficiently complete the dynamic matching between subtask structure diagrams and devices based on scheduling strategies when task execution is abnormal or resources are uneven, effectively improving the overall scheduling flexibility and resource utilization efficiency of the system.

[0034] In this embodiment, step six specifically includes: Extract the structure diagram identification information of all subtasks after the task rescheduling strategy is executed and the corresponding equipment identification information of the takeover equipment; construct a task mapping change table, in which each record includes the subtask structure diagram number, the original equipment identification, and the takeover equipment identification. Traverse all task nodes in the task structure diagram and identify the set of task node numbers belonging to each sub-task structure diagram; based on the task mapping change table, update the device assignment field corresponding to each task node from the original device identifier to the takeover device identifier, and construct the updated task allocation mapping relationship table. Collect real-time task execution information from various smart devices or drones; the task execution information is generated by the device-side task management module and includes the completed task node number, task execution time, power consumption value and actual path offset distance; the execution information is periodically transmitted back to the task management center through the communication module; Each task execution information is parsed; the corresponding task node in the task structure diagram is located according to the task node number; the task node status field is updated, which includes task completion status, task duration, power consumption, and trajectory offset; the task completion status is obtained by fitting the task time and the degree of matching between the task trajectory. A task status synchronization table is constructed, which adopts a key-value mapping structure. The key is the task node number, and the value is the set of updated task status fields. The task status synchronization table is bound to the attribute table of each node in the task structure diagram in real time to dynamically reflect the task execution progress and device load status.

[0035] This implementation method can achieve dynamic updates of task assignment relationships in the task structure diagram and high-frequency synchronization of task status, thereby improving the accuracy of task allocation and the real-time performance of system scheduling.

[0036] In this embodiment, step seven specifically includes: Extract the set of task node numbers from all subtask graphs, and extract the task status fields corresponding to each task node, including task execution duration, trajectory offset, and task completion status fields; construct a task completion judgment function based on preset task completion judgment conditions. The task completion judgment function makes logical judgments based on whether the task execution duration is not less than the estimated time and whether the task trajectory offset is within the tolerance range, and outputs the task completion flag. Execute the task completion judgment function for all task nodes in each subtask graph to generate a Boolean vector of task node completion status; count the number of nodes in the Boolean vector that are in the completion status and compare it with the total number of task nodes; if the two are equal, mark the subtask graph as a completed task block and update the task flag field of the corresponding subtask graph in the task graph identifier table. For all task nodes in the completed task block, update the node attribute table in the task structure graph, set the task completion status field to completed, and append the completion timestamp information. Construct a task completion record table. Each record in the task completion record table includes the completed task block number, the device identification information, and the task completion time. The record information is used for subsequent task scheduling statistical analysis and device scheduling efficiency evaluation. Traverse all nodes in the task structure graph by their task status fields, count the number of completed task nodes and the total number of task nodes, and generate a task completion progress table. The task completion progress table fields include the total number of task nodes, the number of completed task nodes, and the number of task nodes to be completed. If all nodes in the task structure graph are marked as completed, then update the overall task status of the task structure graph to completed.

[0037] In this embodiment, it also includes: constructing a task monitoring module, specifically: The task status monitoring unit sets a monitoring cycle time interval. The task status fields of all task nodes in the task structure diagram are periodically extracted. The task status fields include task completion status, execution duration, power consumption and trajectory offset. The above fields are combined into a state sampling matrix, and a task status time series is constructed by time sliding window method to analyze the state evolution trend of task nodes and task delay fluctuation characteristics. The device status monitoring unit collects spatial location information, power level, task execution load, and communication link status of each device based on its unique device identifier, with a collection frequency set to [missing information]. Generate a device status sampling matrix; analyze the device load growth curve and resource consumption rate curve through the device operation status model to identify device resource bottlenecks and potential task execution risk windows. The task anomaly handling unit constructs an anomaly event judgment function. This function combines indicators such as a sudden increase in task incomplete rate, abnormal power consumption, execution time exceeding expectations, and trajectory deviation exceeding limits in the task status time series to determine whether a task node has failed. It also combines indicators such as device communication disconnection, sudden power drop, or execution status freeze in the device status sampling matrix to determine whether the device has experienced an operational malfunction. For nodes or devices that meet the anomaly judgment function, anomaly events are registered, an anomaly task list is constructed, and the anomaly type, anomaly start time, affected task node number, and historically assigned device number are recorded. The task status field of the abnormal task node is reset to pending assignment, and the task assignment mapping table in the task structure diagram is updated synchronously. The abnormal task node is input into the task rescheduling process, and the adaptation score is recalculated by combining the current device status information and the task structure vector. The score value is generated by the rescheduling score function and the scheduling decision is completed.

[0038] In this embodiment, the task monitoring module can realize online dynamic monitoring and anomaly response throughout the entire task execution process, improve the adaptability and robustness of task scheduling in complex environments, effectively avoid problems such as the spread of task execution failures, waste of equipment resources and redundant waiting of task nodes, and support the stability and task completion efficiency improvement of the intelligent device and UAV collaborative navigation system.

[0039] Example 1: To verify the feasibility of this invention in practice, it was applied to an emergency supplies delivery scenario in a mountainous area. This scenario is complex, dynamically changing, and has an urgent need for resource scheduling.

[0040] A large-scale flood occurred in the mountainous area of ​​a city, causing road closures and rendering manual delivery impossible. The task required the coordinated deployment of ground-based intelligent equipment and aerial drones to deliver disaster relief supplies within the designated area. The area encompassed multiple temporary resettlement sites, affected areas, and mountainous outage points, necessitating the precise delivery of essential supplies, rescue equipment, and communication modules. A total of 213 task points were identified, with varying priorities and densities based on spatial clustering and importance. The task involved 17 ground-based wheeled intelligent devices and 9 multi-rotor drones, exhibiting a non-uniform density and high dynamism in its overall distribution.

[0041] In this scenario, the system first collects spatial coordinates of the task area, path accessibility map, obstacle distribution map, and real-time status of each device, including location, battery level, load capacity, and task execution records, to construct an initial task structure graph. Each node in the task structure graph represents a task to be executed, edges represent the feasibility of adjacent paths, and node attributes include task density, task cost, and urgency level. Subsequently, the graph structure information and node attributes are input into an improved KAN model. Spatial connectivity is modeled using structural channel functions, and device attributes and task characteristics are fused using spatial channel functions to generate node task structure vectors, which are then used for subsequent task allocation strategy execution.

[0042] During the scheduling phase, the model evaluates the suitability of the task structure vector with each device using a scoring function, constructs an initial scoring matrix, and automatically divides the task structure graph into 26 sub-task graphs based on a load-balanced clustering method. After initial allocation, the task structure graphs enter the execution phase, with devices transmitting task status and device information in real time. The system dynamically evaluates the task splitting score based on task completion rate, path deviation, and execution time. When some sub-task graphs exhibit excessive density or abnormal offset, the system automatically performs structure splitting to generate new sub-task structure graphs and reschedules the tasks.

[0043] During the three-month deployment period, the system executed 294 rounds of delivery scheduling, 87 task refactoring and rescheduling operations, 36 anomaly handling incidents, and 9 equipment failure handling incidents, achieving an overall task success rate of 98.6%. Under the same conditions, the success rate of traditional centralized scheduling systems was 91.2%. Furthermore, the system significantly outperformed traditional methods in key indicators such as scheduling time efficiency, average response time, and power consumption.

[0044] The following table shows the execution data of the system in actual task scheduling during the three-month statistical period.

[0045] Table 1. Statistics on Task Execution and Scheduling Efficiency ;

[0046] As shown in Table 1, this invention significantly reduces the average task response time to only 13.8 seconds, a reduction of approximately 35.8% compared to traditional methods, indicating that the system has higher responsiveness to task changes. Simultaneously, this invention achieves a larger average size of the subtask graph, effectively reducing scheduling frequency and improving task continuity. Regarding task completion rate, this invention achieves 98.6%, significantly higher than the traditional 91.2%, demonstrating stronger task execution stability. Furthermore, the average duration of a single round of scheduling is reduced to 3.6 seconds, improving scheduling efficiency. Power utilization efficiency is increased to 85.4%, indicating that the system more rationally controls energy consumption distribution during task planning. The automatic fault takeover rate reaches 94.7%, fully demonstrating the robustness and intelligence of this invention in fault detection and scheduling recovery, ensuring highly reliable task execution in dynamic environments.

[0047] Further statistics were compiled on the overall performance of each device type during task execution within the system's operating cycle. A performance comparison was made between ground-based intelligent devices and drones, as shown in the table below.

[0048] Table 2: Comparison of Equipment Task Execution Performance ;

[0049] Table 2 compares and analyzes intelligent ground equipment and multi-rotor UAVs from six dimensions: mission trajectory deviation, power consumption, failure rate, rescheduling frequency, mission continuity, and frequency of human intervention. It can be seen that intelligent ground equipment performs better in mission execution stability, with a mission trajectory deviation rate of only 3.2% and a mission failure rate of 1.6%. It also possesses a longer continuous mission execution capability, making it suitable for stable execution along medium- to long-distance paths. Although UAVs have a higher rescheduling frequency during execution, their mission trajectory deviation is smaller, indicating stronger path adaptability in complex terrain. Despite a slightly higher mission failure rate, UAVs consume less power and have a faster response speed, making them suitable for rapid backup and mission takeover. Both types of equipment leverage their respective strengths in the method of this invention, achieving complementary cooperation through dynamic mission structure reconstruction and scheduling mechanisms. This improves the overall scheduling flexibility and coverage of the system, reduces the frequency of human intervention, and effectively supports the intelligent execution of complex tasks.

[0050] This embodiment fully demonstrates that the deep learning-based intelligent device and UAV collaborative navigation method provided by the present invention has significant advantages in many aspects such as task allocation, rescheduling, structural splitting and task state synchronization in complex environments. It can achieve optimal task coverage for multi-device heterogeneous collaborative navigation under high dynamic conditions and has good universality, stability and scalability in practical applications.

[0051] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning, characterized in that, Includes the following steps: Step 1: Collect spatial information and device status information of the target task area to construct a task structure diagram; the nodes of the task structure diagram represent individual task points to be completed, and the edges represent the spatial adjacency relationships between task points. Step 2: Input the task structure graph and device status information into the improved KAN model, perform graph embedding and structure encoding, and output the task structure vector corresponding to the node; Step 3: Based on the task structure vector and device status information, execute the task allocation strategy, generate multiple sub-task graphs, and allocate them to smart devices or drones; Step 4: Collect device status information in real time and perform subtask splitting score calculation; if the splitting score is greater than the preset score threshold, perform structural splitting on the corresponding subtask graph to generate multiple subtask structure graphs. Step 5: Execute the task rescheduling strategy based on the subtask structure diagram, and assign the subtask structure diagram to idle smart devices or drones for execution; Step Six: Update the task allocation mapping relationship and synchronize the task status information of the corresponding nodes in the task structure diagram; Step 7: If all node tasks in the subtask graph are completed, mark the subtask graph as a completed task block until all node tasks in the task structure graph are completed.

2. The method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 1, characterized in that, Step one specifically involves: Collect spatial information of the target task area, including two-dimensional or three-dimensional spatial coordinate information of task points, regional boundary data, and geographic or topological relationship information between task points; Collect device status information, including the spatial location, battery level, task execution capability parameters, and device identification information of each smart device or drone; A task structure diagram is constructed based on the spatial information and device status information. The nodes of the task structure diagram are the task points to be executed, the edges are the spatial adjacency relationships between the task points, and the node attributes include task coordinates, task priority, task cost, and task density index.

3. The method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 2, characterized in that, Step two specifically involves: The graph structure information, node attribute information, and device status information of the task structure graph are input into the improved KAN model; the graph structure information includes the adjacency matrix or edge weight representation between task points; the node attribute information includes task coordinates, task priority, task cost, and task density. The improved KAN model includes a structural channel function and a spatial channel function; the structural channel function receives graph structure information and performs structural feature modeling; the spatial channel function receives node attribute information and device status information and performs spatial feature modeling. By jointly modeling with structural channel functions and spatial channel functions, the task structure vector corresponding to each node in the task structure graph is output.

4. The method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 3, characterized in that, The improved KAN model is specifically as follows: Extract the graph structure information and node attribute information from the task structure graph; The graph structure information is input into the structure channel function, which performs structural feature encoding between task points and outputs a structure channel feature representation. The structure channel function includes a graph attention mechanism module and a graph structure feature aggregation module. The graph attention mechanism module calculates attention coefficients based on the adjacency matrix and structural correlation between task points, and the attention coefficients are normalized using the Softmax function. The graph structure feature aggregation module performs a weighted summation operation on the feature vectors of adjacent task points based on the normalized attention coefficients, and outputs a structural channel feature representation. Node attribute information and device status information are input into the spatial channel function to perform joint modeling of node spatial attributes and device status, and output spatial channel feature representation; the spatial channel function includes a node status fusion module and a spatial feature extraction module, and the node status fusion module performs feature concatenation on node attribute information and device status information; The spatial feature extraction module uses a two-layer perceptron structure for nonlinear mapping, and each layer of the perceptron structure includes a linear transformation layer and a ReLU activation function; The structural channel feature representation and the spatial channel feature representation are concatenated along the feature dimension to generate a fused feature representation; The fused feature representation is input into a three-layer feedforward neural network. The first two layers use the ReLU activation function, and the last layer outputs a fixed-dimensional task structure vector, thus obtaining the task structure vector corresponding to each node in the task structure graph.

5. The method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 4, characterized in that, Step three specifically involves: Extract the task structure vector and device status information corresponding to each node in the task structure graph; the device status information includes device spatial location, power level, task execution capability parameters, and task load information. Calculate the adaptation score between the task structure vector and the status information of each device; the adaptation score is generated by a weighted combination of spatial distance factor, power adaptation factor, task load factor and execution capability factor; Construct a scoring matrix; the rows of the scoring matrix represent task points, the columns represent devices, and each element in the matrix represents the matching score between the corresponding task point and the device; The graph partitioning operation is performed based on the scoring matrix, and the task structure graph is divided into multiple sub-task graphs using a load-balanced clustering partitioning method. Each subtask graph is assigned to the device or drone with the highest fitting score in the scoring matrix to build the initial task assignment mapping relationship.

6. The method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 5, characterized in that, Step four specifically involves: Collect device status information of intelligent devices or drones executing subtasks; the device status information includes device spatial location, battery level, task execution progress, task path offset value, and number of task failures; Input the task structure vector corresponding to each task node in the subtask graph and the device status information of the corresponding device into the subtask splitting scoring model, and perform splitting score calculation. The subtask splitting scoring model constructs a scoring function based on the path connection relationship between task nodes, the gradient of task density change, the task delay ratio and the task completion rate. The scoring function is linearly combined with multiple weighting factors to output a splitting score value. The split score is compared with the split score threshold. If the split score is greater than the split score threshold, the corresponding subtask graph is split. The structural splitting operation includes performing region partitioning based on the spatial adjacency relationship between task nodes and the difference in task density in the task structure graph. This divides the task nodes into multiple structurally independent subgraph regions, and each structurally independent subgraph region constitutes a subtask structure graph.

7. The method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 6, characterized in that, Step five specifically involves: Extract the task node information and task structure vector corresponding to each subtask structure diagram, and extract the device status information of all smart devices or drones. Construct a rescheduling scoring function. The rescheduling scoring function performs scoring calculations based on task node density, the probability of task execution failure of the device from which the structure splits originate, the idle degree of the device and the task path reconstruction cost, and outputs an adaptation score value. Based on the task rescheduling strategy, the subtask structure diagram and device status information are input into the rescheduling scoring function to calculate the compatibility score between each subtask structure diagram and all smart devices or drones. Construct a task redistribution matrix. The rows of the task redistribution matrix represent the subtask structure diagram, the columns represent idle or low-load smart devices or drones, and the matrix elements are the adaptation score values. Select the smart device or drone with the highest adaptation score as the scheduling object of the subtask structure diagram, and assign the subtask structure diagram to the smart device or drone to perform task takeover operations.

8. The method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 7, characterized in that, Step six specifically involves: Extract the subtask structure diagram and corresponding takeover device identification information after the task rescheduling strategy is executed; Update the task assignment mapping relationship between all task nodes and takeover devices in the subtask structure diagram of the task structure diagram, and replace the original assigned device identifier with the takeover device identifier. Extract real-time task execution information from various smart devices or drones, including completed task node identifiers, task execution time, power consumption, and actual path offset. Update the task status information of the corresponding task node in the task structure diagram according to the task execution information. The task status information includes the task completion status, execution duration, power consumption and task trajectory offset. Construct a task status synchronization table, store the updated task status information in the task status synchronization table, and bind it in real time with the node attributes in the task structure graph.

9. A method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 8, characterized in that, Step seven specifically involves: The task status information of all task nodes in each subtask graph is statistically analyzed to determine whether the task status is completed. The criteria for determining whether the task status is completed are that the task execution duration is not less than the estimated time and the task trajectory offset is within the tolerance range. When all task nodes are in a completed state, the subtask graph completion check is triggered, and the task flag field is updated by identifying the completed task blocks in the subtask graph. Set the task status field of all task nodes in the completed task block to the completed status in the task structure diagram. Record the identifiers of completed task blocks and their respective device identifiers, and update the task completion progress table in the task structure diagram. The task completion progress table includes the total number of task nodes, the number of completed task nodes, and the number of task nodes to be completed. If all nodes in the task structure graph are in the "completed" state, then update the overall task status of the task structure graph to "completed".

10. A method for collaborative navigation between intelligent devices and unmanned aerial vehicles based on deep learning according to claim 9, characterized in that, Also includes: The task monitoring module is constructed as follows: The task monitoring module includes a task status monitoring unit, a device status monitoring unit, and a task anomaly handling unit. The task status monitoring unit periodically acquires the task status information of each task node in the task structure diagram to monitor the task completion progress and changes in the task node status. The equipment status monitoring unit continuously collects the spatial location, power level, operating status and communication connection status of smart devices or drones, and analyzes the equipment operation trend and resource bottlenecks. The task exception handling unit determines whether there is a task failure, equipment malfunction, or communication interruption event based on the information provided by the task status monitoring unit and the equipment status monitoring unit. If an abnormal event is detected, an abnormal task list is constructed, recording the abnormal type, scope of impact, and corresponding task node identifier; The abnormal task nodes are remarked as pending assignment, and a rescheduling process based on the task structure diagram and device status information is triggered to rebuild the adaptation score and perform scheduling and assignment operations.

Images