Unmanned system-oriented dual-channel knowledge-driven network trajectory prediction method and system

Abstract

The invention belongs to the technical field of artificial intelligence and unmanned systems, and particularly discloses an unmanned system-oriented dual-channel knowledge-driven network trajectory prediction method, which comprises the following steps of: receiving scene data of a dynamic target in an operation scene of an unmanned system, and constructing a dynamic heterogeneous graph comprising a plurality of semantic nodes and a plurality of connection relationships based on the scene data; a two-channel encoder is adopted to encode the dynamic heterogeneous graph, and a first feature sequence representing dynamic interaction and a second feature sequence representing instantaneous physical geometry are obtained; the dual-channel encoder comprises a knowledge-driven dynamic interaction channel and a parallel instantaneous physical geometry channel; fusing the features of the first feature sequence and the second sequence by adopting a dynamic gating mechanism to obtain a unified scene context vector; and decoding the scene context vector by adopting a query-based multi-mode decoder to generate at least one group of prediction tracks and determine confidence scores corresponding to the prediction tracks. According to the invention, the accuracy and reliability of trajectory prediction can be improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence and unmanned systems technology, and more specifically, relates to a dual-channel knowledge-driven network trajectory prediction method and system for unmanned systems. Background Technology

[0002] Trajectory prediction is a key technology for ensuring the safe and efficient operation of unmanned systems in dynamic environments, and it is essential for path planning, decision-making, and collision avoidance. Whether it is an unmanned vehicle driving on urban roads, a drone shuttling in low-altitude airspace, or an unmanned vessel operating on busy waterways, it is necessary to accurately predict the future trajectories of other dynamic entities in the surrounding area (such as other vehicles and pedestrians).

[0003] Research on trajectory prediction models has evolved from physics-based models to data-driven deep learning methods. Early deep learning methods often employed rasterized representations, rendering scenes as bird's-eye views and processing them using convolutional neural networks (CNNs). However, this approach suffers from information loss and computational inefficiency, especially when dealing with sparse but structured networks of roads or flight paths. To overcome these limitations, the industry shifted to vectorization-based methods, utilizing graph neural networks (GNNs) to directly process the vectorized representations of scene elements, which has become the current state-of-the-art approach. Models such as VectorNet and LaneGCN have made significant progress by constructing graphs to capture scene topology and relationships between entities.

[0004] However, existing GNN-based methods still suffer from several key bottlenecks. First, there's the "black box" learning problem. Many models rely on general graph aggregation mechanisms, such as standard Graph Attention Networks (GATs), which essentially treat complex traffic interactions as a "black box" to be learned. This purely data-driven paradigm requires the model to learn basic physical intuitions or behavioral norms from scratch using massive amounts of data, leading to low learning efficiency and a lack of transparency and interpretability in the model's decision-making process. In safety-critical autonomous systems, a model that cannot explain its reasoning process poses a significant risk; it is sensitive to data bias, performs poorly in rare "long-tail scenarios," and is difficult to trust.

[0005] Secondly, there is the problem of incomplete semantic representation. Existing technologies also have significant shortcomings in terms of the granularity of graph construction and semantic completeness. Even advanced heterogeneous graph network models have relatively coarse node partitioning. More importantly, they often treat abstract operational rules (such as speed limits, right-of-way, and no-fly zone regulations) merely as additional attributes of physical entities, failing to model them as independent elements in the graph that can directly participate in reasoning. This approach limits the depth of the model's understanding of the scene, making it difficult to truly reason about "how rules affect behavior within the physical space," thus affecting the accuracy of predictions.

[0006] Therefore, there is an urgent need in this field for a new technical solution that can construct a more granular and semantically powerful general scene graph, especially one that can materialize abstract rules and design a network architecture that can directly inject domain knowledge. This would solve the problems in existing technologies, such as the lack of effective integration of domain knowledge in the learning process, resulting in poor model interpretability, weak generalization ability, incomplete semantic representation, and superficial understanding of rules. Summary of the Invention

[0007] In view of the shortcomings of the prior art, the purpose of this application is to solve the problems in the prior art, such as the lack of effective integration of domain knowledge in the learning process, resulting in poor model interpretability, weak generalization ability, incomplete semantic representation and superficial understanding of rules, and to improve the accuracy and reliability of prediction.

[0008] To achieve the above objectives, in a first aspect, this application provides a dual-channel knowledge-driven network trajectory prediction method for unmanned systems, comprising: Receive scene data of dynamic targets within the unmanned system operation scenario, and construct a dynamic heterogeneous graph including multiple semantic nodes and multiple connection relationships based on the scene data; The dynamic heterogeneous graph is encoded using a dual-channel encoder to obtain a first feature sequence representing dynamic interaction and a second feature sequence representing instantaneous physical geometry; A dynamic gating mechanism is used to fuse the first feature sequence and the second feature sequence to obtain a unified scene context vector; The scene context vector is decoded using a query-based multimodal decoder to generate at least one set of predicted trajectories and determine the confidence scores corresponding to the predicted trajectories.

[0009] Optionally, the method for constructing the dynamic heterogeneous graph includes: The dynamic target's historical trajectory data and static environment data are acquired as the scene data, and all scene data are transformed into a local coordinate system; the local coordinate system takes the current position of the target to be predicted as the origin and the current orientation as the reference. Based on the transformed scene data, define and initialize the multi-type nodes in the dynamic heterogeneous graph; Based on the spatial and semantic relationships between the various types of nodes, a variety of edges are constructed in the dynamic heterogeneous graph, and the construction of the dynamic heterogeneous graph is completed based on the various types of nodes and edges.

[0010] Optionally, the multi-type nodes of the dynamic heterogeneous graph include at least dynamic target nodes, static path element nodes, and abstract rule nodes that are the embodiment of abstract running rules; The dynamic target nodes include nodes corresponding to drones, unmanned vehicles, or unmanned boats; The static path element nodes include nodes corresponding to lanes, routes, or waterways; The abstract rule nodes include nodes that materialize abstract operational rules for speed limits, no-fly zones, or right-of-way.

[0011] Optionally, the multi-type edges include at least: interaction edges between dynamic target nodes, interaction edges between dynamic target nodes and static path element nodes, topological edges between static path element nodes, and connection edges between abstract rule nodes and the static path element nodes under their jurisdiction.

[0012] Optionally, the dual-channel encoder includes a knowledge-driven dynamic interaction channel and an instantaneous physical geometry channel; The encoding process of the dynamic interactive channel includes: Determine the edge characteristics and edge types between the target dynamic node and its neighboring nodes, and determine the message passing function corresponding to the edge type; The hidden states from each neighboring node at the previous time step are aggregated using the message passing function to obtain the context message of the target dynamic node; The context message, along with the original input features of the target dynamic node at the current time step and its hidden state at the previous time step, are input into the gated loop unit to update the hidden state of the target dynamic node at the current time step. Iterative processing is performed on all time steps of the historical sequence to output the feature sequence of each target dynamic node capturing the dynamic interaction history, which serves as the first feature sequence of the dynamic interaction channel.

[0013] Optionally, the encoding process of the instantaneous physical geometry channel includes: At each time step of the historical sequence, search for all neighboring targets within the target radius of the target to be predicted, and transform the coordinates of all neighboring targets to the local coordinate system of the target to be predicted to obtain an aligned list of neighboring coordinates; The list of neighboring coordinates is input into a shared multilayer perceptron to encode and generate a vector representing the current instantaneous geometric layout; Alignment and vector encoding operations are repeatedly performed on each time step of the historical sequence, and a feature sequence composed of instantaneous geometric layout vectors of each time step is output as the second feature sequence of the instantaneous physical geometry channel.

[0014] Optionally, the process of obtaining the scene context vector includes: The first feature sequence and the second feature sequence are concatenated using a multilayer perceptron to calculate the fusion gate. Based on the multilayer perceptron corresponding to the first feature sequence combined with the fusion gate, the first weighted feature of the first feature sequence after weighting by the fusion gate is obtained by performing element-wise multiplication through the hyperbolic tangent activation function. Based on the multilayer perceptron corresponding to the second feature sequence combined with the fusion gate, the element-wise multiplication calculation is performed through the hyperbolic tangent activation function to obtain the second weighted feature of the second feature sequence after being weighted by the fusion gate. The first weighted feature and the second weighted feature are weighted and fused to obtain the fused feature sequence; The fused feature sequences are aggregated using a recurrent neural network to generate the unified scene context vector.

[0015] Optionally, the step of using a query-based multimodal decoder to decode the scene context vector, generate at least one set of predicted trajectories, and determine the confidence scores corresponding to the predicted trajectories includes: Learnable modal anchors are combined with the scene context vector and initialized together to determine the initial hidden state; For each time step, a temporary state to be optimized is determined based on the state of the previous time step and the predicted position, and the temporary state is used as a query vector based on the cross-attention mechanism. The query context vector is determined by querying the final set of feature vectors of all nodes in the dynamic heterogeneous graph at the end of the historical sequence, output by the dynamic interaction channel, based on the query vector. The final hidden state is generated by fusing the query context vector and the temporary state, and the final hidden state is input into the prediction head to generate the prediction position point of the current time step. The prediction position points of each time step are combined to obtain the prediction trajectory. Determine the final hidden state of each modal anchor point after the prediction trajectory generation process is completed, and input each final hidden state into the confidence prediction head to obtain the confidence score corresponding to each prediction trajectory.

[0016] This application provides a dual-channel knowledge-driven network trajectory prediction system for unmanned systems, including: The graph construction module is used to receive scene data of dynamic targets in the unmanned system operation scenario, and construct a dynamic heterogeneous graph including multiple semantic nodes and multiple connection relationships based on the scene data. A dual-channel encoding module is used to encode the dynamic heterogeneous graph using a dual-channel encoder to obtain a first feature sequence representing dynamic interaction and a second feature sequence representing instantaneous physical geometry; The feature fusion module is used to fuse the first feature sequence and the second feature sequence using a dynamic gating mechanism to obtain a unified scene context vector; The decoding module is used to decode the scene context vector using a query-based multimodal decoder, generate at least one set of predicted trajectories, and determine the confidence scores corresponding to the predicted trajectories.

[0017] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0018] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0019] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0020] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0021] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application constructs a structured, semantically rich, dynamic heterogeneous graph, which uniformly represents dynamic targets, static environments, and abstract operating rules as nodes in the graph. Various types of connecting edges are established based on their spatial, topological, and semantic relationships, thus forming a complete and hierarchical scene semantic representation. A dual-channel encoding architecture is adopted to decouple scene understanding into knowledge-driven interactive reasoning and instantaneous geometric snapshot capture. A gating mechanism is designed for dynamic fusion, enabling the model to simultaneously learn deliberate behavioral patterns and rapid reaction instincts. During decoding, learnable anchors are used to initialize multimodal intents, and an autoregressive mechanism that actively queries the global scene context in each generation step ensures that long-term predictions remain consistent with the dynamic environment. Finally, trajectory confidence is evaluated based on the complete generation history. This application deeply embeds domain knowledge into the model structure, providing not only a clear explanation path but also enhancing the model's generalization ability across different unmanned system platforms through the design of nodes and edges. This improves model interpretability, enhances semantic representation and rule understanding capabilities, and ultimately improves the accuracy and reliability of predictions.

[0022] (2) This application injects domain knowledge as an inductive bias into the network structure, combined with precise capture of scene geometry and an active decoding mechanism, to better understand complex dynamic environments. This application eliminates the reliance on black-box general graph attention mechanisms. By explicitly modeling abstract rules as reasonable graph nodes and designing dedicated message passing functions with clear physical or behavioral meanings for different interaction types, the model's decision-making process becomes more transparent and interpretable. In safety-critical unmanned system applications, this enhances the system's reliability, facilitating model debugging, verification, and gaining the trust of regulatory agencies and users.

[0023] (3) The algorithm framework proposed in this application has high modularity and scalability. By abstracting scene elements into general categories such as dynamic targets, path elements and abstract rules, the framework can be seamlessly applied to unmanned systems in different fields such as land, sea and air. It can quickly adapt to new application scenarios by simply replacing or extending the definitions of nodes and edges in heterogeneous graphs and adjusting the knowledge-driven message passing function accordingly, thus demonstrating stronger robustness and generalization ability.

[0024] (4) This application, through an innovative query-based decoder, combined with learnable modal anchors and an active scene context query mechanism, can generate diverse future trajectories that are highly logically consistent with the scene context. This effectively overcomes the limitation of traditional decoders in long-term predictions, which gradually become detached from the environment due to the use of open-loop generation, and ensures that the prediction results remain reasonable throughout the entire time domain. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the dual-channel knowledge-driven network trajectory prediction method for unmanned systems provided in an embodiment of this application. Figure 2 This is a schematic diagram of the network architecture of the prediction system model provided in the embodiments of this application; Figure 3 This is a schematic diagram of a complex traffic scenario provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of the dual-channel knowledge-driven network trajectory prediction device for unmanned systems provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0028] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0029] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0030] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0031] The embodiments of this application are described below with reference to the accompanying drawings.

[0032] This application provides a dual-channel knowledge-driven network trajectory prediction method for unmanned systems, including: Receive scene data of dynamic targets within the unmanned system operation scenario, and construct a dynamic heterogeneous graph including multiple semantic nodes and multiple connection relationships based on the scene data; The dynamic heterogeneous graph is encoded using a dual-channel encoder to obtain a first feature sequence representing dynamic interaction and a second feature sequence representing instantaneous physical geometry; A dynamic gating mechanism is used to fuse the first feature sequence and the second feature sequence to obtain a unified scene context vector; The scene context vector is decoded using a query-based multimodal decoder to generate at least one set of predicted trajectories and determine the confidence scores corresponding to the predicted trajectories.

[0033] Reference Figure 1 In summary, the above process includes the following steps: S1: Dynamic heterogeneous graph construction; S2: Scene coding based on a dual-channel architecture; S3: Dynamic fusion and contextual aggregation of dual-channel features; S4: Multimodal trajectory decoding based on query mechanism.

[0034] The corresponding system network architecture is as follows: Figure 2 As shown, the system first encodes features for vehicles and map elements and constructs a dynamic heterogeneous map. A dual-channel encoder processes interactive and geometric information in parallel. After fusion, the features are fed into a query-based decoder to ultimately generate a multimodal trajectory.

[0035] Specifically, in S1, this embodiment first receives raw data of the unmanned system's operating scenario, including the historical trajectory of dynamic targets and the static environment. This scenario data is uniformly converted to a local coordinate system centered on the current target to be predicted. Then, this heterogeneous information is constructed into a structured dynamic heterogeneous graph. The multi-type nodes of the dynamic heterogeneous graph include at least dynamic target nodes, static path element nodes, and abstract rule nodes that represent abstract operating rules. The dynamic target nodes include nodes corresponding to drones, unmanned vehicles, or unmanned surface vessels; the static path element nodes include nodes corresponding to lanes, flight paths, or waterways; and the abstract rule nodes include nodes that represent abstract operating rules for speed limits, no-fly zones, or right-of-way. The multi-type edges include at least: interaction edges between dynamic target nodes, interaction edges between dynamic target nodes and static path element nodes, topological edges between static path element nodes, and connection edges between abstract rule nodes and the static path element nodes they govern.

[0036] It should be further explained that the abstract rule node establishes an explicit connection with the static path element node under its jurisdiction through the map-rule connection edge, thereby providing a structured path for the constraint effect of the model inference rule on the physical space.

[0037] To comprehensively understand the scenario, this embodiment employs a parallel dual-channel encoder in step S2 to process the dynamic heterogeneous graph. One channel is a knowledge-driven dynamic interaction graph network that iteratively runs over historical time series, simulating domain-compliant interactions between targets and between targets and the environment. This channel selectively aggregates information by designing dedicated message passing functions for different types of edges, ultimately outputting a feature sequence that encodes historical interaction patterns and behavioral logic. The other parallel channel is an instantaneous physical geometry encoder, focused on capturing snapshots of the scene at each historical moment. By aligning neighboring targets to the coordinate system of the target to be predicted and encoding their relative positions, it obtains a geometric feature sequence representing the instantaneous spatial layout.

[0038] Furthermore, after obtaining the two feature sequences mentioned above, S3 is used to intelligently fuse these feature sequences through an adaptive gating mechanism. This mechanism calculates a dynamic fusion weight for each historical moment, which is jointly determined by the features of the two channels. This weight is used to weigh whether the current time step should rely more on the behavioral knowledge learned from historical interactions or on the real-time geometric layout information.

[0039] The weighted and fused feature sequences are then input into a recurrent neural network for temporal aggregation, ultimately generating a unified and condensed scene context vector. This scene context vector integrates the dynamic evolution history, physical structure, and implicit rule semantics of the scene, representing a deep understanding of the current situation.

[0040] Finally, through S4, the decoder generates multiple sets of future trajectories based on the aforementioned scene context vectors through proactive querying. The decoding process begins by combining a set of learnable modal anchors with the scene context to initialize multiple decoding branches representing different higher-order behavioral intentions. At each step of the autoregressive trajectory generation, each branch uses its current decoding state as a query, proactively performing cross-attention retrieval on the global scene node features retained by the encoder, thereby dynamically acquiring the environmental information most relevant to the current generation intention. The retrieved information is used to correct and decode the state and predict the next location point.

[0041] After all trajectories have been generated, the completeness and rationality of each trajectory are evaluated based on the final state accumulated by each branch at the end of the entire generation cycle, and the corresponding confidence score is output, thus completing the entire process from scene understanding to multimodal, interpretable trajectory prediction.

[0042] Optionally, the method for constructing the dynamic heterogeneous graph includes: The dynamic target's historical trajectory data and static environment data are acquired as the scene data, and all scene data are transformed into a local coordinate system; the local coordinate system takes the current position of the target to be predicted as the origin and the current orientation as the reference. Based on the transformed scene data, define and initialize the multi-type nodes in the dynamic heterogeneous graph; Based on the spatial and semantic relationships between the various types of nodes, a variety of edges are constructed in the dynamic heterogeneous graph, and the construction of the dynamic heterogeneous graph is completed based on the various types of nodes and edges.

[0043] This embodiment provides a detailed description of the process of constructing the dynamic heterogeneous graph of S1 described above.

[0044] This step aims to transform the complex physical scenario of unmanned system operation into a structured mathematical representation rich in semantic information, namely a dynamic heterogeneous graph. This provides high-quality input for subsequent deep learning models.

[0045] Specifically, the following steps are included: S1-1: Scene Data Acquisition and Feature Encoding This method first acquires and processes raw, heterogeneous scene data. This data includes, but is not limited to, trajectory data of dynamic targets (such as unmanned vehicles, drones, and unmanned boats) over historical time periods, and high-precision map information of the static environment. To ensure the consistency and robustness of the model processing, all data is transformed to a unified coordinate system, namely a local coordinate system centered on the target to be predicted, with its origin at the target's current position and the x-axis pointing towards its current orientation. This coordinate system normalization step ensures the method's invariance to translation and rotation, enhancing its applicability to different physical scenarios.

[0046] S1-2: Definition and representation of multiple types of nodes in heterogeneous graphs: To capture the characteristics and interactions of different elements in a scene with fine detail, this application defines several types of nodes, each carrying unique semantic information. For example... Figure 3 As shown, a complex traffic scenario is presented, which includes vehicles (dynamic target nodes), static path element nodes, and abstract rule nodes such as speed limits. The node design will be explained in detail below.

[0047] (1) Dynamic target node Each dynamic target in the scene is defined as a dynamic target node. Its state is represented as a historical time series. The sequence contains information about the target in the past. The 8-dimensional kinematic state within a time step specifically includes: position ,speed( Orientation angle And higher-order dynamic characteristics obtained by first-order difference calculation of velocity and orientation: acceleration and angular velocity The feature sequence is encoded by a multilayer perceptron (MLP) encoder. It is mapped to the uniform hidden dimension required by the model. .

[0048] (2) Static path element nodes In this scenario, all paths or channels that can be followed by unmanned systems are defined as static path element nodes. The feature representation of this node is a fusion of geometric and semantic information. Its geometric features... By employing a PointNet-like architecture to encode the sampled point cloud of the path centerline, its precise geometry and semantic features are captured. It is a one-hot encoded vector used to represent the type of the path element. For example, it is "lane" for ground unmanned systems, "route" for air unmanned systems, and "channel" and boundary attributes for water surface unmanned systems.

[0049] These two feature vectors are concatenated and then finally mapped using a fusion layer (such as an MLP). This universal definition of "path element" enables this application to be seamlessly applied to unmanned systems in different fields such as land, sea, and air.

[0050] (3) Abstract rule nodes This is a core innovation of this application. Unlike existing technologies that treat rules as attributes of physical entities, this application instantiates abstract operational rules as an independent class of nodes in a graph. These rules can be "speed limit of 60 km / h" or "stop and yield" for ground systems, "airspace control instructions" for air systems, or "collision avoidance rules" for surface water systems. Each node can be represented as a one-hot encoded vector directly corresponding to a specific rule. This design materializes abstract concepts, making rules first-class citizens in the graph structure that can directly participate in information transmission and reasoning. This provides a structured foundation for the model to learn "how rules affect behavior within physical space," enhancing the model's interpretability and reasoning ability. S1-3: Definition and Construction of Multiple Edge Types in Heterogeneous Graphs: Based on the aforementioned multiple node types, this application constructs a dynamic heterogeneous graph containing various types of connection relationships (edges). Edge set It includes static topological connections and dynamic interaction relationships; the characteristics of an edge are determined by the relative poses of the connected nodes. Composition. The key edge type definitions are as follows: (1) Target-target interaction edge A bidirectional edge is dynamically established between dynamic targets based on a distance criterion, used to model the direct mutual influence between targets.

[0051] (2) Target-map interaction edge Dynamic connections are established between a dynamic target and various map elements (such as path elements and environmental element nodes) within its perception range.

[0052] (3) Path-path topology edges Static edges, established based on prior knowledge of the map, represent the inherent topological relationships between path elements, such as successor, predecessor, left neighbor, and right neighbor relationships.

[0053] (4) Map-rule connecting edges This involves connecting abstract rule nodes to the path element nodes they govern. For example, a "speed limit 60" rule node will be connected to all path nodes with a speed limit of 60. This explicit connection provides a direct, structured pathway for the constraint effect of model inference rules on specific physical spaces.

[0054] Optionally, the dual-channel encoder includes a knowledge-driven dynamic interaction channel and an instantaneous physical geometry channel; The methods for obtaining the first feature sequence and the second feature sequence include: Using the dynamic interaction channel, message passing and state updates are performed on the dynamic heterogeneous graph through an iteratively evolving heterogeneous graph neural network to simulate a dynamic interaction process that conforms to domain knowledge and output the first feature sequence. Using the instantaneous physical geometry channel, the coordinates of other dynamic targets near the target to be predicted are transformed to the local coordinate system and encoded at each historical moment to obtain the second feature sequence.

[0055] Furthermore, the encoding process of the dynamic interactive channel includes: Determine the edge characteristics and edge types between the target dynamic node and its neighboring nodes, and determine the message passing function corresponding to the edge type; The hidden states from each neighboring node at the previous time step are aggregated using the message passing function to obtain the context message of the target dynamic node; The context message, along with the original input features of the target dynamic node at the current time step and its hidden state at the previous time step, are input into the gated loop unit to update the hidden state of the target dynamic node at the current time step. Iterative processing is performed on all time steps of the historical sequence to output the feature sequence of each target dynamic node capturing the dynamic interaction history, which serves as the first feature sequence of the dynamic interaction channel.

[0056] Correspondingly, the encoding process of the instantaneous physical geometry channel includes: At each time step of the historical sequence, search for all neighboring targets within the target radius of the target to be predicted, and transform the coordinates of all neighboring targets to the local coordinate system of the target to be predicted to obtain an aligned list of neighboring coordinates; The list of neighboring coordinates is input into a shared multilayer perceptron to encode and generate a vector representing the current instantaneous geometric layout; Alignment and vector encoding operations are repeatedly performed on each time step of the historical sequence, and a feature sequence composed of instantaneous geometric layout vectors of each time step is output as the second feature sequence of the instantaneous physical geometry channel.

[0057] Specifically, this application employs an innovative dual-channel encoder architecture, aiming to decouple the complex scene understanding task into two parallel and complementary sub-tasks. The underlying logic of this architecture is that the autonomous navigation of an agent requires a balance between two different cognitive modes: one is deliberate reasoning based on history, rules, and semantics, and the other is rapid, instinctive reactions to the current urgent physical situation. The knowledge-driven channel is responsible for the former, while the geometric channel focuses on the latter. The subsequent fusion step acts as a dynamic arbitrator, determining which information is more important in a specific context.

[0058] It should be further explained that the knowledge-driven dynamic interaction channel in this embodiment adopts a heterogeneous graph neural network. This network uses specific message passing functions containing domain knowledge for different types of interaction edges. Specifically, for the interaction between dynamic target nodes, a dynamic gating mechanism controlled by the relative pose features between nodes is adopted to adaptively adjust the information flow intensity from neighboring nodes.

[0059] S2-1: Knowledge-Driven Dynamic Interactive Graph Network Coding This channel aims to simulate dynamic interaction processes within a scenario that evolve over time, conforming to domain knowledge. Its core is a historical sequence... A heterogeneous graph neural network that iteratively evolves at each time step. At any time step... t Any dynamic target node v Hidden state Update as follows: (1) Spatial information aggregation based on specific edge types: First, through a heterogeneous graph convolution operation, information is generated from the neighbors of node v. Aggregate spatial context information to generate a context message. :

[0060] in, and Neighbor nodes j and central node v In the hidden state of the previous moment, It is the edge feature between them, and These are message passing functions specific to edge types. The core of this application lies in designing dedicated functions imbued with domain knowledge for different interactions.

[0061] (a) For target-target interaction: a feature based on relative pose is introduced. A dynamic gating mechanism for control. This mechanism allows the model to adaptively adjust the information flow intensity from a neighboring target based on the relative position and orientation of that neighbor. Its message function is defined as:

[0062]

[0063] in, It is the Sigmoid activation function. This represents element-wise multiplication. This gating mechanism allows the model to intelligently filter and utilize key interaction information; for example, the influence weight of vehicles in front is usually higher than that of vehicles behind, which aligns with driver intuition and is an important inductive bias.

[0064] (b) For target-path and path-path interactions: a strategy combining specialized and general models is adopted. For edges with clear topological relationships (such as the "successor" relationship of a path), a dedicated update function is designed; for other interactions with unclear physical meaning, a general graph attention operator (such as GATConv) is retained as a supplement to balance the injection of prior knowledge into the model and the ability of data-driven learning.

[0065] (2) Time status update of dynamic target nodes: aggregated spatial messages With the original input features of the node itself The computational unit (GRUCell) that is fed into a gated loop unit is used to update its hidden state:

[0066] go through Through iterative steps, this channel outputs a feature sequence for each dynamic target that captures its dynamic interaction history. .

[0067] S2-2: Instantaneous physical geometry encoding: In parallel with S2-1, this channel focuses on capturing the current time step. t A snapshot of the scene's physical layout serves as a supplement to dynamic interaction relationships. Its processing is more direct, aiming to preserve the most original and immediate spatial information, avoiding smoothing or loss in complex graph interactions. At each time step t of the historical sequence, for each target to be predicted, the following operations are performed: First, use Euclidean distance search to find all neighboring targets within a certain radius; then, transform the coordinates of all neighbors to the target's local coordinate system; finally, feed the aligned list of neighbor coordinates into a shared MLP, encoding it into a single vector representing the current instantaneous geometric layout. By repeating this operation at each time step of the historical sequence, a purely geometric feature sequence is obtained. .

[0068] Optionally, the process of obtaining the scene context vector includes: The first feature sequence and the second feature sequence are concatenated using a multilayer perceptron to calculate the fusion gate. Based on the multilayer perceptron corresponding to the first feature sequence combined with the fusion gate, the first weighted feature of the first feature sequence after weighting by the fusion gate is obtained by performing element-wise multiplication through the hyperbolic tangent activation function. Based on the multilayer perceptron corresponding to the second feature sequence combined with the fusion gate, the element-wise multiplication calculation is performed through the hyperbolic tangent activation function to obtain the second weighted feature of the second feature sequence after being weighted by the fusion gate. The first weighted feature and the second weighted feature are weighted and fused to obtain the fused feature sequence; The fused feature sequences are aggregated using a recurrent neural network to generate the unified scene context vector.

[0069] Specifically, semantically rich interactive feature sequences are obtained from a dual-channel encoder. and pure geometric feature sequences Then, this step aims to intelligently fuse them into a unified, comprehensive scene context vector. This is for use by subsequent decoders.

[0070] The dynamic fusion and context aggregation of the above dual-channel features include the following steps: S3-1: Dual-channel feature fusion based on gating mechanism: Gated fusion is performed on features from the two channels. The core of this step is a dynamic fusion gate. It is determined by the characteristics of the two channels and is used to adaptively weigh at each time step whether the model should trust knowledge-driven interaction information more or instantaneous geometric information more.

[0071] (1) Fusion gate calculation: , where [ ; ] represents vector concatenation.

[0072] (2) Feature fusion: Features after fusion Calculated in the following way:

[0073]

[0074]

[0075] This mechanism gives the model the flexibility to dynamically adjust its information processing strategy according to the specific context. For example, in emergency obstacle avoidance scenarios that require rapid response (such as close-range conflicts in a drone swarm), the model may assign higher weight to instantaneous geometric information; while in rule-following cruise scenarios (such as unmanned surface vessels navigating within a designated channel), it may rely more on dynamic knowledge learned from historical interactions.

[0076] S3-2: Aggregation generation of unified scene context vectors: Finally, the gated and fused feature sequences are fed into a final GRU network. This network effectively aggregates the evolutionary history of a dynamic process into a context vector that represents its final state. This vector represents the final, comprehensive, and in-depth understanding of the scene in which the target to be predicted is located.

[0077] Optionally, the method for obtaining the confidence score includes: Learnable modal anchors are combined with the scene context vector and initialized together to determine the initial hidden state; For each time step, a temporary state to be optimized is determined based on the state of the previous time step and the predicted position. The temporary state is used as a query vector based on the cross-attention mechanism. The final feature vector set of all nodes in the dynamic heterogeneous graph at the end of the historical sequence is queried based on the query vector to determine the query context vector. The final hidden state is generated by fusing the query context vector and the temporary state. The final hidden state is input into the prediction head to generate the predicted position point of the current time step. The predicted position points of each time step are combined to obtain the predicted trajectory. Determine the final hidden state of each modal anchor point after the predicted trajectory generation process is completed. Each final hidden state is input into the confidence prediction head to obtain the confidence score corresponding to each predicted trajectory.

[0078] Specifically, this application designs a query-based autoregressive decoder, whose core innovation lies in shifting from a "passive generation" paradigm to an "active query" paradigm. Traditional decoders, after receiving the initial context, have an open-loop generation process, which is prone to becoming out of touch with the dynamically changing scene over time. The decoder in this application, at each step of generating the future trajectory, actively "looks around," revising and guiding the next generation step by querying the global scene context. This process simulates the behavior of an intelligent agent continuously perceiving the environment and self-correcting during planning, thereby ensuring the sustained rationality of long-term predictions and a high degree of consistency with the environment.

[0079] Specifically, the above-mentioned query-based multimodal trajectory decoding includes the following steps: S4-1: Multimodal initialization based on learnable anchors: The model's multimodal prediction capability stems from a set of M learnable modal anchors. During training, each anchor vector It is expected to spontaneously associate with a specific, high-level behavioral pattern (such as "go straight", "turn left", "hover", "change lanes", etc.). For each modality m to be generated, the initial hidden state of its decoding process. Scene context vector output by the encoder and corresponding modal anchor points Common initialization:

[0080] This approach injects a unique, semantically biased initial state into each prediction pattern, forming the fundamental source of trajectory diversity, and is more structured and controllable than random sampling from an uninterpretable latent space.

[0081] S4-2: Autoregressive trajectory generation based on scenario context query: Decoder in a continuous Trajectory points are generated in an autoregressive loop at each time step. At each decoding time step... All M modalities perform the operations in parallel, the core of which lies in the introduction of a scene-trajectory cross attention mechanism.

[0082] (1) Internal state update: A decoder GRU updates its hidden state based on the previous time step. and the predicted position at the previous moment This generates a temporary, unoptimized hidden state.

[0083]

[0084] (2) Scene context query: In order to enable the trajectory generation process to continuously and dynamically adapt to the global scene, a cross-attention mechanism is introduced here, which temporarily hides the state. This will be used as a query (Q) to actively retrieve the final set of feature vectors for all nodes in the graph at the end of the historical sequence, provided by the encoder. This set serves as both the key (K) and value (V) in the cross-attention mechanism:

[0085] This mechanism allows each modality to proactively "look around" the global scene at each step of generation, dynamically focusing its attention on scene elements most relevant to its current predictive intent (e.g., a modality executing a "turn left" intent will focus more on the left-hand path and conflicting targets from the left). This ensures that long-term predictions remain highly consistent with the dynamically evolving environment.

[0086] (3) Fusion and prediction: Context vectors retrieved from the scene Temporary hidden state By fusing residual connections and layer normalization, the final hidden state at the current time step is obtained. The current location point is then fed into a prediction head (such as an MLP). :

[0087]

[0088] S4-3: Confidence assessment based on complete generation history After all M modalities of trajectory decoding are completed, M final hidden states are obtained. Each final hidden state Each trajectory contains the complete generation history of its corresponding trajectory and all its dynamic interaction information with the scene, thus serving as the best basis for evaluating the trajectory's quality. This method feeds these final states into a confidence prediction head (such as an MLP) to generate a confidence score for each trajectory:

[0089] This strategy of evaluating the trajectory after it has been fully generated ensures that the confidence assessment is based on the complete performance of the entire trajectory, rather than just on the initial intent, thus providing greater logical consistency and reliability.

[0090] Reference Figure 4 This application provides a dual-channel knowledge-driven network trajectory prediction system for unmanned systems, comprising: Graph construction module 410 is used to receive scene data of dynamic targets in the unmanned system operation scenario, and construct a dynamic heterogeneous graph including multiple semantic nodes and multiple connection relationships based on the scene data; The dual-channel encoding module 420 is used to encode the dynamic heterogeneous graph using a dual-channel encoder to obtain a first feature sequence representing dynamic interaction and a second feature sequence representing instantaneous physical geometry; The feature fusion module 430 is used to fuse the first feature sequence and the second feature sequence using a dynamic gating mechanism to obtain a unified scene context vector; The decoding module 440 is used to decode the scene context vector using a query-based multimodal decoder, generate at least one set of predicted trajectories, and determine the confidence score corresponding to the predicted trajectories.

[0091] Optionally, the method for constructing the dynamic heterogeneous graph includes: The dynamic target's historical trajectory data and static environment data are acquired as the scene data, and all scene data are transformed into a local coordinate system; the local coordinate system takes the current position of the target to be predicted as the origin and the current orientation as the reference. Based on the transformed scene data, define and initialize the multi-type nodes in the dynamic heterogeneous graph; Based on the spatial and semantic relationships between the various types of nodes, a variety of edges are constructed in the dynamic heterogeneous graph, and the construction of the dynamic heterogeneous graph is completed based on the various types of nodes and edges.

[0092] Optionally, the multi-type nodes of the dynamic heterogeneous graph include at least dynamic target nodes, static path element nodes, and abstract rule nodes that are the embodiment of abstract running rules; The dynamic target nodes include nodes corresponding to drones, unmanned vehicles, or unmanned boats; The static path element nodes include nodes corresponding to lanes, routes, or waterways; The abstract rule nodes include nodes that materialize abstract operational rules for speed limits, no-fly zones, or right-of-way.

[0093] Optionally, the multi-type edges include at least: interaction edges between dynamic target nodes, interaction edges between dynamic target nodes and static path element nodes, topological edges between static path element nodes, and connection edges between abstract rule nodes and the static path element nodes under their jurisdiction.

[0094] Optionally, the dual-channel encoder includes a knowledge-driven dynamic interaction channel and an instantaneous physical geometry channel; Dynamic interactive channels are specifically used for: Determine the edge characteristics and edge types between the target dynamic node and its neighboring nodes, and determine the message passing function corresponding to the edge type; The hidden states from each neighboring node at the previous time step are aggregated using the message passing function to obtain the context message of the target dynamic node; The context message, along with the original input features of the target dynamic node at the current time step and its hidden state at the previous time step, are input into the gated loop unit to update the hidden state of the target dynamic node at the current time step. Iterative processing is performed on all time steps of the historical sequence to output the feature sequence of each target dynamic node capturing the dynamic interaction history, which serves as the first feature sequence of the dynamic interaction channel.

[0095] Optionally, the instantaneous physical geometry channel is specifically used for: At each time step of the historical sequence, search for all neighboring targets within the target radius of the target to be predicted, and transform the coordinates of all neighboring targets to the local coordinate system of the target to be predicted to obtain an aligned list of neighboring coordinates; The list of neighboring coordinates is input into a shared multilayer perceptron to encode and generate a vector representing the current instantaneous geometric layout; Alignment and vector encoding operations are repeatedly performed on each time step of the historical sequence, and a feature sequence composed of instantaneous geometric layout vectors of each time step is output as the second feature sequence of the instantaneous physical geometry channel.

[0096] Optionally, the process of obtaining the scene context vector includes: The first feature sequence and the second feature sequence are concatenated using a multilayer perceptron to calculate the fusion gate. Based on the multilayer perceptron corresponding to the first feature sequence combined with the fusion gate, the first weighted feature of the first feature sequence after weighting by the fusion gate is obtained by performing element-wise multiplication through the hyperbolic tangent activation function. Based on the multilayer perceptron corresponding to the second feature sequence combined with the fusion gate, the element-wise multiplication calculation is performed through the hyperbolic tangent activation function to obtain the second weighted feature of the second feature sequence after being weighted by the fusion gate. The first weighted feature and the second weighted feature are weighted and fused to obtain the fused feature sequence; The fused feature sequences are aggregated using a recurrent neural network to generate the unified scene context vector.

[0097] Optionally, the method for obtaining the confidence score includes: Learnable modal anchors are combined with the scene context vector and initialized together to determine the initial hidden state; For each time step, a temporary state to be optimized is determined based on the state of the previous time step and the predicted position. The temporary state is used as a query vector based on the cross-attention mechanism. The final feature vector set of all nodes in the dynamic heterogeneous graph at the end of the historical sequence is queried based on the query vector to determine the query context vector. The final hidden state is generated by fusing the query context vector and the temporary state. The final hidden state is input into the prediction head to generate the predicted position point of the current time step. The predicted position points of each time step are combined to obtain the predicted trajectory. Determine the final hidden state of each modal anchor point after the predicted trajectory generation process is completed. Each final hidden state is input into the confidence prediction head to obtain the confidence score corresponding to each predicted trajectory.

[0098] Furthermore, to verify the effectiveness of the proposed method and system, prototype verification and a series of experiments were conducted on the publicly authoritative INTERACTION dataset in the field of autonomous driving. This dataset is known for containing a large number of high-intensity interaction scenarios (such as unsignalized intersections and roundabouts), making it an ideal platform for testing the model's interaction understanding capabilities.

[0099] 1. Superior predictive performance: The method proposed in this application achieves superior performance on key prediction metrics. As shown in Table 1, on the test set of the INTERACTION dataset, the method achieved a minADE@6 (minimum average displacement error among 6 predicted trajectories) of 0.1991 meters and a minFDE@6 (minimum final displacement error among 6 predicted trajectories) of 0.6030 meters. This result significantly outperforms several mainstream baseline models, including UQnet, DenseTNT, and MoliNet, strongly demonstrating the advancement and effectiveness of the overall architecture of this application. These data are not merely numbers; they are direct quantitative evidence that the technical solution of this application can bring substantial performance improvements, thus confirming the beneficial effect of "enhanced prediction performance and accuracy."

[0100] Table 1: Performance Comparison on the INTERACTION Dataset

[0101] 2. Quantitative contribution of each core innovation component: To deeply analyze the actual contribution of each innovative component in this application, a series of rigorous ablation experiments were conducted. By systematically removing or replacing key parts of the model, the impact of each design choice on the final performance was quantified. As shown in Table 2, these experimental results provide strong evidence for the non-obviousness of this application. This table is a key basis for demonstrating that the various technical features of this application are not simply a collection of elements, but rather a mutually supportive and indispensable collaborative system.

[0102] Table 2: Ablation experiments of key architectural components

[0103] As can be clearly seen from Table 2: (1) Effectiveness of knowledge-driven design: When the message passing function with inductive bias customized for inter-target interaction and target-path interaction in this application was replaced with a general graph convolution (ablation experiment 1), the minADE / FDE index deteriorated significantly by about 30%. This proves the core hypothesis of this application: injecting physical and behavioral prior knowledge directly into the network structure is far more effective than letting a general model learn these relationships from scratch.

[0104] (2) Necessity of dual-channel design: After removing the parallel instantaneous physical geometry encoding channel (ablation experiment 2), the minADE / FDE index increased by about 18.0%. This indicates that in complex interaction processes, the knowledge-driven GNN channel may smooth out some key geometric details, while the independent geometric channel retains this original information and effectively enhances the model's final understanding ability through the gating fusion mechanism.

[0105] (3) The key role of the query-based decoder: When the query-based decoder was removed and replaced with a more passive decoding method (ablation experiment 3), the minADE / FDE index deteriorated by nearly 55%. This proves that the passive decoder is prone to becoming detached from the scene after receiving the initial context, while the decoder of this application re-examines the global scene at each step of generation, enabling it to make long-term predictions that are highly consistent with the dynamically evolving environment.

[0106] 3. Enhanced interpretability and universality: By explicitly modeling abstract rule nodes and designing physically meaningful specialized functions for different interactions, the decision-making process in this application is more transparent and interpretable than a "black box" model. This is crucial for debugging, validating, and building trust in AI systems in safety-critical applications.

[0107] More importantly, the algorithmic framework presented in this application is highly modular and scalable. By replacing or extending the definitions of nodes and edges in heterogeneous graphs, and adjusting the knowledge-driven message passing function, it can quickly adapt to different types of unmanned systems and application scenarios. For example, for drones, "lane" can be replaced with "flight route," and "traffic light" can be replaced with "airspace control instructions"; for unmanned surface vessels, environmental nodes such as water flow and wind speed can be introduced. This versatility enables the model to exhibit stronger robustness and generalization ability when facing unseen and novel operating scenarios.

[0108] In summary, this application, through a series of closely integrated and mutually supportive innovative designs, including semantically rich general heterogeneous graph construction, knowledge-driven dual-channel encoder, dynamic gating fusion, and active query decoder, has successfully solved many problems in the prior art and achieved substantial progress and significant beneficial effects in the unmanned target trajectory prediction task.

[0109] Reference Figure 5 Based on the methods in the above embodiments, this application provides an electronic device that may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions stored in the memory 530 to execute the methods in the above embodiments.

[0110] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0111] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0112] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0113] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0114] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0115] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0116] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0117] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A dual-channel knowledge-driven network trajectory prediction method for unmanned systems, characterized in that, The method comprises the following steps: receiving scene data of dynamic targets in an unmanned system operation scene, and constructing a dynamic heterogeneous graph comprising a plurality of semantic nodes and a plurality of connection relationships based on the scene data; encoding the dynamic heterogeneous graph by using a double-channel encoder to obtain a first feature sequence representing dynamic interaction and a second feature sequence representing instantaneous physical geometry; fusing the first feature sequence and the second sequence feature by using a dynamic gating mechanism to obtain a unified scene context vector; decoding the scene context vector by using a query-based multi-modal decoder to generate at least one group of predicted trajectories and determine a confidence score corresponding to the predicted trajectories.

2. The dual-channel knowledge-driven network trajectory prediction method for unmanned systems of claim 1, wherein, The construction method of the dynamic heterogeneous graph comprises: obtaining dynamic target historical trajectory data and static environment data as the scene data, and converting all the scene data to a local coordinate system; the local coordinate system takes the current position of the target to be predicted as the origin and the current direction as the reference; defining and initializing a plurality of types of nodes in the dynamic heterogeneous graph based on the converted scene data; constructing a plurality of types of edges in the dynamic heterogeneous graph based on the spatial and semantic relationships between the plurality of types of nodes, and completing the construction of the dynamic heterogeneous graph based on the plurality of types of nodes and the plurality of types of edges.

3. The dual-channel knowledge-driven network trajectory prediction method for unmanned systems of claim 2, wherein, The plurality of types of nodes of the dynamic heterogeneous graph at least comprises dynamic target nodes, static path element nodes, and abstract rule nodes which are abstracted from running rules; the dynamic target nodes comprise nodes corresponding to unmanned aerial vehicles, unmanned vehicles or unmanned ships; the static path element nodes comprise nodes corresponding to lanes, routes or channels; the abstract rule nodes comprise nodes which are abstracted from running rules such as speed limits, no-fly zones or traffic rights; the plurality of types of edges at least comprises interaction edges between dynamic target nodes, interaction edges between dynamic target nodes and static path element nodes, topological edges between static path element nodes, and connection edges between abstract rule nodes and static path element nodes under the jurisdiction of the abstract rule nodes.

4. The dual-channel knowledge-driven network trajectory prediction method for unmanned systems of claim 2, wherein, The double-channel encoder comprises a knowledge-driven dynamic interaction channel and an instantaneous physical geometry channel; the encoding process of the dynamic interaction channel comprises: determining the edge features and edge types between the target dynamic node and the neighbor nodes, and determining the message passing function corresponding to the edge type; aggregating the hidden states from each neighbor node at the previous time step by using the message passing function to obtain the context message of the target dynamic node; inputting the context message, the original input features of the target dynamic node at the current time step, and the hidden state of the target dynamic node at the previous time step into a gated recurrent unit to update and obtain the hidden state of the target dynamic node at the current time step; iteratively processing all time steps of the historical sequence to output the feature sequence of each target dynamic node capturing the dynamic interaction history as the first feature sequence of the dynamic interaction channel.

5. The dual-channel knowledge-driven network trajectory prediction method for unmanned systems of claim 4, wherein, The encoding process of the instantaneous physical geometry channel comprises: at each time step of the historical sequence, searching for all neighboring targets within a target radius range of the target to be predicted, and converting the coordinates of all the neighboring targets to the local coordinate system of the target to be predicted to obtain an aligned neighboring coordinate list; inputting the list of neighboring coordinates into a shared multi-layer perceptron, encoding to generate a vector representing the current instantaneous geometric layout; repeating the alignment and vector encoding operations for each time step of the historical sequence, outputting a feature sequence composed of instantaneous geometric layout vectors at each time step, as the second feature sequence of the instantaneous physical geometry channel.

6. The dual-channel knowledge-driven network trajectory prediction method for unmanned systems of claim 1, wherein, The acquisition process of the scene context vector includes: vector splicing of the first and second feature sequences through a multi-layer perceptron to calculate a fusion gate; based on the fusion gate, combining the multi-layer perceptron corresponding to the first feature sequence, performing element-level multiplication calculation through the hyperbolic tangent activation function to obtain the first weighted feature of the first feature sequence after weighting by the fusion gate; based on the fusion gate, combining the multi-layer perceptron corresponding to the second feature sequence, performing element-level multiplication calculation through the hyperbolic tangent activation function to obtain the second weighted feature of the second feature sequence after weighting by the fusion gate; weighting and fusing the first and second weighted features to obtain a fused feature sequence; aggregating the fused feature sequence through a recurrent neural network to generate the unified scene context vector.

7. The dual-channel knowledge-driven network trajectory prediction method for unmanned systems of claim 4, wherein, The decoding of the scene context vector by the query-based multi-modal decoder to generate at least one set of predicted trajectories and determine the confidence score corresponding to the predicted trajectory includes: combining the learnable modal anchor point with the scene context vector and jointly initializing to determine an initial hidden state; for each time step, determining a temporary state to be optimized based on the state and predicted position at the previous time, and taking the temporary state as a query vector based on the cross-attention mechanism; querying the final feature vector set of all nodes of the dynamic heterogeneous graph at the end of the historical sequence output by the dynamic interaction channel based on the query vector to determine a query context vector; generating a final hidden state based on the fusion of the query context vector and the temporary state, and inputting the final hidden state into the prediction head to generate the predicted position point at the current time step, combining the predicted position points at each time step to obtain the predicted trajectory; determining the final hidden state of each modal anchor point after the end of the predicted trajectory generation process, inputting each final hidden state into the confidence prediction head to obtain the confidence score corresponding to each predicted trajectory.

8. A dual-channel knowledge-driven network trajectory prediction system for unmanned systems, comprising: It includes: a graph construction module for receiving scene data of dynamic targets in an unmanned system running scene, and constructing a dynamic heterogeneous graph including multiple semantic nodes and multiple connection relationships based on the scene data; a dual-channel encoding module for encoding the dynamic heterogeneous graph using a dual-channel encoder to obtain a first feature sequence representing dynamic interaction and a second feature sequence representing instantaneous physical geometry; a feature fusion module for fusing the first and second feature sequences using a dynamic gating mechanism to obtain a unified scene context vector; a decoding module for decoding the scene context vector using a query-based multi-modal decoder to generate at least one set of predicted trajectories and determine the confidence score corresponding to the predicted trajectory.

9. An electronic device, comprising: It includes: at least one memory for storing a computer program; at least one processor configured to perform a program stored by the memory, the processor configured to perform the method of any of claims 1-7 when the program stored by the memory is executed.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. a computer program configured to, when run on a processor, cause the processor to perform the method of any of claims 1-7.

Images