A multi-layer semantic extraction method based on hierarchical semantic causal graph

Abstract

The application discloses a multi-layer semantic extraction method based on hierarchical semantic causal graphs (HSCG), and belongs to the technical field of artificial intelligence and causal reasoning. The method comprises the following steps: constructing a signal / feature layer and an object / entity layer, and extracting higher-level semantics layer by layer through a homogenization iterative semantic layer construction process; each semantic layer generates candidate semantic nodes and associated edges by using an object correlation analysis operator, and infers the causal direction and strength by using a causal discovery algorithm. The object correlation analysis operator comprises geometric correlation, statistical mutual information, semantic similarity, attention correlation, task priori, and newly-added neural similarity operators and space-time similarity operators; the neural similarity operator calculates the nonlinear similarity between nodes through a learnable neural network, and the space-time similarity operator evaluates dynamic space-time correlation based on trajectory prediction gain. High-level causal priori is propagated from top to bottom to fine-tune low-level correlation, and the final hierarchical semantic causal graph is output after the joint optimization of the parameters of each layer. The application realizes deep semantic extraction and causal explainable reasoning, and can be applied to the fields of multi-agent cooperative scene explanation, robot task planning, automatic driving and medical event causal analysis.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and causal reasoning technology, and in particular to a multi-layer semantic extraction method based on hierarchical semantic causal graphs, which is applicable to scenarios requiring deep dynamic semantic modeling, such as scene interpretation for multi-agent collaboration, robot task planning, autonomous driving, and causal analysis of medical events. Background Technology

[0002] With the development of deep learning, scene understanding and semantic extraction techniques have made significant progress. Traditional methods typically construct single-level scene graphs or temporal models, directly using detected objects and their relationships for downstream tasks. However, these methods have the following shortcomings: Single semantic level: Existing scene graphs focus more on static relationships between objects, making it difficult to express higher-level semantics such as events, behaviors, and intentions, resulting in insufficient ability to interpret complex dynamic scenes.

[0003] Confusion between association and causation: Most methods construct relationship edges based solely on statistical correlation or attention weights, failing to distinguish between genuine causal relationships and spurious correlations, resulting in poor robustness and weak interpretability of inference.

[0004] Difficulty in hierarchical expansion: Different model structures need to be designed for different semantic granularities, there is a lack of a unified hierarchical construction mechanism, and it is difficult to flexibly adjust the depth of abstraction according to task requirements.

[0005] Insufficient spatiotemporal dynamic modeling: Existing methods mainly rely on simple distance or IoU to model the spatiotemporal interaction between objects, lacking consideration of the ability to predict future trajectories and making it difficult to capture dynamic causal relationships.

[0006] Feature similarity measurement is singular: Traditional similarity calculations often use fixed functions such as cosine similarity, which cannot adapt to complex nonlinear associations in cross-modal and high-dimensional semantic spaces.

[0007] Therefore, there is an urgent need for a method that can extract semantics hierarchically, distinguish between correlation and causation, support arbitrary depth expansion, and effectively integrate multiple correlation signals. Summary of the Invention

[0008] This invention aims to overcome the shortcomings of the prior art by providing a multi-layer semantic extraction method based on hierarchical semantic causal graphs (HSCG). Furthermore, through a homogeneous and iterative semantic layer design, combined with extended object association analysis operators and causal discovery algorithms, it achieves layer-by-layer semantic abstraction from low-level features to high-level intents, while ensuring the extracted semantics have causal interpretability (see appendix). Figure 1 ).

[0009] To achieve the above objectives, the present invention provides the following technical solution: a multi-layer semantic extraction method based on hierarchical semantic causal graphs, comprising the following steps: Step 1: Acquire multimodal input data, extract low-level signal features, and construct signal / feature layer L0; Step 2: Perform target detection, segmentation or clustering on the signal / feature layer to construct the object / entity layer L1, where nodes are objects or entities and edges are the initial associations between objects; Step 3: Based on the object / entity layer, candidate semantic nodes and candidate association edges are generated through object association analysis operators, and the causal direction and strength are inferred on the candidate associations through the causal discovery algorithm to construct the first semantic layer L2; Step 4: Using the current semantic layer as input, repeat the homogenization construction process in Step 3 to generate higher-level semantic layers L layer by layer. i (i>2), until the preset stopping condition is met (see Appendix) Figure 2 ); Step 5: Propagate the causal priors of the higher semantic layers from top to bottom, and fine-tune and optimize the lower-level related edges; Step 6: Jointly optimize the parameters of all layers and output the final hierarchical semantic causal graph. The object association analysis operator includes at least one or more of the following combinations (see Appendix). Figure 3 ): Neural similarity operator: A neural similarity algorithm based on feature similarity, which calculates the nonlinear similarity between nodes through a learnable neural network; Spatiotemporal similarity operator: A spatiotemporal similarity algorithm based on spatiotemporal trajectory prediction, which evaluates the dynamic spatiotemporal correlation between nodes through trajectory prediction gain; Geometric correlation operator; Statistical mutual information operators; Semantic similarity operators; Attention association operator; Task prior association operator.

[0010] Furthermore, the neural similarity operator (see Appendix) Figure 4 The calculation method for s is as follows: ij neural = σ(F θ ([e xi ;e yj ;e xi ⊙e yj ;||e xi -e y ||])) Among them, e xi and e yj They are nodes x i and candidate node yj The semantic embedding is defined by [⋅;⋅], where [⋅;⋅] represents vector concatenation, ⊙ represents element-wise multiplication, ||⋅|| represents element-wise absolute difference, σ is the sigmoid activation function, and F... θ Learnable parameters θ The neural network.

[0011] Furthermore, the spatiotemporal similarity operator is calculated as follows: s ij st = max(0,(err(p yj ~tT + Δt ) −err(p yj ^tT + Δt )) / (err ( p yj ~tT + Δt ) + ϵ )) Where, p yj ~tT + Δt Based solely on node y j The predicted position of its own trajectory, p yj ^tT + Δt To simultaneously based on node x i and y j The predicted position of the trajectory, err(⋅) is the prediction error function, and ϵ is a small constant to avoid division by zero.

[0012] Furthermore, the causal discovery algorithm includes, but is not limited to, one or more of NOTEARS[1], DAG-GNN[2], PCMCI+[3], DYNOTEARS[4], LiNGAM[5], and ANM[6], and is selected according to the static or dynamic characteristics of the semantic layer.

[0013] Furthermore, the stopping condition in step 4 is at least one of the following: Reaching the preset maximum semantic layer N max ; The current number of semantic layer nodes is less than the preset threshold δ; The semantic information gain is lower than the preset threshold ϵ.

[0014] Furthermore, the loss function jointly optimized in step 6 includes: Related loss: L assoc = ||A − A supervision ||, where A is the incidence matrix, A supervison For monitoring signals; Causal loss includes directed acyclic graph constraints and prediction error; Semantic loss: The loss of downstream tasks (such as interpretation generation and anomaly detection).

[0015] The system implementation of the present invention includes: The feature extraction module is used to acquire multimodal inputs and extract low-level features to construct a signal / feature layer; The object building module is used to detect or segment objects from the signal / feature layer and build the object / entity layer; The semantic layer iterative building module is used to iteratively build higher-level semantic layers from the current layer using object association analysis operators and causal discovery algorithms; The bidirectional propagation module is used to propagate high-level causal priors from top to bottom and optimize low-level associations. The joint optimization module is used to jointly train the parameters of all layers; the output module is used to output the final hierarchical semantic causal graph.

[0016] Compared with existing cause-effect graph construction techniques, this invention has the following advantages: Deep semantic extraction: By designing a homogeneous and iterative semantic layer, semantics of any depth, such as relationships, events, and intentions, can be extracted from the underlying objects step by step, breaking through the limitation of traditional methods that can only express a single level of relationships.

[0017] Decoupling of causality and correlation: Candidate correlations are first generated through object correlation analysis operators, and then causal edges are filtered through causal discovery algorithms, which effectively avoids spurious correlations and improves the robustness and interpretability of reasoning.

[0018] Powerful association modeling capabilities: Through neural similarity operators and spatiotemporal similarity operators, it can capture complex nonlinear semantic associations and dynamic spatiotemporal causal interactions, significantly improving the accuracy of semantic extraction.

[0019] Flexible expansion and adaptive depth: All semantic layers adopt the same construction process, supporting dynamic adjustment of abstraction depth according to task requirements, avoiding over-abstraction or under-abstraction.

[0020] Bidirectional optimization mechanism: High-level causal priors can inversely constrain low-level associations, forming a closed-loop optimization and improving the semantic consistency of the overall graph structure. Attached Figure Description

[0021] Figure 1 A schematic diagram of the overall framework of a hierarchical semantic causal graph; Figure 2 A schematic diagram illustrating the construction process of a homogenized semantic layer; Figure 3 Schematic diagram of the set of object association analysis operators; Figure 4 Flowchart of neural similarity operator computation; Figure 5 Block diagram of a multi-layer semantic extraction system.

[0022] DAGs with NO TEARS: Continuous Optimization for Structure Learning,By Xun Zheng, Bryon Aragam, Pradeep Ravikumar, Eric P. Xing 2018 DAG-GNN: DAG Structure Learning with Graph Neural Networks, By YueYu, Jie Chen, Tian Gao, Mo Yu. 2019 Discovering contemporaneous and lagged causal relations inautocorrelated nonlinear time series datasets, By Jakob Runge 2020 DYNOTEARS: Structure Learning from Time-Series Data, By RoxanaPamfil, Nisara Sriwattanaworachai, Shaan Desai, Philip Pilgerstorfer,Konstantinos Georgatzis, Paul Beaumont, Bryon Aragam. 2020 A Linear Non-Gaussian Acyclic Model for Causal Discovery, By ShoheiShimizu, Patrik O. Hoyer, Aapo Hyvärinen, Antti Kerminen. 2006 Nonlinear causal discovery with additive noise models, By Patrik O.Hoyer, Dominik Janzing, Joris M. Mooij, Jonas Peters, Bernhard Schölkopf.2009 Attention is All You Need. By Ashish Vaswani, Noam Shazeer, NikiParmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, IlliaPolosukhin. 2017 An algorithm for fast recovery of sparse causal graphs, By PeterSpirtes, Clark Glymour. 1991 Estimation of a Structural Vector Autoregression Model Using Non-Gaussianity, By Aapo Hyvärinen, Kun Zhang, Shohei Shimizu, Patrik O. Hoyer.2010 DoWhy: An End-to-End Library for Causal Inference, By Amit Sharma,Emre Kiciman. 2020 Detailed Implementation

[0023] Based on the aforementioned design method of the multi-layer semantic extraction system based on hierarchical semantic causal graph, the specific implementation method is explained here.

[0024] 1. System Overall Architecture As attached Figure 5 As shown, the overall system architecture includes a feature extraction module, an object construction module, a semantic layer iterative construction module, a bidirectional propagation module, a joint optimization module, and an output module, connected sequentially. The semantic layer iterative construction module can be repeatedly invoked to construct any number of semantic layers (from the 1st semantic layer to the Nth semantic layer, where N≥1). The system input is multimodal raw data, and the output is a hierarchical semantic causal graph stack. This stack contains a signal / feature layer, an object / entity layer, and N semantic layers. Each layer contains a set of nodes, a set of edges, and node attributes.

[0025] 2. Layer Definition The system internally maintains a hierarchical graph structure, which includes the following levels: Signal / Feature Layer L0: This is the lowest layer of the system, generated by the feature extraction module. Nodes are local feature vectors extracted from the original input; each node corresponds to a spatiotemporal location or frequency domain component in the input data. Edges represent local adjacency relationships or statistical correlations between nodes and are not used for causal inference. This layer does not participate in semantic interpretation; it only provides underlying evidence.

[0026] Object / Entity Layer L1: This layer is generated by the object building module. Nodes are entity objects detected, segmented, or clustered from the signal / feature layer. Each node has attributes such as semantic embedding, appearance features, and spatiotemporal state. Edges represent the initial spatial or functional relationships between objects, such as proximity or co-occurrence. This layer is the smallest unit for subsequent semantic analysis.

[0027] Semantic layers L2 to LN are homogeneous iterable layers, each generated from nodes in the previous layer through iterative semantic layer building blocks. Nodes in the i-th semantic layer (i≥2) are semantic units extracted from nodes in the (i-1)-th layer, with semantic granularity progressively increasing from concrete to abstract. For example, lower semantic layers can express relationships or simple interactions between objects, while higher semantic layers can express complex events, behavioral patterns, intentions, or scene types. Each layer contains a directed acyclic graph or a temporally directed graph, with edges representing causal relationships between semantic nodes within the layer. Layers are connected by associative and causal edges, forming a cross-layer semantic network.

[0028] All semantic layers employ the exact same construction process, differing only in their input node sets and node features; hence, they are called homogenized semantic layers (see appendix). Figure 2 Users can set the maximum number of layers N according to the complexity of the task, or the system can automatically determine the number of layers based on information gain.

[0029] 3. Detailed Description of Core Modules

[0030] This module receives multimodal input data, such as image sequences, point clouds, audio, structured sensor data, etc., and extracts multi-scale feature maps or feature sequences through pre-trained convolutional neural networks, Transformer[7] or time series models. Each feature position or time step is a feature node, and the node features are high-dimensional vectors. The module outputs the set of nodes and adjacency relationships of the signal / feature layer L0.

[0031] Object building modules This module takes L0 as input and uses object detection, semantic segmentation, instance segmentation, or clustering algorithms to generate object candidate regions or entity instances from the feature map. Each object node is assigned the following attributes: Semantic embedding vectors: obtained from intermediate features of the detector or segmentation model; Appearance features: color, texture, shape, etc.; Spacetime state: position, velocity, acceleration, time interval of occurrence, etc.; Confidence score.

[0032] Initial edges between objects can be established based on spatial proximity, appearance similarity, or predefined rules, and are used for initializing subsequent association calculations.

[0033] Semantic layer iterative building module This module is the core of the system, responsible for building the next semantic layer from the current layer. It contains four sub-units: a candidate node generation unit, an association calculation unit, a causal discovery unit, and a semantic annotation unit. This module can be called repeatedly, each time generating a new semantic layer, until a stopping condition is met.

[0034] Candidate Node Generation Unit The input is a set of low-level nodes (for the first semantic layer, the input is the object layer L1; for the i-th semantic layer, the input is the (i-1)-th semantic layer). This unit generates candidate semantic nodes in the following way: Clustering of low-level nodes, such as spectral clustering or graph clustering based on semantic embedding; Mining frequent subgraph patterns or temporal patterns in low-level nodes; Matching is performed using semantic templates from a prior knowledge base; Low-level nodes that are temporally continuous and spatially close are combined into candidate composite nodes.

[0035] Each candidate node is initialized as an empty semantic unit of undetermined type, and its features are obtained by aggregating the features of low-level nodes.

[0036] Associated computing unit This unit calculates the association strength between low-level nodes and candidate nodes, generating an association matrix A. Association calculation is based on multiple operators, and the scores of all operators are fused using learnable weights. The operator set includes: Geometric association operator: Calculates geometric similarity based on the distance and overlap between low-level nodes and candidate nodes in the spatial or state space; Statistical mutual information operator: Calculates normalized mutual information based on the time series corresponding to low-level nodes and candidate nodes; Semantic similarity operator: cosine similarity based on pre-trained semantic embeddings; Attention association operator: Utilizes the cross-attention mechanism to calculate the attention weights of low-level node features on candidate node features; Prior association operator for tasks: Assign prior weights to known associations based on the domain knowledge graph or rule base; Neural similarity operator: This operator uses a learnable neural network as input, taking the concatenated features of low-level nodes and candidate nodes, element-wise multiplication, and absolute difference as input, and outputting a non-linear similarity score. This operator can capture complex relationships across modalities and high-dimensional semantic spaces. Spatiotemporal similarity operator: Using a trajectory prediction model, the prediction error is calculated separately based on the candidate node's own trajectory and the prediction error based on both the low-level node's and the candidate node's trajectory. The reduction in the prediction error is used as the correlation strength.

[0037] All operator outputs are normalized to the [0,1] interval. The fusion method is: A ij =σ(∑ K k=1 w k s k ij +b) Among them, w k The weights are learnable, and σ is the sigmoid function. After fusion, candidate edges are selected using a threshold or top-k method to obtain a candidate edge set E. cand This operator reflects the causal influence of lower-level nodes on the future dynamics of candidate nodes.

[0038] (3) Causal discovery unit This unit runs a causal discovery algorithm on the candidate edge set to remove spurious correlations and determine causal directions. Depending on the static or dynamic characteristics of the semantic layer, one or a combination of the following algorithms is selected: For the static semantic layer: use NOTEARS[1], DAG-GNN[2] or PC algorithm[8] to learn a directed acyclic graph on the candidate nodes; For dynamic semantic layers: use PCMCI+[3], DYNOTEARS[4] or VAR-LiNGAM[9] to handle time-lag causality and instantaneous causality; for high-level layers that require counterfactual inference: use ANM[6] or DoWhy

[10] frameworks.

[0039] Causal discovery output layer causal graph G i =(V i E i ), where V i E is a filtered set of semantic nodes. i These are directed edges with causal strength and confidence level. Simultaneously, cross-layer related edges are also preserved, forming inter-layer connections.

[0040] Semantic annotation unit This unit performs semantic type annotation on each node in the causal graph. Annotation methods include: Voting is based on the type of low-level node in the aggregation; Use a pre-trained classifier to classify the semantic embeddings of nodes; Rule matching is performed by combining the task's prior knowledge base.

[0041] The annotation results are stored in the node attributes, and the types include, but are not limited to, abstract level tags such as "relationship", "event", "behavior", "intent", and "scenario".

[0042] This module implements top-down prior propagation. After the causal graph of the high-level semantic layer is generated, the causal edges and node types are used as priors to adjust the low-level association matrix in reverse. Specifically, for a low-level node x... i With candidate node y j The correlation score, if there is a correlation from y in the high-level causal graph j A causal path to a certain higher-level node, and that path supports x. i Activation enhances A. ij Otherwise, suppress it. The adjusted formula is: A posterior ij = A ij ⋅ exp(γ ⋅ consistency(x i , y j G high )) Here, γ is the adjustment coefficient, and consistency(⋅) is the consistency metric function. This module, through iterative execution, can gradually eliminate low-level spurious associations and improve the semantic consistency of the overall graph structure.

[0043] This module performs end-to-end or staged joint optimization of the learnable parameters of all modules. The loss function consists of three parts: Association loss: measures the difference between the association matrix and the supervision signal or pseudo-label, such as mean squared error or cross-entropy; Causal loss includes directed acyclic graph constraints (such as h(W)=0 in NOTEARS) and prediction errors based on causal graphs (such as reconstruction errors or future state prediction errors). Semantic loss: Loss in downstream tasks, such as semantic graph classification, interpretation generation, anomaly detection, etc.

[0044] The optimization process can employ an alternating optimization strategy: first, fix the parameters of the correlation computation unit and optimize the causal discovery unit; then, fix the causal graph and optimize the correlation operator and feature extraction module. End-to-end joint training can also be performed.

[0045] The output module encapsulates the final hierarchical semantic causal graph stack, providing standard interfaces for downstream applications to call. The output includes: The collection and attributes of nodes at each level; Causal edges and their strength within each layer; Interlayer association edges and weights; Semantic type annotation results; Optional graph visualization or natural language interpretation generation.

[0046] The raw multimodal data is input into the feature extraction module to generate an L0 feature node set; The object construction module detects or segments objects from L0 and generates L1 object nodes and initial edges; The semantic layer iterative construction module takes L1 as input, executes the first call, and generates the first semantic layer L2; Using L2 as input, the semantic layer iterative building module is repeatedly called to generate L3, and so on, until the preset maximum number of layers N is reached or the stopping condition is met. The bidirectional propagation module starts from the highest semantic layer and propagates causal priors layer by layer downwards, updating the weights of the associated edges at each layer. The joint optimization module optimizes all parameters; The output module outputs the final hierarchical semantic causal graph stack.

[0047] 5. Scalability The core advantage of the system architecture of this invention lies in the homogeneity and iterability of the semantic layers. Since the construction process of each semantic layer is completely consistent, differing only in input, semantic abstraction of arbitrary depth can be achieved through simple iterative calls. Furthermore, both the object association analysis operator and the causal discovery algorithm adopt modular interfaces, allowing users to replace or add new operators and algorithms according to domain characteristics without affecting the overall architecture. Stopping conditions are flexibly configurable, for example: Reaching the preset maximum number of layers N max ; The number of nodes in the current layer is less than the threshold δ; The semantic information gain is below the threshold ϵ, and the information gain can be calculated based on graph structure entropy or node semantic diversity. Through the above design, this system can adapt to scene understanding tasks of varying complexity, from simple object relationship extraction to deep intent reasoning, all of which can be achieved through the same architecture.

Claims

1. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, Includes the following steps: Acquire multimodal input data, extract underlying signal features, and construct a signal / feature layer; The signal / feature layer is subjected to target detection, segmentation, or clustering to construct the object / entity layer; Based on the object / entity layer, candidate semantic nodes and candidate association edges are generated through object association analysis operators, and causal direction and causal strength are inferred on the candidate association edges through a causal discovery algorithm to construct the first semantic layer. Using the currently constructed semantic layer as input, the homogeneous construction process of generating candidate semantic nodes and candidate association edges and inferring causal direction and causal strength is repeated to generate higher-level semantic layers layer by layer until a preset stopping condition is met. The causal prior of the highest-level semantic layer is propagated from top to bottom to fine-tune and optimize the lower-level association edges. The parameters of all layers are jointly optimized to output the final hierarchical semantic causal graph.

2. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, Object association analysis operators include neural similarity operators, which are neural similarity algorithms based on feature similarity. They use learnable neural networks to calculate the nonlinear similarity between nodes.

3. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, The object association analysis operator includes the spatiotemporal similarity operator, which is based on the spatiotemporal similarity algorithm of spatiotemporal trajectory prediction. The dynamic spatiotemporal association between nodes is evaluated through trajectory prediction gain.

4. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, The neural similarity operator is computed as follows: s ij neural = σ(F θ ([e xi ;e yj ;e xi ⊙e yj ;||e xi -e y ||])) 。 Among them, e xi and e yj They are nodes x i and candidate node y j The semantic embedding is defined by [⋅;⋅], where [⋅;⋅] represents vector concatenation, ⊙ represents element-wise multiplication, ||⋅|| represents element-wise absolute difference, σ is the sigmoid activation function, and F... θ Learnable parameters θ The neural network.

5. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, The spatiotemporal similarity operator is calculated as follows: s ij st = max(0,(err(p yj ~tT + Δt ) − err(p yj ^tT + Δt )) / (err ( p yj ~tT + Δt ) + ϵ )). Where, p yj ~tT + Δt Based solely on node y j The predicted position of its own trajectory, p yj ^tT + Δt To simultaneously based on node x i and y j The predicted position of the trajectory, err(⋅) is the prediction error function, and ϵ is a small constant to avoid division by zero.

6. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, For the static semantic layer, a causal discovery algorithm based on directed acyclic graph constraints is selected; for the dynamic semantic layer, a causal discovery algorithm based on temporal conditional independence or dynamic Bayesian networks is selected. A multi-layer semantic extraction method based on hierarchical semantic causal graphs is characterized by jointly optimizing the parameters of all layers by minimizing a weighted sum of the following loss functions, including: association loss: L assoc = ||A − A supervision ||, where A is the incidence matrix, A supervison The supervisory signal is used for causal loss, which includes the directed acyclic graph constraint and prediction error. The semantic loss is the loss of downstream tasks (such as interpretation generation and anomaly detection).

7. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, The system implementation includes:

1. A feature extraction module, used to acquire multimodal input and extract low-level features to construct a signal / feature layer; 2. An object construction module, used to detect, segment, or cluster objects from the signal / feature layer to construct an object / entity layer; 3. A semantic layer iterative construction module, used to iteratively construct higher-level semantic layers from the current layer using object association analysis operators and causal discovery algorithms; the semantic layer iterative construction module includes: a candidate node generation unit, used to perform clustering or subgraph pattern mining on low-level nodes to generate candidate semantic nodes; and an association calculation unit, used to calculate low-level... The system comprises: 1) an association score between a node and candidate semantic nodes, including at least one of neural similarity operators and spatiotemporal similarity operators; 2) a causal discovery unit, used to run a causal discovery algorithm on candidate association edges to obtain an intra-layer causal graph; 3) a semantic annotation unit, used to annotate the semantic layer nodes with types including relations, events, or intentions; 4) a bidirectional propagation module, used to propagate the causal priors of higher-level semantic layers from top to bottom, and to fine-tune and optimize the lower-level association edges; 5) a joint optimization module, used to jointly train the parameters of all layers; and 6) an output module, used to output the final hierarchical semantic causal graph.

8. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, The association calculation unit specifically includes: a neural similarity calculation subunit, used to calculate the nonlinear similarity between low-level nodes and candidate semantic nodes through a learnable neural network; and a spatiotemporal similarity calculation subunit, used to calculate the dynamic spatiotemporal association between low-level nodes and candidate semantic nodes based on trajectory prediction gain.

9. A multi-layer semantic extraction method based on hierarchical semantic causal graphs, characterized in that, It supports N semantic layers, where the maximum value of N (the preset stopping condition) includes:

1. Reaching the preset maximum number of semantic layers N. max 2. The number of current semantic layer nodes is less than a preset threshold δ; the semantic information gain is less than a preset threshold ϵ, and the semantic information gain is calculated based on graph structure entropy or semantic coverage.

Images