Causal dependency tree construction method and device

Abstract

The invention relates to a causal dependency tree construction method and device, and belongs to the technical field of natural language process.The method comprises the steps that semantic features of a to-be-recognized text are obtained; constructing a syntactic dependency tree of the to-be-recognized text, and obtaining a dependency relationship between segmented words in the to-be-recognized text according to the syntactic dependency tree; taking segmented words in the to-be-recognized text as nodes, adding directed connection edges among the segmented word nodes with the dependency relationship, and connecting the semantic features with the segmented word nodes to generate a syntactic dependency graph; extracting structural features of the to-be-recognized text from the syntactic dependency graph; and identifying the semantic features and the structural features to obtain causal relationship information of the to-be-identified text, and generating a causal dependency tree according to the causal relationship information. By means of the method, the complex structure relation in the to-be-recognized text can be captured, so that the more accurate causal dependence tree is generated, and the method is more universal.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method and apparatus for constructing a causal dependency tree. Background Technology

[0002] Causality is a crucial foundation for human cognition and understanding of the world. It reveals the causal logic between events and has wide applications in science, engineering, medicine, and social sciences. In Natural Language Processing (NLP), extracting causal relationships from text is one of the core tasks of semantic understanding and reasoning, and it is of great significance for applications such as question-answering systems, event prediction, and knowledge graph construction.

[0003] In recent years, with the rapid development of deep learning technology, causal relationship extraction methods based on neural networks and structured representations have gradually become a research hotspot. Among them, causal dependency trees, as a structured representation method, can clearly depict the dependencies between causal entities, result entities, and their related event attributes, providing important support for causal relationship analysis. However, existing causal dependency tree construction methods have the following problems: (1) Identification of implicit causal relationships: Many causal relationships are not expressed through explicit causal words (such as "because" and "therefore"), but are implied in the context, making them difficult to identify accurately. (2) Handling of multi-cause-multi-effect relationships: In real-world scenarios, causal relationships often involve multiple causes and effects, and how to accurately model such complex causal networks remains a challenge. (3) Existing models perform well in specific fields (such as healthcare and finance), but their generalization ability in cross-domain scenarios is limited. Summary of the Invention

[0004] In view of this, it is necessary to provide a method and apparatus for constructing a causal dependency tree to solve the problems of inaccurate identification of causal relationships and limited generalization ability in existing models.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a method for constructing a causal dependency tree, comprising: Obtain the semantic features of the text to be identified; Construct a syntactic dependency tree of the text to be identified, and obtain the dependency relationships between word segments in the text to be identified based on the syntactic dependency tree; The word segments in the text to be identified are used as nodes. Directed connection edges are added between word segments that have dependencies. The semantic features are connected to each word segment node to generate a syntactic dependency graph. Extract the structural features of the text to be identified from the syntactic dependency graph; The semantic features and structural features are identified to obtain the causal relationship information of the text to be identified, and a causal dependency tree is generated based on the causal relationship information.

[0006] In one possible implementation, obtaining the semantic features of the text to be identified includes: The text to be recognized is input into the BERT text encoder to obtain the semantic features of the text to be recognized.

[0007] In one possible implementation, the method further includes: Based on the syntactic dependency tree, the syntactic relation types between word segments that have dependencies are obtained; The syntax relation type is labeled on the directed connection edge.

[0008] In one possible implementation, the method further includes: When two word segments are consecutive in the text to be identified and are both nouns, a directed connection edge is added between the word segment nodes corresponding to the two word segments.

[0009] In one possible implementation, extracting the structural features of the text to be identified from the syntactic dependency graph includes: The structural features of the text to be identified are extracted from the syntactic dependency graph using a relational graph convolutional network.

[0010] In one possible implementation, the identification of the semantic features and the structural features includes: The semantic features and the structural features are fused using a gate-based network to obtain fused features. The fused features are identified using a trained causal relationship prediction model.

[0011] In one possible implementation, the causal relationship prediction model is trained using sample text containing causal relationships, which is generated by a diffusion model.

[0012] In one possible implementation, the diffusion model generates the sample text using a reverse denoising technique guided by a causal classifier.

[0013] In one possible implementation, the causal relationship prediction model is trained as follows: The entity recognition loss is determined based on the entities and entity attribute labels in the sample text output by the causal relationship prediction model. Based on the dependencies between entities and the dependencies between entities and entity attributes in the sample text output by the causal relationship prediction model, the structural recognition loss is determined. The causal relationship prediction model is updated in reverse based on the entity recognition loss and the structure recognition loss until the entity recognition loss and the structure recognition loss meet the preset requirements, at which point the training of the causal relationship prediction model is considered complete.

[0014] In a second aspect, the present invention also provides a causal dependency tree construction apparatus, comprising: The semantic feature extraction module is used to obtain the semantic features of the text to be recognized; The syntactic dependency tree construction module is used to construct the syntactic dependency tree of the text to be identified, and to obtain the dependency relationship between word segments in the text to be identified based on the syntactic dependency tree; The syntactic dependency graph construction module is used to take the word segments in the text to be identified as nodes, add directed connection edges between word segment nodes with dependency relationships, and connect the semantic features with each word segment node to generate a syntactic dependency graph. The structural feature extraction module is used to extract the structural features of the text to be identified from the syntactic dependency graph. The causal dependency tree generation module is used to identify the semantic features and the structural features, obtain the causal relationship information of the text to be identified, and generate a causal dependency tree based on the causal relationship information.

[0015] The beneficial effects of this invention are: This invention extracts preliminary structural features from the text to be identified by constructing a syntactic dependency tree. These preliminary structural features include the dependencies between word segments in the text. Then, combining the semantic features and preliminary structural features, a syntactic dependency graph is generated. Each word segment node in the syntactic dependency graph contains rich semantic information. Therefore, further structural feature extraction from the syntactic dependency graph can capture complex structural relationships in the text, such as multi-cause-multi-effect relationships and implicit causal relationships. Finally, by identifying the semantic features and the further extracted structural features, causal relationship information of the text can be obtained, such as entities, entity attributes, and dependencies between entities. Based on this causal relationship information, a causal dependency tree can be generated. In summary, the method of this invention can generate a more accurate causal dependency tree, and by deeply mining the semantic information of the text to be identified to achieve causal relationship identification, this invention can adapt to changes in different semantic scenarios and has stronger scene adaptability and generalization ability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of a causal dependency tree is provided for this invention; Figure 2 A schematic diagram of an implicit causal dependency tree containing causal relation words provided by the present invention; Figure 3 A schematic diagram of an implicit causal dependency tree without causal relation words provided by the present invention; Figure 4 A schematic diagram of another implicit causal dependency tree without causal relation words provided by the present invention; Figure 5 A flowchart illustrating an embodiment of the causal dependency tree construction method provided by the present invention; Figure 6 A flowchart illustrating another embodiment of the causal dependency tree construction method provided by the present invention; Figure 7 This is a schematic diagram of an embodiment of the causal dependency tree construction device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] In the description of the embodiments of this invention, unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," etc., used in the embodiments of this invention are used to distinguish similar objects, and are not used to describe a specific order or sequence, nor to indicate or imply their relative importance or implicitly specify the number of indicated technical features. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, and the number of objects is not limited; for example, a first object can be one or more.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] Before demonstrating the embodiments, the causal dependency tree will be explained in detail.

[0022] A causal dependency tree is a directed tree structure used to represent causal relationships in text. Its core objective is to clearly depict the causal dependencies between cause entities, result entities, and their related attributes through hierarchical node relationships. The root node of a causal dependency tree is the causal relation word (ROOT) in the sentence, used to identify the core semantics of the causal relationship. In the causal dependency tree structure, the cause entity (CAUSE) and the result entity (RESULT) are child nodes of the root node, directly dependent on the causal relation word. Furthermore, to more completely express the causal relationship, the cause and result entities usually need to be further associated with their attributes, namely, cause attributes (CAUSE_P) and result attributes (RESULT_P). Cause attributes describe the actions, states, or other characteristics of the cause entity, while result attributes describe the actions, states, or other characteristics of the result entity. These attribute nodes depend on their corresponding cause entity nodes or result entity nodes, thus forming a complete causal dependency tree.

[0023] Specifically, the structure of a causal dependency tree can be formally represented as: ROOT: A causal term, representing the root node of a tree.

[0024] CAUSE: The cause entity, which depends on the ROOT node.

[0025] CAUSE_P: The cause attribute, which depends on the CAUSE node, and is used to describe the specific characteristics of the cause entity.

[0026] RESULT: The resulting entity, which depends on the ROOT node.

[0027] RESULT_P: Result attribute, dependent on the RESULT node, used to describe the specific characteristics of the result entity.

[0028] Through this structured representation method, causal dependency trees can clearly capture the hierarchical information of causal relationships in text, providing strong support for causal relationship extraction and reasoning tasks.

[0029] To address different scenarios, this invention defines two structurally different causal dependency trees: an explicit causal dependency tree and an implicit causal dependency tree. The difference between the explicit and implicit causal dependency trees lies in whether they contain explicit causal relation terms.

[0030] I. Explicit Causal Dependency Tree.

[0031] Reference Figure 1 This diagram illustrates a causal dependency tree provided by the present invention. The text corresponding to the causal dependency tree is: "Smelters are making substantial profits, leading to a further increase in domestic refined zinc supply." The causal relation word is: "leading to." This causal relation word directly reflects the strong causal logical relationship between the preceding and following text, and its causal logical relationship does not require inference from the preceding and following text.

[0032] II. Implicit Causal Dependency Tree.

[0033] (1) Implicit causal dependency tree containing causal relation words.

[0034] Reference Figure 2 This diagram illustrates an implicit causal dependency tree containing causal relation words provided by the present invention. The text corresponding to the causal dependency tree is: "With the year-end approaching and holidays imminent, orders and purchases have declined, and cotton prices have fallen significantly." The causal relation word is: "With the year-end approaching." This causal relation word implies causal logic and only serves as a hint; the causal logical relationship in the text still needs to be inferred in conjunction with the surrounding context.

[0035] (2) Implicit causal dependency tree without causal relation words.

[0036] Texts lacking causal conjunctions can be further divided into two types: those with demonstrative pronouns and those without. (See reference...) Figure 3 This diagram illustrates an implicit causal dependency tree without causal relation words provided by the present invention. The corresponding text for the causal dependency tree is: He became famous for catching pickpockets, and some pickpockets were so afraid of him that they were practically terrified of him. (See reference...) Figure 4 This diagram illustrates another implicit causal dependency tree provided by the present invention, which does not contain causal relation words. The text corresponding to the causal dependency tree is: I overslept today. — Yes, from staying up late.

[0037] To accurately construct the various forms of causal dependency trees mentioned above, the present invention provides a method and apparatus for constructing causal dependency trees, which will be described below.

[0038] Reference Figure 5 The diagram illustrates a flowchart of an embodiment of the causal dependency tree construction method provided by the present invention, the method comprising: S501, Obtain the semantic features of the text to be recognized.

[0039] The semantic features of the text to be identified can be extracted using methods such as Transformer encoders, generative models, and sentence embedding.

[0040] In one example, a bidirectional pre-trained language model based on Transformers (BERT) can be used for extraction.

[0041] The BERT model is able to capture contextual information in the text to be recognized and generate high-quality vector representations for each word segment. For example, given an input sentence... ,in Indicates the first in the sentence i For each word segmentation, the BERT model first converts it into a corresponding word embedding, position embedding, and segment embedding. Then, these embedding vectors are summed to form the first word segmentation. i The input representation of each word segmentation:

[0042] in, It is the first i The input representation of each word segmentation, This indicates the sentence to which the word belongs (usually 0 in single-sentence tasks). For example, for the sentence "The rise in pork prices has attracted great attention from the government," the BERT model generates an input representation for each word segment (such as "pork," "price," "rise," etc.).

[0043] Then, after passing through multiple Transformer encoders, the BERT model outputs a context-dependent vector representation for each word segment. For the input sentence... The output of BERT can be represented as:

[0044] in, It is the first i The context-related vector representation of each word segment; cls and sep These are two extra characters added to the sentence by the BERT model. cls The embedding contains the semantic information of the entire sentence.

[0045] S502, construct the syntactic dependency tree of the text to be recognized, and obtain the dependency relationship between word segments in the text to be recognized based on the syntactic dependency tree.

[0046] A syntactic dependency tree is a tree structure used to represent the syntactic relationships between word segments in a sentence. Each node in the syntactic dependency tree represents a word segment, and the edges between nodes represent the syntactic dependency relationships between word segments. The syntactic dependency tree of the text to be recognized can be constructed as follows: (1)Word segmentation and词性标注.

[0047] First, segment the text to be recognized and label the词性 (Part-of-Speech, POS) for each word segment. For example, the sentence "The rising price of pork has drawn the government's high attention" will be segmented into "pork", "price", "rising", etc., and labeled with词性 such as noun, verb, etc.

[0048] (2)Dependency relationship parsing.

[0049] Parse the syntactic dependency relationships between word segments through a syntactic analyzer, such as the Stanford Parser software or the Spacy software, etc. For example, "rising" may depend on "price", indicating "price rising".

[0050] After the syntactic dependency tree is constructed, the syntactic structure information of the text to be recognized can be output. The syntactic structure information can be a series of triples , where and respectively represent two word segments with a dependency relationship, represents the type of syntactic relationship between them, such as subject-predicate relationship, verb-object relationship, etc.

[0051] S503, use the word segments in the text to be recognized as nodes, add directed connection edges between the word segment nodes with a dependency relationship, and connect the semantic features to each word segment node to generate a syntactic dependency graph.

[0052] In this embodiment, based on the syntactic dependency tree, a syntactic dependency graph is further constructed to more flexibly represent the syntactic structure in the sentence. The syntactic dependency graph is a directed graph , where the node set V represents the word segments in the sentence, and each word segment node corresponds to a word segment ; the edge set E represents the syntactic dependency relationships between word segments, and each edge goes from to . [[ID=并将语义特征与各个分词节点连接以生成句法依赖图

[0053] Then cls can be embedded and connected to each word segment node to obtain a syntactic dependency graph.

[0054] S504, extract the structural features of the text to be recognized from the syntactic dependency graph.

[0055] Structural features of the text to be identified can be extracted from the syntactic dependency graph using graph neural networks.

[0056] S505 identifies semantic and structural features to obtain causal relationship information of the text to be identified, and generates a causal dependency tree based on the causal relationship information.

[0057] Semantic and structural features can be fused to obtain fused features. Then, a trained causal relationship prediction model is used to identify the causal relationship information of the text to be identified. The causal relationship information can include entities, entity attributes, dependencies between entities, and dependencies between entities and entity attributes. Finally, a causal dependency tree is generated based on the causal relationship information.

[0058] The causal dependency tree construction method provided in this embodiment can be applied to a causal dependency tree construction system, which can be a software system running on a terminal device. The terminal device can be a tablet computer, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), mobile phone, etc. This embodiment does not impose any restrictions on the specific type of terminal device.

[0059] In summary, this invention extracts preliminary structural features from the text to be identified by constructing a syntactic dependency tree. These preliminary structural features include the dependencies between word segments in the text. Then, combining the semantic features and preliminary structural features, a syntactic dependency graph is generated. Each word segment node in the syntactic dependency graph contains rich semantic information. Therefore, further structural feature extraction from the syntactic dependency graph can capture complex structural relationships in the text, such as multi-cause-multi-effect relationships and implicit causal relationships. Finally, by identifying the semantic features and the further extracted structural features, causal relationship information of the text can be obtained, such as entities, entity attributes, and dependencies between entities. Based on this causal relationship information, a causal dependency tree can be generated. In conclusion, the method of this invention can generate more accurate causal dependency trees and is more universal.

[0060] In some embodiments of the present invention, the method further includes: obtaining the syntactic relation types between word segments that have dependencies based on the syntactic dependency tree; and labeling the syntactic relation types on directed connection edges. Specifically, this can be done on each edge. The above indicates the dependency type. r For example, if "rise" depends on "price", then add an edge to the graph pointing from "rise" to "price" and label it "subject-verb relationship".

[0061] In some embodiments of the present invention, the method further includes: when two segmented words are in consecutive positions in the text to be identified and are both nouns, adding a directed connection edge between the segmented nodes corresponding to the two segmented words. For example, "pork" and "price" are in consecutive positions in the sentence and are both nouns. Therefore, an edge is added between "pork" and "price" to facilitate the subsequent combination of "pork" and "price" as candidate entities.

[0062] In some embodiments of the present invention, S504 includes: extracting structural features of the text to be identified from the syntactic dependency graph using a Relational Graph Convolutional Network (RGCN).

[0063] RGCN is a neural network capable of processing graph-structured data, particularly well-suited for capturing complex relationships between nodes in a graph. By applying RGCN to syntactic dependency graphs, structural information in sentences can be effectively extracted, supporting subsequent causal relationship extraction tasks.

[0064] RGCN is an extension of Graph Convolutional Network (GCN) specifically designed to process graph data with diverse edge types. Unlike traditional GCNs, RGCN considers edge type information during convolution operations, thus better capturing the diverse relationships between nodes in the graph. The core idea of ​​RGCN is to aggregate information from neighboring nodes to the target node through a message-passing mechanism, and to weight the information according to the edge type.

[0065] The convolution operation of RGCN can be represented as:

[0066] in: Indicates the first Layer nodes Feature representation; Represents the set of edge types; Represents nodes Through type The set of neighboring nodes connected by the edge; and They are the first Layer for edge type The weight matrix of self-loops; It is a normalization constant, usually taking the value of 1. . It is a non-linear activation function (such as ReLU).

[0067] The process of sentence structure feature extraction based on RGCN is as follows: (1) Multi-relation modeling: RGCN designs independent transformation matrices for different types of relations (such as syntactic relations in dependency trees), enabling the model to explicitly distinguish the impact of different relations on node features. This feature significantly improves the model's ability to capture the semantics of complex relations.

[0068] (2) Graph structure feature extraction: By converting the input dependency tree into a graph structure (containing nodes, edges and relation types), RGCN performs relation-aware convolution operations on the node features layer by layer. Each layer aggregates neighbor information through a relation-specific weight matrix and introduces a non-linear transformation by combining the ReLU activation function. The final output is a high-order feature that integrates local and global relations.

[0069] By employing basis decomposition and regularization strategies, RGCN can effectively control the complexity of multiple relation parameters, avoid overfitting, and improve the model's generalization ability on unseen relation patterns.

[0070] In some embodiments of the present invention, S505 includes: fusing semantic features and structural features through a gate-based network to obtain fused features; and identifying the fused features through a trained causal relationship prediction model.

[0071] Gated fusion networks dynamically calculate the weights of semantic and structural features and then fuse them in a weighted manner to generate richer feature representations. This multi-feature fusion mechanism can effectively combine semantic and structural information, providing more comprehensive feature support for subsequent causal relationship extraction tasks.

[0072] The core idea of ​​gate-structured networks is to dynamically adjust the weights of semantic and structural features through a learnable gating mechanism. Specifically, given semantic features... and structural features The gate structure network first calculates the weights of the semantic features. Then use The weights, representing structural features, are then summed according to their weights to obtain the fused feature representation. Among them, semantic features The output of the BERT model, while structural features This is the output of RGCN.

[0073] More specifically, semantic feature weight calculation: The weights of semantic features are calculated using a fully connected layer (FC) and a sigmoid activation function. :

[0074] in, and These are learnable parameters. This represents the concatenation of semantic and structural features; It is the Sigmoid function, which restricts the weights to the range [0,1].

[0075] Feature fusion: based on the calculated weights The semantic and structural features are weighted and fused together.

[0076] in, It represents the feature representation after fusion, and "·" indicates element-wise multiplication.

[0077] Gating networks can dynamically adjust the weights of semantic and structural features based on the characteristics of the input sentence. For example, for sentences rich in semantic information, the model can assign higher weights to semantic features; while for sentences where structural information is more crucial, the model will increase the weights of structural features. Semantic and structural features capture semantic and syntactic information in a sentence, respectively. By fusing these two through a gating mechanism, the original semantics of the text to be identified are preserved while enhancing the expressive power of structured relational information. This allows the causal relationship prediction model to more comprehensively understand the content of the sentence in subsequent processes, thereby improving the accuracy of causal relationship extraction.

[0078] In some embodiments of the present invention, the causal relationship prediction model is trained using sample text containing causal relationships, and the sample text is generated using a diffusion model.

[0079] In some embodiments of the present invention, the diffusion model generates sample text using a reverse denoising technique guided by a causal classifier.

[0080] The process of generating sample text may specifically include: (1) Obtain the initial dataset.

[0081] The initial dataset can be obtained by collecting some publicly available causal relationship datasets. The dataset is shown in Table 1 below.

[0082] Table 1 Initial Dataset

[0083] (2) Train the diffusion model based on the initial dataset.

[0084] The core of the diffusion model is learning the data distribution through forward noise diffusion and reverse denoising processes. For text generation tasks, we adopt a continuous embedding space diffusion model, mapping discrete text sequences to a continuous embedding space for diffusion and generation. The specific design is as follows: Forward diffusion process: given the embedded representation of the original causal text Gaussian noise is gradually added during the forward process to generate a noisy embedding sequence. ,in Let be the number of diffusion steps. The noise addition process at each step can be represented as:

[0085] in, These are noise scheduling parameters that control the noise intensity at each step.

[0086] Reverse denoising process: The reverse process involves training a neural network. Gradually from noise embedding Restore the original embedding Each step of the reverse process is defined as follows:

[0087] in, It is the denoised mean of the model prediction. Fixed variance.

[0088] Training objective: The training objective of the diffusion model is to minimize the noise prediction error.

[0089] in, It is real noise added during the forward process. , .

[0090] (3) Generate sample text using a diffusion model.

[0091] To generate text that conforms to causal logic, we introduce causal constraint guidance into the diffusion model to ensure that the generated sentences contain explicit or implicit causal relationships. In the reverse denoising process, we use classifier guidance to integrate semantic constraints of causal relationships into the generation process. Specifically, this is achieved through a pre-trained causal relationship classifier. ( y Adjust the generation direction for causal labels:

[0092] in, λThe strength coefficient serves as a guide, used to balance the diversity of generation with the accuracy of causal logic.

[0093] (4) Sample text post-processing and diffusion model adjustment.

[0094] To generate sample text that conforms to causal logic, causal constraints can be introduced into the diffusion model to ensure that the generated sentences contain explicit or implicit causal relationships. The specific steps are as follows: ① Deduplication and filtering: Use embedding-based similarity calculation (such as cosine similarity) to remove semantically repetitive sentences in the sample text, and filter sample text that does not conform to the causal structure through rule matching.

[0095] ② Manual verification: Randomly selected parts are marked by domain experts to verify the rationality of their causal logic, and the model parameters and guidance strategies are adjusted based on the feedback.

[0096] ③ Enhanced diversity: Temperature scaling is introduced to regulate the randomness of the generation process, ensuring the diversity of generated text in terms of sentence structure, use of causal words, and entity combinations.

[0097] This embodiment utilizes a diffusion model to generate a sample text dataset containing causal relationships, effectively alleviating the problem of insufficient real-world corpora and providing high-quality, diverse data support for the training of subsequent causal relationship prediction models. This method can also be extended to text generation tasks in other low-resource scenarios.

[0098] In some embodiments of the present invention, the causal relationship prediction model is trained in the following manner: determining the entity recognition loss based on the entities and entity attribute labels in the sample text output by the causal relationship prediction model; determining the structure recognition loss based on the dependencies between entities and the dependencies between entities and entity attributes in the sample text output by the causal relationship prediction model; and performing reverse parameter updates on the causal relationship prediction model based on the entity recognition loss and the structure recognition loss until the entity recognition loss and the structure recognition loss meet preset requirements, at which point the causal relationship prediction model is determined to be trained successfully.

[0099] In the causal dependency tree extraction task, designing a suitable loss function is crucial for model training and optimization. To simultaneously optimize entity extraction and causal dependency tree structure prediction, this embodiment designs a multi-task loss function that combines dependency tree entity recognition loss and dependency tree structure recognition loss. Specifically: (1) Entity recognition loss.

[0100] Entity recognition is a fundamental task in causal relationship tree extraction, aiming to identify causal entities (CAUSE), result entities (RESULT), and their associated attributes (CAUSE_P and RESULT_P) from sentences. The cross-entropy loss function can be used to measure the error in entity recognition.

[0101] For each word segmentation The model predicts the corresponding entity label. (such as CAUSE, RESULT, etc.), the real tags are Entity recognition loss Defined as:

[0102] in, It refers to the number of participles in the sentence; It is a collection of entity tags (such as CAUSE, RESULT, CAUSE_P, RESULT_P, etc.); It is a participle Is the real label? (1 indicates yes, 0 indicates no); It is a model predicting word segmentation For tags The probability of.

[0103] (2) Structural recognition loss.

[0104] The task of predicting the structure of a causal dependency tree is to determine the dependencies between causal entities, result entities, and their attributes. Negative log-likelihood loss (NLL loss) can be used to measure the prediction error of the dependency tree structure.

[0105] For each dependency ,in It is the parent node. It is a child node. It is a dependency relationship, and the model predicts that its existence is probable. Dependency tree structure loss Defined as:

[0106] in, It represents the total number of dependencies; It is a collection of real dependency tree structures; It is the model's predictive dependency. The probability of its existence.

[0107] (3) Multi-task loss function To simultaneously optimize entity recognition and dependency tree structure prediction, the entity recognition loss and dependency tree structure loss can be weighted and summed to obtain the final multi-task loss function. :

[0108] in, and These are hyperparameters used to control the weights of the entity recognition loss and the dependency tree structure loss. During training, we use gradient descent (such as the Adam optimizer) to minimize the multi-task loss function. .

[0109] This embodiment combines entity recognition loss and structure recognition loss to train the causal prediction model, which can effectively guide the model to learn how to extract causal entities, result entities and their related attributes from sentences, and construct the correct causal dependency tree structure.

[0110] Reference Figure 6 This diagram illustrates a flowchart of another embodiment of the causal dependency tree construction method provided by the present invention. The causal dependency tree is obtained by jointly predicting using a BERT model, an RGCN model, a gate structure network, and a causal relationship prediction model. In one embodiment, a sample text dataset is generated using a diffusion model, and the BERT model, RGCN model, gate structure network, and causal relationship prediction model are jointly trained using the sample text dataset.

[0111] Reference Figure 7 The diagram illustrates a structural schematic of an embodiment of the causal dependency tree construction apparatus provided by the present invention. The apparatus 700 includes: The semantic feature extraction module 701 is used to obtain the semantic features of the text to be recognized; The syntactic dependency tree construction module 702 is used to construct the syntactic dependency tree of the text to be recognized, and to obtain the dependency relationship between word segments in the text to be recognized based on the syntactic dependency tree; The syntactic dependency graph construction module 703 is used to take the word segments in the text to be identified as nodes, add directed connection edges between word segment nodes with dependency relationships, and connect semantic features with each word segment node to generate a syntactic dependency graph. The structural feature extraction module 704 is used to extract the structural features of the text to be identified from the syntactic dependency graph. The causal dependency tree generation module 705 is used to identify semantic features and structural features, obtain causal relationship information of the text to be identified, and generate a causal dependency tree based on the causal relationship information.

[0112] It should be noted that the implementation principles or processes of the above modules can be referred to the aforementioned implementation examples of the causal dependency tree construction method, and will not be elaborated here.

[0113] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0114] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a causal dependency tree, characterized in that, include: Obtain the semantic features of the text to be identified; Construct a syntactic dependency tree of the text to be identified, and obtain the dependency relationships between word segments in the text to be identified based on the syntactic dependency tree; The word segments in the text to be identified are used as nodes. Directed connection edges are added between word segments that have dependencies. The semantic features are connected to each word segment node to generate a syntactic dependency graph. Extract the structural features of the text to be identified from the syntactic dependency graph; The semantic features and structural features are identified to obtain the causal relationship information of the text to be identified, and a causal dependency tree is generated based on the causal relationship information.

2. The causal dependency tree construction method according to claim 1, characterized in that, The acquisition of semantic features of the text to be identified includes: The text to be recognized is input into the BERT text encoder to obtain the semantic features of the text to be recognized.

3. The causal dependency tree construction method according to claim 1, characterized in that, The method further includes: Based on the syntactic dependency tree, the syntactic relation types between word segments that have dependencies are obtained; The syntax relation type is labeled on the directed connection edge.

4. The causal dependency tree construction method according to claim 1, characterized in that, The method further includes: When two word segments are consecutive in the text to be identified and are both nouns, a directed connection edge is added between the word segment nodes corresponding to the two word segments.

5. The causal dependency tree construction method according to claim 1, characterized in that, The extraction of structural features of the text to be identified from the syntactic dependency graph includes: The structural features of the text to be identified are extracted from the syntactic dependency graph using a relational graph convolutional network.

6. The causal dependency tree construction method according to claim 1, characterized in that, The identification of the semantic features and the structural features includes: The semantic features and the structural features are fused using a gate-based network to obtain fused features. The fused features are identified using a trained causal relationship prediction model.

7. The causal dependency tree construction method according to claim 6, characterized in that, The causal relationship prediction model is trained using sample text containing causal relationships, which is generated by a diffusion model.

8. The causal dependency tree construction method according to claim 7, characterized in that, The diffusion model generates the sample text using a reverse denoising technique guided by a causal classifier.

9. The causal dependency tree construction method according to claim 7, characterized in that, The causal relationship prediction model is trained in the following manner: The entity recognition loss is determined based on the entities and entity attribute labels in the sample text output by the causal relationship prediction model. Based on the dependencies between entities and the dependencies between entities and entity attributes in the sample text output by the causal relationship prediction model, the structural recognition loss is determined. The causal relationship prediction model is updated in reverse based on the entity recognition loss and the structure recognition loss until the entity recognition loss and the structure recognition loss meet the preset requirements, at which point the training of the causal relationship prediction model is considered complete.

10. A causal dependency tree construction device, characterized in that, include: The semantic feature extraction module is used to obtain the semantic features of the text to be recognized; The syntactic dependency tree construction module is used to construct the syntactic dependency tree of the text to be identified, and to obtain the dependency relationship between word segments in the text to be identified based on the syntactic dependency tree; The syntactic dependency graph construction module is used to take the word segments in the text to be identified as nodes, add directed connection edges between word segment nodes with dependency relationships, and connect the semantic features with each word segment node to generate a syntactic dependency graph. The structural feature extraction module is used to extract the structural features of the text to be identified from the syntactic dependency graph. The causal dependency tree generation module is used to identify the semantic features and the structural features, obtain the causal relationship information of the text to be identified, and generate a causal dependency tree based on the causal relationship information.

Images