A method for implicit discourse relationship analysis based on semantic decomposition

Abstract

The present invention relates to an implicit chapter relationship analysis method based on semantic decomposition, which belongs to the field of natural language processing application technology. This method processes different semantic information in arguments separately by designing a diversion network structure and a complete process, including using a semantic transfer network and a semantic decomposition network to realize information decomposition of argument semantics. Information that is useful for implicit chapter relationship analysis flows into a task-related network, and information that is of no practical use for implicit chapter relationship analysis flows into a task-independent network. In this way, the model's ability to screen and understand semantic information is improved, achieving the technical effect of improving implicit chapter relationship analysis. This method can effectively realize semantic decomposition and information diversion, and can enable users to quickly and accurately obtain implicit chapter relationship analysis results.

Description

Technical Field

[0001] The present invention relates to an implicit text relationship analysis method based on semantic decomposition, and belongs to the technical field of natural language processing applications. Background Art

[0002] Discourse relationship analysis is a key task in natural language processing (NLP). It aims to identify and categorize the relationships between sentences in multi-sentence text. Effectively identifying and applying discourse relationships can improve the performance of various downstream NLP tasks, such as machine translation and text summarization.

[0003] Discourse relations can be categorized as explicit or implicit, depending on the presence of discourse connectives (such as "but" and "so") between sentences. Identifying implicit discourse relations without connectives presents a major challenge. Due to the lack of connectives, implicit discourse relations can only be identified by understanding the underlying semantics of the text to identify the logical relationships between sentences. However, current deep learning models lack the ability to filter and understand truly valuable semantic information when mining deep semantics.

[0004] Correctly analyzing implicit discourse relations often requires the model to understand deeper semantics. However, not all semantic information is useful. For example, in Example 1, "<Argument 1> Its index inched up to 47.6% in October from 46% in September. <Argument 2> Any reading below 50% suggests the manufacturing sector is generally declining." The specific indexes for September and October revealed in Argument 1, and even the time information of September and October, have no practical value for implicit discourse relation classification. In other words, whether or not the model captures the index information for each month will not affect its ability to make correct implicit discourse relation judgments. Therefore, when understanding deeper semantics, the model also needs to distinguish these semantic information, thereby filtering and transforming the semantic information that is truly useful for the final implicit discourse relation analysis.

[0005] By recreating the human process of understanding implicit discourse relations and analyzing the value of different information for model classification, we found that the semantic properties of the entire argument and the core events described by the argument are more valuable for implicit discourse relationship analysis. However, the topic, environment, and other descriptive details involved in the argument are of little help in the final classification. Summary of the Invention

[0006] This paper addresses the technical problem of existing implicit discourse relationship analysis techniques failing to distinguish between the different contents and values ​​of semantic information by proposing an implicit discourse relationship analysis method based on semantic decomposition. This method utilizes semantic decomposition and simultaneously constructs a dual-network classifier to filter and understand deep semantics, thereby improving the effectiveness of implicit discourse relationship analysis.

[0007] The innovation of this invention lies in the design of a branching network structure and a complete process to separately process different semantic information in arguments, including the use of a semantic transfer network and a semantic decomposition network to achieve information decomposition of argument semantics. Information useful for implicit discourse relationship analysis is fed into a task-relevant network, while information of no practical use is fed into a task-independent network. This improves the model's ability to filter and understand semantic information, achieving the technical effect of enhancing implicit discourse relationship analysis.

[0008] To achieve the above objectives, the present invention adopts the following technical solutions.

[0009] A method for implicit discourse relation analysis based on semantic decomposition. First, an encoder is used to semantically encode the two arguments of a sample. Then, a semantic decomposition network is used to decompose the argument semantics into task-relevant semantics that are useful for the implicit discourse relation analysis task and task-irrelevant semantics that are not valuable for the task. The different semantics decomposed from the two arguments are then combined into semantic pairs and fed into the task-relevant and task-irrelevant networks, respectively. By combining a unique data construction scheme and model training method, the argument semantic model can effectively perform semantic decomposition, allowing the model to focus on semantic information that is more valuable to the task. Finally, the argument semantic model is used to analyze implicit discourse relations.

[0010] Beneficial effects

[0011] Compared with the prior art, the method of the present invention has the following advantages:

[0012] 1. This method can effectively achieve semantic decomposition and information diversion. In the process of argument semantic modeling, it distinguishes the content of different semantic information and the different values ​​that different semantic information provides in the implicit discourse relationship analysis task.

[0013] 2. This method can effectively improve the effect of implicit text relationship recognition.

[0014] 3. This method enables users to quickly and accurately obtain the analysis results of implicit text relations. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the method of the present invention;

[0016] Figure 2This is a system architecture diagram of the method of the present invention. DETAILED DESCRIPTION

[0017] The method of the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0018] like Figure 1 As shown in FIG, a method for analyzing implicit discourse relations based on semantic decomposition includes the following steps:

[0019] Step 1: Use the encoder to semantically encode the two arguments of the sample and obtain the corresponding semantic representation.

[0020] Specifically, the method includes the following steps:

[0021] Step 1.1: Preprocess the samples in the PDTB2.0 corpus and convert all tokens in the samples to lowercase.

[0022] Step 1.2: Use the encoder to semantically encode the two arguments (arg1, arg2) of each sample to obtain the corresponding semantic representation.

[0023] Among them, the semantic representation includes the state vector (H arg1 ,H arg2 ) and sentence vector (S arg1 ,S arg2 ). Among them, H arg1 is the state vector in argument 1, m is the number of tokens in argument 1, Represents the vector representation of each token in arg1, d h is the spatial dimension of the state vector, R represents the dimension m*d h Matrix; H arg2 is the state vector in argument 2, n is the number of tokens in argument 2, Represents the vector representation of each token in arg2. S arg1 is the sentence vector of argument 1, S arg2 is the sentence vector of argument 2.

[0024] Step 2: With the help of interactive attention mechanism, the weight distribution of each token in the two arguments (Att1, Att2) is calculated. Att1 represents the weight distribution of argument 1, and Att2 represents the weight distribution of argument 2. Then the final weighted representation of each argument (S′) is calculated. arg1 ,S′ arg2 ), S′ arg1 represents the weight of argument 1, S′ arg2 Represents the weight of argument 2.

[0025] The specific method is as follows:

[0026] Step 2.1: Sentence vector S of argument 2 arg2 and the state vector H of argument 1 arg1 Perform dot multiplication and calculate the weight distribution Att1 of argument 1 through softmax.

[0027] Step 2.2: Sentence vector S of argument 1 arg1 and the state vector H of argument 2 arg2 Perform dot multiplication and calculate the weight distribution Att2 of argument 2 through softmax.

[0028] Step 2.3: Distribute the weights of the two arguments (Att1, Att2) and the state vector (H arg1 ,H arg2 ) is weighted and summed to obtain the final weighted representation (S′ arg1 ,S′ arg2 ).

[0029] Step 3: Decompose the argument semantics into task-relevant semantics that are useful for the implicit discourse relation analysis task and task-irrelevant semantics that are of no value to the task.

[0030] Specifically, the method includes the following steps:

[0031] Step 3.1: Use the semantic transfer network (a symmetric MLP neural network) to calculate the final weighted representation (S′) of each argument. arg1 ,S′ arg2 ) to perform semantic transfer and obtain high-order semantic representations that are relatively easy to decompose (Q arg1 ,Q arg2 ).

[0032] First, the final weighted representation of each argument (S′ arg1 , S′ arg2 ) is compressed in half to obtain the space vector (P arg1 ,P arg2 Then, it is restored to its original dimension to obtain a high-order semantic representation (Q arg1 ,Q arg2 ). Among them, P arg1 represents the space vector of argument 1, P arg2 A space vector representing argument 2. Q arg1 Represents the high-order semantics of argument 1, Q arg2 Represents the high-order semantics of argument 2.

[0033] Step 3.2: Use semantic decomposition network to represent the high-level semantics of each argument (Q arg1 ,Qarg2 ) for semantic decomposition. Each argument will get two different semantic expressions and form two sets of semantic pairs. Among them, the high-order semantic decomposition of argument 1 is The higher-order semantic decomposition of argument 2 yields And form two groups of semantic pairs.

[0034] Furthermore, the following method can be used to form semantic pairs:

[0035] First, use the mapping matrix M to (Q arg1 ,Q arg2 ) to do spatial mapping and obtain task-related semantic pairs Among them, x represents task related, represents the task-related representation of arg1, Represents the task-dependent representation of arg2.

[0036] Then, based on the task-related semantics, the task-independent semantic pairs are calculated. Right now Among them, y means task-irrelevant, represents the task-independent semantics of arg1, Indicates the task-independent semantics of arg2.

[0037] Step 4: The different semantics decomposed from the two arguments are combined into semantic pairs and sent to the task-related network and the task-independent network respectively. Through data construction and model training, the argument semantic model can effectively perform semantic decomposition, allowing the model to focus on semantic information that is more valuable to the task. Figure 2 shown.

[0038] Specifically, a dual-network classifier is constructed to classify the task-related semantic pairs in the semantic decomposition results of the two arguments. After splicing, the task-related network is input to determine the implicit textual relationship between the two arguments; at the same time, the task-irrelevant semantic pairs in the semantic decomposition results of the two arguments are Perform splicing and input into the task-independent network to determine whether the two arguments constitute a chapter association.

[0039] The dual-network classifier consists of a task-relevant network and a task-independent network. When training the dual-network classifier, in addition to using argument pairs with textual associations as positive samples, negative examples are also constructed for argument pairs without textual associations. During training, the cross-entropy loss function is used as the objective function, and the Adam gradient update algorithm is employed to update the parameters of the argument semantic model.

[0040] Step 5: Use the argument semantic model to analyze implicit discourse relations.

[0041] Specifically, after the dual-network classifier is trained, the argument pair to be identified is input into the dual-network classifier to obtain output results, including the output of the task-related network and the output of the task-independent network. The output of the task-related network is recorded as the recognition result of this chapter relationship.

[0042] Example

[0043] This embodiment describes the specific implementation process of automatically summarizing text using the method of the present invention.

[0044] Existing automatic text summarization technologies only model sentence content, the order of sentences, and paragraph information, but do not effectively model the inherent semantic discourse relationships between sentences. To address this issue, the present invention proposes a discourse relationship analysis method to help improve the performance of downstream automatic text summarization technologies. The specific implementation method is as follows:

[0045] Step 1: For each text sample, construct argument pairs.

[0046] First, the preceding and following clauses within each sentence are extracted as candidate argument pairs. Then, the preceding and following entire sentences are extracted as candidate argument pairs. Using the constructed rules, useful argument pairs are selected from the candidate argument pairs to serve as samples for discourse analysis.

[0047] Step 2: For the argument pairs obtained in step 1, apply the text analysis technology of the method of the present invention to obtain the text relationship of each argument pair.

[0048] Step 3: In the existing automatic text summarization model, additional modeling of paragraph relationships is added to enable the model to more effectively understand the semantic connections between different clauses and different whole sentences, thereby improving the final effect of automatic text summarization.

Claims

1. A method for analyzing implicit discourse relations based on semantic decomposition, characterized in that: The following steps are involved: Step 1: Use the encoder to semantically encode the two arguments (arg1, arg2) of each sample to obtain the corresponding semantic representation; Among them, the semantic representation includes the state vector (H arg1 ,H arg2 ) and sentence vector (S arg1 ,S arg2 ); where H arg1 is the state vector in argument 1, m is the number of tokens in argument 1, Represents the vector representation of each token in arg1, d h is the spatial dimension of the state vector, R represents the dimension m*d h Matrix; H arg2 is the state vector in argument 2, n is the number of tokens in argument 2, Represents the vector representation of each token in arg2; S arg1 is the sentence vector of argument 1, S arg2 is the sentence vector of argument 2; Step 2: With the help of interactive attention mechanism, the weight distribution of each token in the two arguments (Att1, Att2) is calculated. Att1 represents the weight distribution of argument 1, and Att2 represents the weight distribution of argument 2. Then the final weighted representation of each argument (S′) is calculated. arg1 ,S′ arg2 ), S′ arg1 represents the weight of argument 1, S′ arg2 represents the weight of argument 2; Step 3: Decompose the argument semantics into task-related semantics that are useful for the implicit discourse relation analysis task and task-irrelevant semantics that are not valuable for the task; including the following steps: Step 3.1: Using the semantic transfer network, the final weighted representation (S′) of each argument arg1 ,S′ arg2 ) to perform semantic transfer and obtain high-order semantic representations that are relatively easy to decompose (Q arg1 ,Q arg2 ); First, the final weighted representation of each argument (S′ arg1 , S′ arg2 ) is compressed in half to obtain the space vector (P arg1 ,P arg2 ); then, restore it to its original dimension to obtain a high-order semantic representation (Q arg1 ,Q arg2 ); where P arg1 represents the space vector of argument 1, P arg2 The space vector representing argument 2; Q arg1 Represents the high-order semantics of argument 1, Q arg2 Represents the higher-order semantics of argument 2; Step 3.2: Use semantic decomposition network to represent the high-level semantics of each argument (Q arg1 ,Q arg2 ) for semantic decomposition; each argument will get two different semantic expressions and form two sets of semantic pairs; among them, the high-order semantic decomposition of argument 1 is The higher-order semantic decomposition of argument 2 yields And form two sets of semantic pairs; Step 4: The different semantics decomposed from the two arguments are combined into semantic pairs and fed into the task-related network and the task-independent network respectively. Through data construction and model training, the argument semantic model can effectively perform semantic decomposition, allowing the model to focus on semantic information that is more valuable to the task. The dual-network classifier includes a task-related network and a task-independent network. When training the dual-network classifier, in addition to using argument pairs with textual associations as positive samples, it is also necessary to construct argument pairs without textual associations as negative samples for training. During training, the cross-entropy loss function is used as the objective function, and the parameters in the argument semantic model are updated. Step 5: Use the argument semantic model to analyze implicit discourse relations.

2. The implicit text relationship analysis method based on semantic decomposition according to claim 1, characterized in that: Step 2 includes the following steps: Step 2.1: Translate the sentence vector S of argument 2 arg2 and the state vector H of argument 1 arg1 Perform dot multiplication and calculate the weight distribution Att1 of argument 1 through softmax; Step 2.2: Sentence vector S of argument 1 arg1 and the state vector H of argument 2 arg2 Perform point multiplication and calculate the weight distribution Att2 of argument 2 through softmax; Step 2.3: Distribute the weights of the two arguments (Att1, Att2) and the state vector (H arg1 ,H arg2 ) is weighted and summed to obtain the final weighted representation (S′ arg1 ,S′ arg2 ).

3. The implicit text relationship analysis method based on semantic decomposition according to claim 1, characterized in that: In step 3.2, the semantic pairs are formed using the following method: First, use the mapping matrix M to (Q arg1 ,Q arg2 ) to do spatial mapping and obtain task-related semantic pairs Among them, x represents task related, represents the task-related representation of arg1, represents the task-related representation of arg2; Then, based on the task-related semantics, the task-independent semantic pair is calculated. Right now Among them, y means task-irrelevant, represents the task-independent semantics of arg1, Indicates the task-independent semantics of arg2.

4. The implicit text relationship analysis method based on semantic decomposition according to claim 1, characterized in that: In step 4, a dual-network classifier is constructed to classify the task-related semantic pairs in the semantic decomposition results of the two arguments. After splicing, the task-related network is input to determine the implicit textual relationship between the two arguments; at the same time, the task-irrelevant semantic pairs in the semantic decomposition results of the two arguments are Perform splicing and input into the task-independent network to determine whether the two arguments constitute a text association.

5. The implicit text relationship analysis method based on semantic decomposition according to claim 1, characterized in that: In step 5, after the dual-network classifier training is completed, the argument pair to be identified is input into the dual-network classifier to obtain the output results, including the output results of the task-related network and the results of the task-independent network; finally, the results of the task-related network are recorded as the recognition results of this chapter relationship.

Images