Implicit chapter relation identification method for confronting soft hint learning of common label knowledge perception

Abstract

The invention discloses an implicit chapter relation recognition method for confronting soft prompt learning and sharing label knowledge perception, which comprises the following steps of: (1) constructing a soft prompt learning template by inserting soft prompt words and mark symbols into event pair or argument pair sentences in an input data set, and in a training process, freezing pre-training language model parameters to obtain a pre-training language model; only updating the soft prompt word and the parameter vector corresponding to the mark symbol mask word, and finally extracting the hidden layer representation corresponding to the mark symbol mask word for classification; (2) the adversarial learning network is composed of a soft prompt puzzle, a source domain label classifier, a target domain label classifier and a domain discriminator, the soft prompt puzzle is used for learning domain-invariant common label knowledge to increase the confusion degree of the domain discriminator, and the domain discriminator can improve the resolution capability; and (3) adversarial training learning: minimizing the loss function of the domain discriminator, improving the domain resolution capability, and alternately training learning.

Description

Technical Field

[0001] The present invention relates to the technical field of chapter analysis in natural language processing, and in particular to a chapter relationship recognition technology, specifically an implicit chapter relationship recognition method based on common label knowledge-aware adversarial soft prompt learning. Background Art

[0002] Implicit discourse relation recognition aims to identify the logical relationship between two adjacent text fragments (such as sentences, clauses, paragraphs, also known as arguments) without connectives as indicative information. Due to the lack of indicative connectives, identifying the logical relationship between two arguments requires in-depth modeling of argument sentence representation and capturing interaction clues between arguments, which is challenging. As a basic subtask of discourse analysis, it can provide technical support for downstream tasks such as dialogue generation, retrieval-based question-answering systems, and translation systems. The commonly used corpus for this task is the Pennsylvania Discourse Relation Treebank (PDTB2.0), which was jointly annotated by scholars from the University of Pennsylvania, the University of Torino, Italy, and the University of Edinburgh, and was officially released at the LDC (linguisticdata consortium) in 2006.

[0003] Most early studies built probabilistic statistical models based on human-constructed linguistic features (such as sentiment polarity, verb phrase length, and lexical features) to enhance the understanding of discourse relations. With the rapid development of deep learning, related studies have adopted convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to encode argument sentence representations. Subsequently, the attention mechanism has received widespread attention and has been used to capture the deep semantic interactions between arguments.

[0004] Implicit discourse relation recognition has the problem of data sparsity. Previous research has been conducted along two lines: 1) Data enhancement: removing connectives from explicit discourse relation samples as synthetic data for implicit discourse relation data or using generative models to generate variants of argument representation. The former uses samples obtained by removing connectives of cohesive semantics as implicit data, which will result in semantic discontinuity, while the latter does not enrich the diversity of argument samples. 2) Integrating external knowledge: Most of this work uses entity-category triple knowledge bases such as WordNet, which have semantic and structural differences from PDTB argument-relation pairs. According to research, the ASER event knowledge graph and PDTB have similar domain backgrounds, and there is overlap in the corpus label sets.

[0005] In 2021, scholars discovered that in the event-centric knowledge graph ASER, most event pairs lacked explicit connective semantics, and defined the implicit event relationship extraction (IERE) task based on this. The ASER and PDTB corpora have the following similarities: 1) Syntactic structure: The ASER corpus is built based on data from news, social media, movies and other fields. PDTB belongs to the news field. Event narratives and news texts have highly similar vocabulary and sentence structures, and both use implicit connectives to connect two different types of text fragments. 2) Semantic labels: PDTB and ASER have many of the same fine-grained labels, such as Contrast, Instantiation, etc. In addition, some of the fine-grained labels of IDRR are generalizations of ASER labels. The Asynchronous relationship in IDRR is equivalent to the Precedence and Succession labels in ASER, and the Cause in IDRR is a generalization of the Reason and Result labels in ASER. Both implicit event relationship extraction and implicit chapter relationship identification tasks are classification tasks, and the corpora corresponding to the two tasks have commonalities in syntactic structure and semantic label space. Therefore, the domain knowledge of event narration can be transferred to implicit chapter relations to alleviate the problem of data sparsity. Summary of the invention

[0006] The purpose of the present invention is to supplement the deficiencies in the prior art and provide an implicit chapter relationship recognition framework based on common label knowledge-aware adversarial soft prompt learning, focusing on the learning of data features corresponding to common label sets.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] A method for implicit chapter relationship recognition based on shared label knowledge-aware adversarial soft-hint learning includes the following steps:

[0009] (1) By inserting soft prompt words and marker symbols into event pairs or argument pairs in the input data set, a soft prompt learning template is constructed. During the training process, the pre-trained language model parameters are frozen, and only the parameter vectors corresponding to the soft prompt words and the marker symbol mask words are updated. Finally, the hidden layer representation corresponding to the marker symbol mask words is extracted for classification; the input data includes source data and target data, both of which are composed of samples and labels; at the same time, the source data labels and the target data labels have an intersection, forming a common label set, and the remaining labels constitute their respective private label sets;

[0010] (2) Design an adversarial learning network that perceives shared label knowledge: The adversarial learning network consists of a soft-hint obfuscator, a source domain label classifier, a target domain label classifier, and a domain discriminator. The soft-hint obfuscator is used to encode the soft-hint learning template to capture feature representations, and transmit the feature representations to the matching domain label classifier according to the domain label: if the domain label category is event, the feature representation is sent to the source domain label classifier; if the domain label category is chapter, it is sent to the target domain label classifier; the domain discriminator is used to learn the difference in feature distribution between source data and target data. The soft-hint obfuscator is used to learn domain-invariant shared label knowledge to increase the perplexity of the domain discriminator, and the domain discriminator can improve the resolution ability;

[0011] (3) Adversarial training and learning: Freeze the domain discriminator parameter vector, minimize the loss function of the source domain label classifier and the target domain label classifier, then fix the source domain label classifier and the target domain label classifier parameters, minimize the loss function of the domain discriminator, and improve the domain discrimination ability, so as to alternately train and learn.

[0012] Furthermore, step (1) specifically includes:

[0013] By inserting k into the event pair or argument pair n soft prompt words P = {[P 1 ],[P 2 ],...,[P k ]} and markers [MASK], [SEP], construct the soft prompt learning template T, as shown in formula (1), where the input consists of two single sentences, sen i is the i-th sentence;

[0014]

[0015] Then, the soft prompt learning template T is sent to the embedding layer of the pre-trained language model M to capture the input word embedding vector e(T), and the soft prompt word is initialized to a normal distribution. During the training process, the parameter vector of the pre-trained language model M is frozen, and only the parameter vectors corresponding to the soft prompt word and the marker mask word [MASK] are updated; and the hidden layer feature representation corresponding to [MASK] is extracted for classification, as shown in formulas (2) and (3):

[0016] H=M(T) (2)

[0017] r=H [MASK] (3)

[0018] Where H is the hidden layer feature representation captured by the pre-trained language model M, and H [MASK] is the hidden layer feature representation corresponding to the mask word of the [MASK] marker symbol, H [MASK] ∈Rd , d is the dimension of the word vector;

[0019] Let c n Represents the total number of categories that the domain label classifier needs to predict. The predicted probability distribution based on the soft prompt learning template T is:

[0020] y=softmax(WH [MASK] ) (4)In formula (4), W is the parameter vector, y is the predicted probability distribution;

[0021] The domain label classifier uses a multi-classification cross entropy loss function:

[0022]

[0023] Where B is the batch size, y it is the probability value of the tth category of the i-th sample. In the specific implicit event relationship extraction task, sen 1 Andsen 2 Corresponding to events 1 and events 2 ; In the implicit text relation recognition task, sen 1 Andsen 2 Respectively represent arg 1 and arg 2 .

[0024] Furthermore, step (2) specifically includes:

[0025] The adversarial learning network consists of a soft hint obfuscator S and a source domain label classifier G. 0 , target domain label classifier G 1 , domain discriminator D; the soft prompt obfuscator encodes the soft prompt learning template T and captures the hidden layer feature representation r corresponding to the mask word of the [MASK] marker symbol. The adversarial learning network sends the mask word representation to the paired domain label classifier according to the domain label corresponding to the prompt learning template T: if the domain category label is event, the feature representation is sent to the source domain label classifier G 0 ; If the domain category label is a chapter, send it to the target domain label classifier G 1 ; Through the paired domain label classifier, the probability distribution formula for predicting multiple categories is as follows:

[0026]

[0027] The domain discriminator loss function is a binary cross entropy loss function, which designs matching weights w(x) for samples corresponding to different category labels, gives higher weights to samples corresponding to label categories in the common label set, and assigns lower weights to samples corresponding to label categories in the private label set; the domain discriminator D is more sensitive to the sample features corresponding to the common label set. In the adversarial learning process, the soft prompt obfuscator can capture the domain-invariant common label knowledge features. The adversarial learning loss is as follows:

[0028]

[0029] The domain discriminator D is used to learn the difference in feature distribution between source data and target data, that is, the soft hint obfuscator S is used to learn the knowledge of domain-invariant common labels to increase the perplexity of the domain discriminator D. During this learning process, the domain discriminator D improves its own domain discrimination ability.

[0030] Furthermore, step (3) specifically includes: adversarial training follows the minimum-maximum game:

[0031]

[0032] are the loss functions of the source domain label classifier and the target domain label classifier, L D is the loss function of the domain discriminator;

[0033] In formula (8), λ is a trade-off parameter. In the game process, we first freeze the parameter vector of the domain discriminator D and minimize the event label classifier G. 0 and the chapter relationship label classifier G 1 The classification loss function is used. In this learning process, the soft hint obfuscator S improves its ability to capture domain-invariant common label knowledge features; then G is fixed. 0 With G 1 The parameters of the domain discriminator D are minimized to minimize the loss function of the domain discriminator D, which enables the domain discriminator D to improve its domain discrimination ability.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the implicit chapter relationship recognition method of common label knowledge-aware adversarial soft prompt learning are implemented.

[0035] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the implicit chapter relationship recognition method of common label knowledge-aware adversarial soft prompt learning.

[0036] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0037] (1) Under the conditions of supervised source and target domains, this paper designs a framework for implicit chapter relationship recognition based on common label knowledge-aware adversarial soft-hint learning. In the field of implicit chapter relationship recognition, this paper is the first work that focuses on extracting domain-invariant common label knowledge features.

[0038] (2) Using the soft prompt learning module, during the training process, by fixing the pre-trained language model parameters, only updating the parameter vectors corresponding to the soft prompt words and the marker mask word [MASK], the shared transferable knowledge is learned. This parameter-efficient update method alleviates the mismatch problem between the larger pre-trained language model parameters and the smaller corpus to a certain extent.

[0039] (3) Design of domain discriminator loss function: Set a higher domain discriminator learning weight for samples corresponding to shared labels, and assign a lower learning weight to samples corresponding to private labels, so as to constrain the model to learn shared label knowledge.

[0040] (4) Effectiveness of the method: The present invention is compared with other similar models on implicit paragraph relationship recognition related datasets, which proves the comparable effect of the present invention. At the same time, ablation experiments prove the effectiveness of the method proposed in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the process of the present invention.

[0042] Figure 2 Schematic diagram of the model involved in the implicit chapter relationship identification method of this embodiment.

[0043] Figure 3 Commonality graphs for implicit event relation extraction and implicit paragraph relation identification tasks. DETAILED DESCRIPTION

[0044] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0045] The present invention focuses on the implicit text relation recognition task. Given a source domain event pair E = (event 1 ,event 2 ), event tag collection Target domain chapter pair D = (arg 1 ,arg 2 ), the corresponding chapter relationship tag set This framework aims to use the common label knowledge in similar fields to assist the implicit chapter relationship recognition task. The overall model architecture of the method of the present invention is as follows: Figure 2 As shown in Figure 2, it consists of a soft hint obfuscator S and a source domain label classifier G. 0 , target domain label classifier G 1 , and domain discriminator D. Among them, the soft prompt obfuscator encodes the soft prompt learning template T, and captures the hidden layer feature representation r corresponding to the mask word of the [MASK] marker symbol. According to its label category, it is sent to the matching source / target domain label classifier. During the training process, most of the parameters of the pre-trained language model are frozen, and only the parameter vectors corresponding to the soft prompt words and the special mask word [MASK] are updated. The parameters of the soft prompt obfuscator are updated in a parameter-effective manner, which alleviates the mismatch problem between the large number of pre-trained language model parameters and the small corpus to a certain extent. In the adversarial learning process, the adversarial domain discriminator D aims to learn the difference in feature distribution between the source data and the target data. Specifically, the soft prompt obfuscator S attempts to learn the knowledge of domain-invariant common labels to increase the perplexity of the domain discriminator D. In this process, the domain discriminator improves its resolution ability.

[0046] Specifically, this embodiment provides a method for identifying implicit chapter relations by using common label knowledge-aware adversarial soft prompt learning. Figure 1 , specifically including the following steps:

[0047] (1) Constructing a soft prompt learning template, and capturing and classifying the common label features based on the soft prompt template;

[0048] By inserting k into the event pair or argument pair b soft prompt words P = {[P 1 ],[P 2 ],...,[P k ]} and markers [MASK], [SEP], construct the soft prompt learning template T, as shown in formula (1), where the input consists of two single sentences, sen i is the i-th sentence;

[0049]

[0050] Then, the soft prompt learning template T is sent to the embedding layer of the pre-trained language model M to capture the input word embedding vector e(T), and the soft prompt word is initialized to a normal distribution. During the training process, most of the parameter vectors of the pre-trained language model M are frozen, and only the parameter vectors corresponding to the soft prompt word and the special symbol [MASK] are updated; and the hidden layer representation corresponding to the [MASK] mask word is extracted for classification, as shown in formulas (2) and (3):

[0051] H=M(T) (2)

[0052] r=H [MASK] (3)

[0053] Among them, H is the hidden layer feature representation captured by the pre-trained language model M, and H [MASK] is the hidden layer feature representation corresponding to the mask word of the [MASK] marker symbol, H [MASK] ∈R d , d is the dimension of the word vector;

[0054] Let c n Represents the total number of categories that the domain label classifier needs to predict. The predicted probability distribution based on the soft prompt learning template T is:

[0055] y=softmax(WH [MASK] ) (4)In formula (4), W is the parameter vector and y is the predicted probability distribution.

[0056] The domain label classifier uses a multi-classification cross entropy loss function:

[0057]

[0058] Where B is the batch size, y it is the probability value of the tth category of the i-th sample. In the specific implicit event relationship extraction task, sen 1 Andsen 2 Corresponding to events 1 and events 2 ; In the implicit text relation recognition task, sen 1 Andsen 2 Respectively represent arg 1 and arg 2 ;

[0059] (2) Adversarial learning network based on shared label knowledge perception;

[0060] This adversarial learning network consists of a soft hint obfuscator S, a source domain label classifier G 0 , target domain label classifier G 1 , domain discriminator D. The soft hint obfuscator encodes the soft hint learning template T and captures the hidden layer representation r corresponding to the mask word of the [MASK] marker symbol. The network sends the mask word representation to the paired domain label classifier according to the domain label corresponding to the soft hint learning template T: If the domain category label is event, the feature representation is sent to the event label classifier G 0 ; If the domain category label is chapter, send it to the chapter relationship label classifier G 1; Through the paired domain label classifier, the probability distribution formula for predicting multiple categories is as follows:

[0061]

[0062] The domain discriminator loss is a binary cross entropy loss. This embodiment designs matching weights w(x) for samples corresponding to different category labels, gives higher weights to samples corresponding to the common label set label category, and assigns lower weights to samples corresponding to the private label set label category. Therefore, the domain discriminator D is more sensitive to the sample features corresponding to the common label set. In the adversarial learning process, the soft prompt obfuscator can capture the domain-invariant common label knowledge features. The adversarial learning loss is as follows:

[0063]

[0064] The domain discriminator D aims to learn the difference in feature distribution between source data and target data. Specifically, the software obfuscator S attempts to learn the knowledge of domain-invariant common labels to increase the perplexity of the domain discriminator D. In this process, the domain discriminator D improves its own domain discrimination ability.

[0065] (3) Adversarial training learning module:

[0066] Adversarial training follows a min-max game:

[0067]

[0068] In formula (8), λ is a trade-off parameter that adjusts the importance of the classification loss of the source domain and the target domain. In the game process, we first freeze the parameter vector of the domain discriminator D and minimize the event label classifier G. 0 and the chapter relationship label classifier G 1 The classification loss; then, fix G 0 With G 1 The parameters of the domain discriminator D are minimized, which enables D to improve its domain discrimination ability. In the adversarial learning process, the soft-hint confuser S captures the domain-invariant common label knowledge features to confuse the domain discriminator D.

[0069] The following is a specific description of the method of this embodiment through a specific experimental environment and evaluation method:

[0070] In this embodiment, the corpus used for the implicit text relation recognition task is PDTB2.0, and the implicit event relation extraction task selects ASER as the corpus.

[0071] PDTB2.0 is a manually annotated text relation corpus based on 2,312 Wall Street Journal articles. It contains three semantic levels: Class level, Type level, and Subtype level. The PDTB2.0 Type-level labels have a high degree of overlap with ASER labels. Therefore, this embodiment conducts an experiment on 11 types of discourse relationship classification at the Type level (the PDTB2.0 corpus Type level contains 16 discourse relationship categories, but 5 of them are removed due to the lack of validation and test sets), including: Temporal.Synchronous, Temporal.Asynchronous, Contingency.Cause, Contingency.PragmaticCause, Comparison.Contrast, Comparison.Concession, Expansion.Conjunction, Expansion.Instantiation, Expansion.Restatement, Expansion.Alternative, and Expansion.List. Following the PDTB-Ji data segmentation method, we selected chapters 2-20 as the training set, chapters 0-1 as the validation set, and chapters 21-22 as the test set.

[0072] Existing large-scale knowledge graphs focus on entity knowledge and ignore activity, state, and event knowledge, which is used to describe the behavior of entities or things in the real world. The ASER (activities, states, events, and their relations) corpus was developed. This is a large-scale event knowledge graph extracted from more than 11 billion labeled unstructured text data. It contains 15 relationship categories, covers 194 million unique events, and 64 million relations. Among them, ASER data covers comments, news, social media, movies and other fields. The event relationship between data follows the definition of chapter relationship in PDTB2.0. Figure 3 The mappability of paragraph and event label categories is demonstrated. The difference between the two is that the paragraph relationship recognition task focuses on the relationship between two sentences / clauses, while the event relationship recognition task focuses on the relationship between two events.

[0073] For the implicit event relationship extraction task, scholars removed the connectives of explicit event relationship instances in the ASER core version, and retained the 2,200 instances with the highest confidence scores for each event category (a total of 14 event categories), and used 2,000 instances corresponding to each event category as training sets, 100 instances as validation sets, and 100 instances as test sets. Among them, there are some event categories with less than 2,000 training examples, specifically: Synchronous: 320, Instantiation: 345, Restatement: 1,102, Exception: 1,414. In this way, we obtained 23,181 training sets, 1,400 validation sets, and 1,400 test sets. The event relationship labels involved are: Synchronous, Precedence, Succession, Reason, Result, Condition, Contrast, Concession, Conjunction, Instantiation, Restatement, Alternative, Exception, and ChoosenAlternative.

[0074] At the same time, this embodiment performs label alignment on ASER and PDTB corpus. The Precedence and Succession labels in ASER are equivalent to the Asynchronous relationship in PDTB. The Reason and Result labels in ASER are the summary of the Cause relationship in PDTB. The Alternative and ChoosenAlternative labels in ASER are equivalent to the Alternative relationship in PDTB. After label alignment, ASER has 11 categories and PDTB still has 11 categories.

[0075] The implicit chapter relationship recognition method of this embodiment is implemented based on the PyTorch framework. In the specific implementation process, the template length of the input prompt learning is set to 512. This embodiment selects the RoBERTa-base model for encoding, which includes 12 multi-head attention modules and a word vector dimension of 768. The optimal batch size is 8, and the learning rates corresponding to the soft prompt obfuscator, source domain label classifier, target domain label classifier, and domain discriminator are 1e-3, 1e-4, 1e-4, and 1e-10, respectively. The number of iterations is 100 times, the hierarchical dual-view contrast learning loss weight is 50, and the optimizer strategy is Adam optimization. The specific experiment of this invention is implemented based on a 1×24GB NVIDIA RTX3090 GPU.

[0076] The evaluation indicators are as follows:

[0077] The evaluation indicators of this embodiment are multi-classification accuracy (Accuracy, Acc) and multi-classification macro-average F1 score (Macro-F 1 score, Macro-F1). If a set of argument pairs contains multiple discourse relations, we consider the model to be correct as long as it hits one of the correct categories. In order to verify the effectiveness of the method of the present invention, the method of the present invention (CCKASPL) is compared with the current advanced and representative methods.

[0078] First, we introduce the comparison model of the PDTB2.0 dataset:

[0079] (IDRR-C&E) The model [1] A knowledge regularization method is implemented to integrate external event knowledge and linguistic constraints based on entity coreference relations into the neural network model.

[0080] (MTL-KT) The model [2] The discourse relations and connectives are predicted simultaneously in a multi-task learning manner, and the implicit connective embedding knowledge is transferred to the discourse relation embedding.

[0081] (FT-RoBERTa) This method is a fine-tuned RoBERTa model [3] Among them, the RoBERTa model improves the performance of BERT by designing dynamic MASK and removing NSP tasks.

[0082] (HierMTN-CRF) This model [4] A hierarchical multi-task neural network with conditional random fields is used to jointly model multi-level semantic classification tasks and encode the semantic hierarchy into features at different levels.

[0083] (MT-BERT) The model [5]Three techniques are used to fine-tune the BERT model to adapt to the implicit text relation recognition task, namely: text relation classification, explicit connective prediction, and implicit connective prediction.

[0084] (BMGF-RoBERTa) This model [6] By integrating the bilateral multi-view matching module and the global information fusion module, it is used to enhance the understanding of textual relationships.

[0085] (CG-T5) This model [7] The implicit discourse relationship recognition task is jointly modeled using classification and generation methods, and a variety of target sentence formats are designed for the generative model to learn and verify knowledge.

[0086] (PCP) The model [8] A cue-based connective prediction method is proposed, which uses the strong correlation between connectives and text relations to assist in identifying implicit text relations.

[0087] (LSDGM) This model [9] It is a label-dependency-aware sequence generation model that regards multi-layer implicit discourse relationship recognition as a conditional label sequence generation task.

[0088] Table 1 Performance comparison of the proposed method and the comparative method on the PDTB2.0 dataset

[0089]

[0090] Table 2 Label-related F1 values ​​of the method of the present invention based on the PDTB2.0 corpus

[0091]

[0092]

[0093] Table 1 shows the performance of the existing model on the PDTB2.0 corpus. This example selects Macro-F1 and Accuracy as the evaluation indicators for the classification of 11 types of text relations at the Type level. The following observations are made:

[0094] (1) Overall, the model involved in the method of the present invention has achieved comparable results compared to other baseline models (but has not reached the SOTA value), indicating that the model of the present invention deeply captures the label knowledge characteristics shared by chapters and events, and uses common knowledge to enhance the encoding representation of implicit chapter argument pairs, thereby assisting the implicit chapter relationship recognition process. The proposed CCKASPL model is effective.

[0095] (2) The proposed model achieved a Macro-F1 value of 41.53%, which is comparable to other models. This shows that the CCKASPL model has a good overall performance in precision and recall. However, the accuracy of this model is 58.14%, which still has room for improvement. The potential reason may be that the event-private semantic labels cause slight disturbances to the soft-prompt obfuscator. The soft-prompt obfuscator captures the domain-invariant common label knowledge features, but is also disturbed by the noise of the event-private label set.

[0096] Table 2 shows the F1 values ​​of the existing CCKASPL model for each label category at the PDTB-L2 level. The model of the present invention exceeds the comparison model (DHCL is the model proposed in Chapter 3) in the three categories of accidental-causal, extension-selection, and extension-enumeration, reaching 64.57%, 70.59%, and 34.78% F1 values, respectively, with an improvement of 0.21%, 7.13%, and 24.25%. Among them, accidental-causal and extension-selection categories are categories in the common label set, and the training set corresponding to the causal and selection categories of the ASER corpus is relatively large, which is 4,000. This shows that the model has effectively transferred the causal and selection category knowledge of the ASER corpus to the IDRR task. Extension-enumeration is a category in the private label set. The reason is speculated: the model has a strong overall perception ability for the extension category, so the recognition effect on the enumeration category is improved.

[0097] Preferably, the embodiment of the present application also provides a specific implementation of an electronic device capable of implementing all steps in the implicit chapter relationship recognition method of common label knowledge-aware adversarial soft prompt learning in the above embodiment, and the electronic device specifically includes the following contents:

[0098] Processor, memory, communications interface and bus;

[0099] Among them, the processor, memory, and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices such as server-side devices, metering devices, and user-side devices.

[0100] The processor is used to call the computer program in the memory, and when the processor executes the computer program, all the steps in the implicit chapter relationship recognition method of common label knowledge-aware adversarial soft prompt learning in the above embodiment are implemented.

[0101] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the implicit chapter relationship identification method for common label knowledge-aware adversarial soft prompt learning in the above-mentioned embodiments. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements all the steps of the implicit chapter relationship identification method for common label knowledge-aware adversarial soft prompt learning in the above-mentioned embodiments.

[0102] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0103] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0105] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0106] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0108] The above contents are intended to illustrate the technical solution of the present invention schematically, and the present invention is not limited to the embodiments described above. Without departing from the scope of the present invention and the scope protected by the claims, a person skilled in the art may make many specific changes under the guidance of the present invention, which are all within the protection scope of the present invention.

[0109] Finally, it should be pointed out that the above examples are only used to illustrate the calculation process of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above examples, a person skilled in the art should understand that the calculation process described in the above examples can still be modified, or some parameters can be replaced by equivalents, and these modifications or replacements do not make the essence of the corresponding calculation method deviate from the spirit and scope of the calculation method of the present invention.

[0110] References:

[0111] [1]Dai Z, Huang RA Regularization Approach for Incorporating EventKnowledge and Coreference Relations into Neural Discourse Parsing[C]. / / EMNLP.2019.

[0112] [2]Van Ngo L,Than K,Nguyen T H. Employing the correspondence ofrelations and connectives to identify implicit discourse relations via labelembeddings[C] / / ACL.2019.

[0113] [3]Liu Y,Ott M,Goyal N,et al.Roberta:A robustly optimized bertpretraining approach[J].arXiv preprint arXiv:1907.11692,2019.

[0114] [4]Wu C,Hu C,Li R,et al.Hierarchical multi-task learning with CRF forimplicit discourse relation recognition[J].Knowledge-Based Systems,2020,195:105637.

[0115] [5]Kishimoto Y,Murawaki Y,Kurohashi S.Adapting bert to implicitdiscourse relation classification with a focus on discourse connectives[C] / / LERC.2020.

[0116] [6]Liu X,Ou J,Song Y,et al.On the importance of word and sentencerepresentation learning in implicit discourse relation classification[C] / / IJCAI.2020.

[0117] [7]Jiang F,Fan Y,Chu X,et al.Not just classification:Recognizingimplicit discourse relation on joint modeling of classification andgeneration[C] / / EMNLP.2021.

[0118] [8]Zhou H,Lan M,Wu Y,et al.Prompt-based Connective Prediction Methodfor Fine-grained Implicit Discourse Relation Recognition[J] / / EMNLPFindings.2022.

[0119] [9]Wu C,Cao L,Ge Y,et al.A label dependence-aware sequence generationmodel for multi-level implicit discourse relation recognition[C] / / AAAI.2022.

Claims

1. A common label knowledge-aware implicit chapter relationship recognition method based on adversarial soft-prompt learning, characterized in that: The following steps are involved: (1) By inserting soft prompt words and marker symbols into event pairs or argument pairs in the input data set, a soft prompt learning template is constructed. During the training process, the pre-trained language model parameters are frozen, and only the parameter vectors corresponding to the soft prompt words and the marker symbol mask words are updated. Finally, the hidden layer representation corresponding to the marker symbol mask words is extracted for classification; the input data includes source data and target data, both of which are composed of samples and labels; at the same time, the source data labels and the target data labels have an intersection, forming a common label set, and the remaining labels constitute their respective private label sets; (2) Design an adversarial learning network that perceives shared label knowledge: The adversarial learning network consists of a soft-hint obfuscator, a source domain label classifier, a target domain label classifier, and a domain discriminator. The soft-hint obfuscator is used to encode the soft-hint learning template to capture feature representations, and transmit the feature representations to the matching domain label classifier according to the domain label: if the domain label category is event, the feature representation is sent to the source domain label classifier; if the domain label category is chapter, it is sent to the target domain label classifier; the domain discriminator is used to learn the difference in feature distribution between source data and target data. The soft-hint obfuscator is used to learn domain-invariant shared label knowledge to increase the perplexity of the domain discriminator, and the domain discriminator can improve the resolution ability; (3) Adversarial training and learning: Freeze the domain discriminator parameter vector, minimize the loss function of the source domain label classifier and the target domain label classifier, then fix the source domain label classifier and the target domain label classifier parameters, minimize the loss function of the domain discriminator, and improve the domain discrimination ability, so as to alternately train and learn.

2. According to claim 1, a method for implicit chapter relationship recognition based on common label knowledge perception and adversarial soft prompt learning is characterized in that: Step (1) specifically includes: By inserting k into the event pair or argument pair n soft prompt words P = {[P1], [P2], ..., [P k ]} and markers [MASK], [SEP], construct the soft prompt learning template T, as shown in formula (1), where the input consists of two single sentences, sen i is the i-th sentence; Then, the soft prompt learning template T is sent to the embedding layer of the pre-trained language model M to capture the input word embedding vector e(T), and the soft prompt word is initialized to a normal distribution. During the training process, the parameter vector of the pre-trained language model M is frozen, and only the parameter vectors corresponding to the soft prompt word and the marker mask word [MASK] are updated; and the hidden layer feature representation corresponding to [MASK] is extracted for classification, as shown in formulas (2) and (3): H=M(T)(2)r=H [MASK] (3) Where H is the hidden layer feature representation captured by the pre-trained language model M, H [MASK] is the hidden layer feature representation corresponding to the mask word of the [MASK] marker symbol, H [MASK] ∈R d , d is the dimension of the word vector; Let c n Represents the total number of categories that the domain label classifier needs to predict. The predicted probability distribution based on the soft prompt learning template T is: y=softmax(WH [MASK] ) (4) In formula (4), W is the parameter vector, y is the predicted probability distribution; The domain label classifier uses a multi-classification cross entropy loss function: Where B is the batch size, y it is the probability value of the tth category of the i-th sample. In the specific implicit event relationship extraction task, sen1 and sen2 correspond to event1 and event2 respectively; in the implicit chapter relationship recognition task, sen1 and sen2 represent arg1 and arg2 respectively.

3. According to claim 1, a method for implicit chapter relationship recognition based on common label knowledge perception and adversarial soft prompt learning is characterized in that: Step (2) specifically includes: The adversarial learning network consists of a soft prompt obfuscator S, a source domain label classifier G0, a target domain label classifier G1, and a domain discriminator D; the soft prompt obfuscator encodes the soft prompt learning template T and captures the hidden layer feature representation r corresponding to the mask word of the [MASK] marker symbol. The adversarial learning network sends the mask word representation to the paired domain label classifier according to the domain label corresponding to the prompt learning template T: if the domain category label is an event, the feature representation is sent to the source domain label classifier G0; if the domain category label is a chapter, it is sent to the target domain label classifier G1; through the paired domain label classifier, the probability distribution formula for predicting multiple categories is as follows: The domain discriminator loss function is a binary cross entropy loss function, which designs matching weights w(x) for samples corresponding to different category labels, gives higher weights to samples corresponding to label categories in the common label set, and assigns lower weights to samples corresponding to label categories in the private label set; the domain discriminator D is more sensitive to the sample features corresponding to the common label set. In the adversarial learning process, the soft prompt obfuscator can capture the domain-invariant common label knowledge features. The adversarial learning loss is as follows: The domain discriminator D is used to learn the difference in feature distribution between source data and target data, that is, the soft hint obfuscator S is used to learn the knowledge of domain-invariant common labels to increase the perplexity of the domain discriminator D. During this learning process, the domain discriminator D improves its own domain discrimination ability.

4. According to claim 1, a method for implicit chapter relationship recognition based on common label knowledge perception and adversarial soft prompt learning is characterized in that: Step (3) specifically includes: adversarial training follows the minimum-maximum game: are the loss functions of the source domain label classifier and the target domain label classifier, L D is the loss function of the domain discriminator; In formula (8), λ is a trade-off parameter. In the game process, the parameter vector of the domain discriminator D is first frozen to minimize the classification loss function of the event label classifier G0 and the chapter relationship label classifier G1. In this learning process, the soft prompt obfuscator S improves its ability to capture domain-invariant common label knowledge features. Then, the parameters of G0 and G1 are fixed to minimize the loss function of the domain discriminator D, which encourages the domain discriminator D to improve its domain identification ability.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the implicit chapter relationship recognition method of common label knowledge-aware adversarial soft prompt learning as described in any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the implicit chapter relationship recognition method of common label knowledge-aware adversarial soft prompt learning as described in any one of claims 1 to 4 are implemented.

Images