A chapter relationship recognition system

Abstract

The present invention discloses a chapter relationship recognition system. The system is based on an explicit chapter relationship data set, constructs an artificial implicit chapter relationship data set that covers multiple fields and is automatically annotated, and uses the artificial implicit chapter relationship data set as training data to extract domain-shared features and domain-private features, thereby outputting prediction results for implicit chapter relationship recognition. At the same time, the present invention introduces adversarial training with a noise filtering mechanism, which can effectively alleviate the noise interference introduced by pseudo-labels; at the same time, the present invention designs domain generalization training of a hybrid expert mechanism, which enhances the system's adaptability to unknown target fields, so that the recognition effect of the present invention when applied to unknown target fields is significantly better than that of existing methods. Moreover, the present invention does not require any manually annotated implicit chapter relationship data during training, is applicable to most languages, and has a wider range of applications.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text relationship recognition system. Background Art

[0002] With the accelerating pace of digitalization, massive amounts of text data continue to emerge in scenarios such as social media, news reports, and online comments. Natural language processing (NLP) technology is playing an increasingly critical role in understanding and analyzing textual information. Discourse relation recognition (DRR), a core task in understanding the structure and meaning of multi-sentence text, has garnered widespread attention. This task aims to identify semantic relationships between adjacent or related segments (e.g., sentences or clauses) in a text, such as causal, parallel, and contrast relationships. Based on the presence of explicit connectives, DRR is generally divided into explicit DRR (Explicit Discourse Relation Recognition) and implicit DRR (Implicit Discourse Relation Recognition). The DRR approach, which relies on direct clues provided by connectives, performs well for model recognition. However, the IDRR approach, lacking explicit markers, requires models to reason comprehensively from semantic, contextual, and even pragmatic perspectives, presenting greater challenges. Although researchers have proposed various neural network models and pre-trained language models in recent years to improve IDRR performance, its recognition accuracy remains significantly lower than that of explicit discourse relation recognition, limiting its practical effectiveness in downstream applications. Therefore, improving the performance of implicit discourse relation recognition remains a research hotspot and a key challenge in the current NLP field.

[0003] Currently, methods for identifying implicit discourse relations have evolved from traditional approaches relying on manual feature engineering to the automatic learning of distributed features using deep learning. Early research primarily relied on manually constructed grammatical, semantic, and lexical features, combined with traditional machine learning algorithms such as support vector machines for relation classification. However, these methods suffered from significant limitations in feature extraction and generalization. With the rapid development of deep learning technology, methods based on neural networks that automatically learn distributed features have gradually become mainstream. Recurrent neural networks (such as LSTM and GRU), convolutional neural networks (CNNs), and attention mechanisms have been widely used to learn semantic representations between arguments. Graph neural networks (GNNs) have also been introduced to construct complex argument relationship graphs, and the integration of attention mechanisms has enhanced the model's ability to model structured information. In recent years, pre-trained language models (such as BERT and RoBERTa) have acquired powerful contextual modeling capabilities through pre-training on large-scale corpora and have demonstrated significant advantages in discourse relation tasks after fine-tuning. Furthermore, emerging techniques such as hint learning and knowledge distillation have begun to be applied to IDRR tasks, helping to improve model recognition performance.

[0004] However, existing implicit discourse relation recognition methods still face two major problems: 1) Due to the limited amount of manually annotated training corpus and the relatively single domain, the model performs poorly in cross-domain applications, especially when applied to unknown domains, the recognition effect decreases significantly; 2) There is no manually annotated implicit discourse relation data in most languages, making it difficult to apply existing methods to these languages, greatly limiting their scope of application. Summary of the Invention

[0005] To this end, an embodiment of the present invention proposes a text relationship recognition system to improve the recognition effect of unknown fields and expand the scope of language application.

[0006] According to an embodiment of the present invention, the chapter relationship recognition system includes an automatic annotation submodule for chapter relationship data, an implicit chapter relationship recognition submodule that integrates domain shared and private features, an adversarial training submodule with a noise filtering mechanism, and a domain generalization training submodule based on a hybrid expert mechanism;

[0007] In the automatic annotation submodule of the text relation data, the heuristic rule extraction K Explicit chapter relation instances in the domain, we get K We divide any explicit chapter relation dataset into instances containing unambiguous connectives and instances containing ambiguous connectives, and automatically annotate the instances containing unambiguous connectives and instances containing ambiguous connectives respectively. Then, we merge the automatic annotation results to obtain the coverage KAn automatically annotated artificial implicit discourse relation dataset in various fields;

[0008] In the implicit chapter relationship recognition submodule that integrates domain-shared and domain-private features, the domain-shared features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-shared feature extraction network, and the domain-private features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-private feature extraction network. Then, based on the domain-shared and domain-private features of each instance, the classification layer outputs the prediction results of implicit chapter relationship recognition.

[0009] In the adversarial training submodule with noise filtering mechanism, based on a domain discrimination layer shared by all experts, the training instances are predicted to be K The predicted probability distribution on the domain, and dynamically calculate the weight of the training instance, and then construct the cost function of the adversarial training phase with a noise filtering mechanism L 1. Minimize the cost function L 1, until convergence;

[0010] In the domain generalization training submodule based on the hybrid expert mechanism, domain experts and non-domain expert groups are defined, and then the domain experts and non-domain expert groups are aligned to construct the cost function L 21 , and align between non-domain expert groups to construct the cost function L 22 , successively minimize the cost function L 21 and L 22 , until convergence.

[0011] According to the chapter relationship recognition system of the embodiment of the present invention, based on the explicit chapter relationship dataset, an artificial implicit chapter relationship dataset covering multiple fields and automatically annotated is constructed, and the artificial implicit chapter relationship dataset is used as training data to extract domain-shared features and domain-private features, thereby outputting the prediction results of implicit chapter relationship recognition. At the same time, the present invention introduces adversarial training with a noise filtering mechanism, which can effectively alleviate the noise interference introduced by pseudo-labels; at the same time, the present invention designs domain generalization training of a hybrid expert mechanism, which enhances the system's adaptability to unknown target fields, so that the recognition effect of the present invention when applied to unknown target fields is significantly better than that of existing methods. Moreover, the present invention does not require any manually annotated implicit chapter relationship data during training, is applicable to most languages, and has a wider range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and / or additional aspects and advantages of the embodiments of the present invention will become apparent and readily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0013] Figure 1 2 is a schematic structural diagram of a chapter relationship recognition system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0015] See also Figure 1 An embodiment of the present invention proposes a chapter relationship recognition system, which includes an automatic annotation submodule for chapter relationship data, an implicit chapter relationship recognition submodule that integrates domain shared and private features, an adversarial training submodule with a noise filtering mechanism, and a domain generalization training submodule based on a hybrid expert mechanism.

[0016] In the automatic annotation submodule of the text relation data, the heuristic rule extraction K Explicit chapter relation instances in the domain, we get K We divide any explicit chapter relation dataset into instances containing unambiguous connectives and instances containing ambiguous connectives, and automatically annotate the instances containing unambiguous connectives and instances containing ambiguous connectives respectively. Then, we merge the automatic annotation results to obtain the coverage K An automatically annotated artificial implicit discourse relation dataset in various fields.

[0017] In most languages, explicit discourse relation data is relatively easy to obtain, allowing for the collection of large amounts of data across multiple domains. On the one hand, removing connectives from explicit discourse relation instances can yield artificial implicit discourse relation instances. On the other hand, most connectives are unambiguous, such as "because" (because) corresponding to "causal relationship" and "but" (but) corresponding to "transformative relationship." Only a small number of connectives are ambiguous, such as "so" (so) corresponding to "causal relationship" or "temporal relationship."

[0018] Specifically, in the automatic annotation submodule of the chapter relationship data, the heuristic rules are first extracted. K Explicit chapter relation instances in the domain, we get K Explicit Text Relationship Dataset , 、 、 The first and k , KAn explicit chapter relation dataset, ,The heuristic rules include the extraction template “argument 1, connective + argument 2”, and the constraint rule “each argument must contain at least one verb”;

[0019] Depending on whether the conjunction in the example is ambiguous, Divide into sets of instances containing unambiguous connectives , and the set of instances containing ambiguous conjunctions ;

[0020] for For any instance in , we first remove the conjunctions. Since these conjunctions are not ambiguous, they can be directly mapped to the corresponding discourse relation categories, and we obtain the first artificial implicit discourse relation dataset with automatic annotation. ;

[0021] for For any instance in , first remove the connectives. Since these connectives are ambiguous, external verification is required before deciding which chapter relationship to map them to. Specifically, two pre-trained models with different properties (e.g., RoBERTa and GPT) are used to predict connectives suitable for connecting two arguments. If the two predicted connectives are different from the original connectives but can both represent the same chapter relationship, the instance is retained and automatically labeled as the corresponding chapter relationship; if the two predicted connectives are the same as the original connectives, or the two predicted connectives are different from the original connectives but cannot represent the same chapter relationship, the instance is discarded. Finally, a second artificial implicit chapter relationship dataset with automatic annotation is obtained. ;

[0022] merge and The automatic annotation instance in k Automatically annotated artificial implicit discourse relation dataset in various fields , and thus covered K A domain-specific, automatically annotated dataset of artificial implicit discourse relations , The first and K This is an artificial implicit text relation dataset with automatic annotation in various fields. The An example is , The artificial implicit text relation of automatic annotation is .

[0023] In the implicit chapter relationship recognition submodule that integrates domain-shared and private features, the domain-shared features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-shared feature extraction network, and the domain-private features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-private feature extraction network. Then, based on the domain-shared features and domain-private features of each instance, the classification layer outputs the prediction results of implicit chapter relationship recognition.

[0024] Existing methods typically only apply domain-shared features to unknown domains, ignoring the role of domain-private features. To better utilize domain-shared and domain-private features, this paper decouples these features and constructs an implicit discourse relationship recognition submodule that integrates domain-shared and domain-private features.

[0025] Specifically, in the implicit chapter relationship recognition submodule that integrates domain-shared and private features, we first construct K An implicit chapter relation recognition model, denoted as K experts, each expert consists of a domain-shared feature extraction network, a domain-private feature extraction network and a classification layer. The domain-shared feature extraction network of each expert uses the same model parameters, while the domain-private feature extraction network has its own independent model parameters.

[0026] The domain-shared features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-shared feature extraction network. The domain-private features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-private feature extraction network. The expression is:

[0027] ;

[0028] ;

[0029] in, for The instances, A domain-shared feature extraction network common to all experts, For the k domain-specific feature extraction networks of experts, is the maximum pooling operation, for Domain-shared features, for Both the domain-shared feature extraction network and the domain-private feature extraction network can use pre-trained models such as BERT or RoBERTa.

[0030] It should be pointed out that when implementing The input usually requires special processing based on the pre-trained language model used, for example, adding placeholders [CLS] and [SEP] at the beginning of the instance and between two arguments respectively.

[0031] Based on the domain-shared features and domain-private features of each instance, the classification layer outputs the prediction results of implicit chapter relationship recognition, which can be expressed as:

[0032] ;

[0033] ;

[0034] in, for The predicted probability distribution over all text relation categories, for The final semantic vector representation of is the normalization function, and For the k The parameters that need to be learned by the classification layer of each expert, It is a vector concatenation operation.

[0035] In the adversarial training submodule with noise filtering mechanism, based on a domain discrimination layer shared by all experts, the training instances are predicted to be K The predicted probability distribution on the domain, and dynamically calculate the weight of the training instance, and then construct the cost function of the adversarial training phase with a noise filtering mechanism L 1. Minimize the cost function L 1, until convergence.

[0036] Training multiple experts for implicit discourse relationship recognition in unknown domains based on automatically annotated multi-domain datasets also faces challenges such as data noise, domain mismatch, and explicit / implicit data mismatch. To address these issues, the following describes adversarial training with a noise filtering mechanism proposed in this paper. Its main objectives are: 1) to minimize noise instances in automatically annotated data, and 2) to better learn domain-shared features and domain-private features across multiple domains to alleviate the domain mismatch problem.

[0037] To more effectively learn cross-domain domain-shared features, adversarial training is employed. A domain-discriminative layer, shared by all experts, is introduced to predict the domain to which an input instance belongs. If the discriminative layer has difficulty accurately identifying the domain to which an instance belongs, it indicates that the extracted domain-shared features are highly domain-independent, indicating that the domain-shared features have been learned more thoroughly.

[0038] Specifically, in the adversarial training submodule with noise filtering mechanism, based on a domain discrimination layer shared by all experts, the training instance is predicted to be K The predicted probability distribution on the domain is expressed as:

[0039] ;

[0040] in, for In all K The predicted probability distribution over the domain, and These are the parameters that need to be learned in the domain discrimination layer. The more uniform the probability value distribution in , the more difficult it is for the domain discrimination layer to accurately identify the domain to which the instance belongs.

[0041] In order to filter out as many noise instances as possible from the automatically labeled data, the present invention prioritizes samples with lower cross-entropy loss and higher adversarial training loss to participate in model training during the training process. A lower cross-entropy loss indicates that the automatic labeling result of the instance has a higher credibility; a higher adversarial training loss indicates that the domain characteristics of the instance are not obvious, which is more conducive to learning domain-related shared features. In each round of training, the weight of the training instance is dynamically calculated, where a weight of 1 indicates that the instance is selected for training, and a weight of 0 indicates that the instance is ignored. By selectively ignoring a certain proportion of samples, the purpose of filtering out noisy training instances can be achieved.

[0042] Specifically, in the process of dynamically calculating the weights of training instances, the following formula is satisfied:

[0043] ;

[0044] in, for The weight of for The artificial implicit chapter relationship of automatic annotation, is the cross entropy cost function for classification, is the function used to calculate entropy, and is a set hyperparameter. The larger the entropy, the more difficult it is for the model to distinguish the domain of the instance, and the corresponding adversarial loss is also higher.

[0045] Based on the dynamically calculated weights of training instances, we can Simultaneous training K Experts define the cost function of the adversarial training phase with noise filtering mechanism L The expression for 1 is:

[0046] ;

[0047] in, for The total number of training instances in , This is the transpose operation of a vector. The closer the value of is to 0, the weaker the correlation between the domain-shared features and domain-private features learned by the model, which helps to effectively decouple the two. The goal of adversarial training with noise filtering mechanism is to minimize L 1, until convergence.

[0048] In the domain generalization training submodule based on the hybrid expert mechanism, domain experts and non-domain expert groups are defined, and then the domain experts and non-domain expert groups are aligned to construct the cost function L 21 , and align between non-domain expert groups to construct the cost function L 22 , successively minimize the cost function L 21 and L 22 , until convergence.

[0049] Among them, in most current studies, people usually only focus on applying domain-shared features to unknown target domains, while ignoring the role of domain-private features. However, when there are large differences between different domains, it is often difficult to achieve ideal results by relying solely on domain-shared features. To this end, the present invention further optimizes the domain-private feature extraction network in the domain generalization training stage based on the hybrid expert mechanism, so that when integrating multiple experts to deal with unknown target domains, domain-private features can be more fully utilized to achieve complementary enhancement. The training stage includes two steps: the first step: alignment of domain experts and non-domain expert groups to further enhance domain generalization capabilities; the second step: alignment between non-domain expert groups to alleviate explicit / implicit data mismatch problems.

[0050] The first step is to train in any field As the current field, experts are the corresponding field experts, and the remaining K -1 experts constitute the non-domain expert group. For any training instance in the data set, the goal of this step of training is to make the prediction results of the corresponding domain experts and the prediction results of the non-domain expert group as consistent as possible. To this end, the cost function is defined L 21 The expression is:

[0051] ;

[0052] ;

[0053] in, represents the Kullback-Leibler distance calculation function, is the prediction result of the non-field expert group, For non-field expert group The prediction results of experts.

[0054] The second step of training is to alleviate the problem of explicit / implicit data mismatch. First, collect data from multiple fields (different from the automatic annotation of text relationship data). K domains) to build a dataset of unlabeled implicit discourse relations ; Then, all non-domain experts in the non-domain expert group are randomly divided into two non-domain expert groups and , and align the prediction results of these two non-domain expert groups. To this end, define the cost function L 22 The expression is:

[0055] ;

[0056] ;

[0057] ;

[0058] in, It is a dataset constructed from unlabeled implicit discourse relation instances. for The total number of instances in , for The instances, The first non-field expert group right The prediction results, For the second non-domain expert right The prediction results, The first non-field expert group The number of experts in the China-Africa field, For the second non-domain expert The number of experts in the China-Africa field, The first non-field expert group Middle Non-domain experts The prediction results, The second non-field expert group Middle Non-domain experts prediction results.

[0059] It should be noted that the domain generalization training phase based on the hybrid expert mechanism fixes the parameters of the domain-shared feature extraction network and only optimizes the parameters of the domain-private feature extraction network and each classification layer of all expert models. The domain generalization training based on the hybrid expert mechanism successively minimizes the cost function L 21 and L 22 , until convergence, that is, the final trained K An expert.

[0060] In specific applications, for input instances in unknown target areas ,joint K The prediction results of implicit chapter relationship recognition output by experts are :

[0061] ;

[0062] in, For the Expert pairs of input instances prediction results.

[0063] In summary, according to the chapter relationship recognition system of this embodiment, based on the explicit chapter relationship dataset, an artificial implicit chapter relationship dataset covering multiple fields and automatically annotated is constructed, and the artificial implicit chapter relationship dataset is used as training data to extract domain shared features and domain private features, thereby outputting the prediction results of implicit chapter relationship recognition. At the same time, the present invention introduces adversarial training with a noise filtering mechanism, which can effectively alleviate the noise interference introduced by pseudo-labels; at the same time, the present invention designs domain generalization training of a hybrid expert mechanism, which enhances the system's adaptability to unknown target fields, so that the recognition effect of the present invention when applied to unknown target fields is significantly better than that of existing methods. Moreover, the present invention does not require any manually annotated implicit chapter relationship data during training, and is applicable to most languages, with a wider range of applications.

[0064] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0065] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A chapter relationship recognition system, characterized by: It includes an automatic annotation submodule for chapter relationship data, an implicit chapter relationship recognition submodule that integrates domain shared and private features, an adversarial training submodule with a noise filtering mechanism, and a domain generalization training submodule based on a hybrid expert mechanism. In the automatic annotation submodule of the text relation data, the heuristic rule extraction K Explicit chapter relation instances in the domain, we get K We divide any explicit chapter relation dataset into instances containing unambiguous connectives and instances containing ambiguous connectives, and automatically annotate the instances containing unambiguous connectives and instances containing ambiguous connectives respectively. Then, we merge the automatic annotation results to obtain the coverage K An automatically annotated artificial implicit discourse relation dataset in various fields; In the implicit chapter relationship recognition submodule that integrates domain-shared and domain-private features, the domain-shared features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-shared feature extraction network, and the domain-private features of each instance in the artificial implicit chapter relationship dataset are extracted based on the domain-private feature extraction network. Then, based on the domain-shared and domain-private features of each instance, the classification layer outputs the prediction results of implicit chapter relationship recognition. In the adversarial training submodule with noise filtering mechanism, based on a domain discrimination layer shared by all experts, the training instances are predicted to be K The predicted probability distribution on the domain, and dynamically calculate the weight of the training instance, and then construct the cost function of the adversarial training phase with a noise filtering mechanism L 1. Minimize the cost function L 1, until convergence; In the domain generalization training submodule based on the hybrid expert mechanism, domain experts and non-domain expert groups are defined, and then the domain experts and non-domain expert groups are aligned to construct the cost function L 21 , and align between non-domain expert groups to construct the cost function L 22 , successively minimize the cost function L 21 and L 22 , until convergence.

2. The chapter relationship recognition system according to claim 1, characterized in that: In the automatic annotation submodule of the chapter relationship data, we first extract K Explicit chapter relation instances in the domain, we get K Explicit Text Relationship Dataset , 、 、 The first and k , K An explicit chapter relation dataset, ; Depending on whether the conjunction in the example is ambiguous, Divide into sets of instances containing unambiguous connectives , and the set of instances containing ambiguous conjunctions ; for For any instance in , we first remove the conjunctions and then directly map it to the corresponding discourse relation category, thus obtaining the first artificial implicit discourse relation dataset with automatic annotation. ; for For any instance in , first remove the connectives, then use two pre-trained models with different properties to predict the connectives suitable for connecting the two arguments. If the two predicted connectives are different from the original connectives but can both represent the same chapter relationship, then retain the instance and automatically annotate it with the corresponding chapter relationship; if the two predicted connectives are the same as the original connectives, or the two predicted connectives are different from the original connectives but cannot represent the same chapter relationship, discard the instance. Finally, we get the automatically annotated second artificial implicit chapter relationship dataset. ; merge and The automatic annotation instance in k Automatically annotated artificial implicit discourse relation dataset in various fields , and thus covered K A domain-specific, automatically annotated dataset of artificial implicit discourse relations , The first and K An automatically annotated artificial implicit discourse relation dataset in various fields.

3. The chapter relationship recognition system according to claim 2, characterized in that: In the implicit chapter relationship recognition submodule that integrates domain-shared and private features, we first construct K An implicit chapter relation recognition model, denoted as K Experts are then trained to extract the domain-shared features of each instance in the artificial implicit chapter relation dataset based on the domain-shared feature extraction network, and the domain-private features of each instance in the artificial implicit chapter relation dataset based on the domain-private feature extraction network. The expression is: ； ； in, for The instances, A domain-shared feature extraction network common to all experts, For the k domain-specific feature extraction networks of experts, is the maximum pooling operation, for Domain-shared features, for Domain-private features of .

4. The chapter relationship recognition system according to claim 3, characterized in that: In the implicit chapter relationship recognition submodule that integrates domain-shared and domain-private features, the classification layer outputs the prediction result of implicit chapter relationship recognition based on the domain-shared features and domain-private features of each instance, which is expressed as: ； ； in, for The predicted probability distribution over all text relation categories, for The final semantic vector representation of is the normalization function, and For the k The parameters that need to be learned by the classification layer of each expert, It is a vector concatenation operation.

5. The chapter relationship recognition system according to claim 4, characterized in that: In the adversarial training submodule with noise filtering mechanism, based on a domain discrimination layer shared by all experts, the training instances are predicted to be K The predicted probability distribution on the domain is expressed as: ； in, for In all K The predicted probability distribution over the domain, and These are the parameters that need to be learned in the domain discrimination layer.

6. The chapter relationship recognition system according to claim 5, characterized in that: In the adversarial training submodule with noise filtering mechanism, the following formula is satisfied during the dynamic calculation of the weights of the training instances: ； in, for The weight of for The artificial implicit chapter relationship of automatic annotation, is the cross entropy cost function for classification, is the function used to calculate entropy, and is the set hyperparameter.

7. The chapter relationship recognition system according to claim 6, characterized in that: Cost function of the adversarial training phase with noise filtering mechanism L The expression for 1 is: ； in, for The total number of training instances in , This is the transpose operation of a vector.

8. The chapter relationship recognition system according to claim 7, characterized in that: Cost function L 21 The expression is: ； ； in, represents the Kullback-Leibler distance calculation function, is the prediction result of the non-field expert group, For non-field expert group The prediction results of experts.

9. The chapter relationship recognition system according to claim 8, characterized in that: Cost function L 22 The expression is: ； ； ； in, It is a dataset constructed from unlabeled implicit discourse relation instances. for The total number of instances in , for The instances, The first non-field expert group right The prediction results, For the second non-domain expert right The prediction results, The first non-field expert group The number of experts in the China-Africa field, For the second non-domain expert The number of experts in the China-Africa field, The first non-field expert group Middle Non-domain experts The prediction results, The second non-field expert group Middle Non-domain experts prediction results.

Images