A cross-domain implicit discourse relation identification method

Abstract

The application provides a cross-field implicit discourse relation recognition method, which comprises the following steps: constructing a multi-granularity hybrid expert coding module based on a multi-layer hybrid expert enhanced Transformer layer; constructing a double-path prediction module based on a conjunction word perception prediction network, a discourse element semantic direct prediction network and a gating mechanism; using a pre-trained language model to encode and process implicit discourse relation instances to obtain basic semantic representations; using the multi-granularity hybrid expert coding module to perform multi-round iterative enhancement processing on the basic semantic representations to obtain the final semantic representation of a global placeholder and the final semantic of a conjunction word placeholder; and using the double-path prediction module to process the final semantic representation of the global placeholder and the final semantic representation of the conjunction word placeholder to obtain a discourse relation recognition result. The application effectively improves the implicit discourse relation recognition performance in the cross-field scene by combining multi-granularity hybrid expert modeling and explicit discourse relation data enhancement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a cross-domain implicit discourse relation identification method. BACKGROUND

[0002] Implicit discourse relation identification aims to determine the semantic relationship between two adjacent or non-adjacent text units (called arguments) in a discourse, which is a basic task in the field of natural language understanding. Compared with explicit discourse relation identification, implicit discourse relation identification lacks obvious connecting words or structural markers, and the relationship determination mainly relies on context semantic reasoning, which is still a challenging task in the era of large models. At present, explicit discourse relation identification can achieve an accuracy of over 95%, while the accuracy of implicit discourse relation identification is only about 70%, and the performance in cross-domain scenarios is even worse. In news reports and user comments and other texts, discourse connecting words are often omitted and semantic associations are directly expressed, resulting in the widespread existence of implicit discourse relations in most languages such as Chinese and English. Accurate identification of implicit discourse relations helps to improve the overall performance of upper-level applications such as text understanding, information extraction and intelligent question answering, and therefore has important significance in academic research and practical application.

[0003] Although existing methods have promoted the development of implicit discourse relation identification technology to some extent, there are still certain limitations in terms of models and data. In terms of models, existing implicit discourse relation identification models mostly rely on fixed structures or single granularity to model the semantic features of argument pairs, making it difficult to balance fine-grained semantic features and global semantic relationship features, and the expressiveness of complex implicit discourse relations is still limited, especially in cross-domain scenarios. In terms of data, existing methods mainly use multi-task learning or pre-training to utilize explicit discourse relation data to assist implicit discourse relation identification, but this utilization method still has limitations. Specifically, the auxiliary role of explicit discourse relation data is usually limited to the training stage, and its deep semantic connection with implicit discourse relations cannot be fully reflected in the model structure, so that it cannot be fully utilized in assisting implicit discourse relation identification. SUMMARY

[0004] In view of the above situation, the main purpose of the present application is to provide a cross-domain implicit discourse relation identification method to solve the above technical problems.

[0005] The present application provides a cross-domain implicit discourse relation identification method, which comprises the following steps:

[0006] Step 1, a word-level hybrid expert network and an instance-level hybrid expert network are introduced into a Transformer layer to obtain a hybrid expert enhanced Transformer layer, a multi-granularity hybrid expert encoding module is constructed based on the multi-layer hybrid expert enhanced Transformer layer; a connection word perception prediction network and a discourse element semantic direct prediction network are respectively constructed based on a Softmax function and a linear transformation mechanism, and a double-path prediction module is constructed based on the connection word perception prediction network, the discourse element semantic direct prediction network and a gating mechanism; a cross-domain implicit discourse relationship identification model is formed by using the multi-granularity hybrid expert encoding module and the double-path prediction module;

[0007] Step 2, the implicit discourse relationship instances are encoded and processed by using a pre-trained language model to obtain a basic semantic representation containing word elements, global placeholders and connection word placeholders;

[0008] Step 3, the basic semantic representation containing word elements, global placeholders and connection word placeholders is subjected to multi-round iteration enhancement processing by using the multi-granularity hybrid expert encoding module to obtain a final semantic representation of the global placeholder and a final semantic representation of the connection word placeholder;

[0009] Step 4, the final semantic representation of the global placeholder and the final semantic representation of the connection word placeholder are processed by using the double-path prediction module to obtain a discourse relationship identification result;

[0010] Step 5, the cross-domain implicit discourse relationship identification model is pre-trained by using a multi-domain explicit discourse relationship data set, and then fine-tuned by using an implicit discourse relationship data set, and an cross-entropy loss function based on the discourse relationship identification result and a balanced loss function based on the weights in the word-level hybrid expert network and the instance-level hybrid expert network are introduced to obtain the trained cross-domain implicit discourse relationship identification model; the trained cross-domain implicit discourse relationship identification model is used to predict the category of the input instance.

[0011] Compared with the prior art, the beneficial effects of the present application are:

[0012] 1. According to the different fields of the input sample, the multi-granularity hybrid expert encoding module can adaptively select different expert networks for reasoning, which effectively improves the performance of the model in the cross-domain scene;

[0013] 2. The double-path prediction module with connection word perception simulates the human cognitive mechanism through the connection word perception prediction network, which improves the performance of implicit discourse relationship identification and enhances the explainability of the model;

[0014] 3. A large amount of multi-domain explicit discourse relationship data is used for data enhancement of the model, and the multi-granularity hybrid expert encoding module and the double-path prediction module are fully trained, which improves the generalization ability of the model in the cross-domain scene.

[0015] Additional aspects and advantages of the present application will be set forth in part in the description which follows, and in part will be obvious from the description, or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A cross-domain implicit discourse relation recognition method according to the present application is shown in the flow chart. DETAILED DESCRIPTION

[0017] Embodiments of the present application are described in detail below with reference to the attached drawings. The embodiments of the present application are shown in the drawings as follows, in which the same or similar components are denoted by the same or similar reference numerals throughout. The embodiments described below with reference to the drawings are exemplary and are intended to explain the present application, and should not be construed as limiting the present application.

[0018] These and other aspects of embodiments of the present application will be more apparent from the following description and accompanying drawings. In these descriptions and drawings, some specific embodiments of embodiments of the present application are specifically disclosed to represent some ways of embodying principles of embodiments of the present application, but it should be understood that the scope of embodiments of the present application is not limited thereto.

[0019] Referring to the following description and drawings, these and other aspects of embodiments of the present application will be apparent. In these descriptions and drawings, some specific embodiments of embodiments of the present application are specifically disclosed to represent some ways of embodying principles of embodiments of the present application, but it should be understood that the scope of embodiments of the present application is not limited thereto. Figure 1 The present embodiment provides a cross-domain implicit discourse relation recognition method, which comprises the following steps:

[0020] Step 1, introduce a word-level mixed expert network and an instance-level mixed expert network in a Transformer layer to obtain a mixed expert enhanced Transformer layer, and construct a multi-granularity mixed expert encoding module based on the multi-layer mixed expert enhanced Transformer layer; construct a connection word perception prediction network and a lemma semantic direct prediction network based on a Softmax function and a linear transformation mechanism, respectively, and construct a dual-path prediction module based on the connection word perception prediction network, the lemma semantic direct prediction network, and a gating mechanism; and use the multi-granularity mixed expert encoding module and the dual-path prediction module to constitute a cross-domain implicit discourse relation recognition model.

[0021] Step 2, use a pre-trained language model to encode and process the implicit discourse relation instance to obtain a basic semantic representation containing word elements, global placeholders, and connection word placeholders.

[0022] In step 2, the pre-trained language model is used to encode and process the implicit discourse relation instance to obtain the basic semantic representation, and the following relationship exists in the corresponding process:

[0023] ;

[0024] in, This represents an instance of implicit discourse relations. Argument 1 represents an instance of an implicit discourse relation. Argument 2 represents an instance of implicit discourse relation. The basic semantic representation of global placeholders. The basic semantic representation of the first lexical unit. Indicates the first The basic semantic representation of each word element Indicates the index of a word, and ; Indicates the first The basic semantic representation of each word element Indicates the number of lexical units. The basic semantic representation of conjunction placeholders. This indicates that the language has been encoded using a pre-trained language model. This represents a natural language prompt template.

[0025] It should be noted that there are many types of natural language prompt templates. For example, when used for implicit discourse relation recognition in English, it might be: <cls> <sep> <sep>the connective between two arguments is <mask>"wherein <sep>For different input units between the delimiter, for Chinese implicit discourse relations to identify the corresponding changes can be, <cls>for global placeholders, <mask>For conjunction word placeholder.

[0026] Step 3, performing multi-round iterative enhancement processing on the basic semantic representation containing word units, global placeholders and conjunction word placeholders by using a multi-granularity hybrid expert encoding module to obtain the final semantic representation of the global placeholders and the final semantic representation of the conjunction word placeholders.

[0027] In step 3, the multi-round iterative enhancement processing is performed on the basic semantic representation containing word units, global placeholders and conjunction word placeholders by using a multi-granularity hybrid expert encoding module to obtain the final semantic representation of the global placeholders and the final semantic representation of the conjunction word placeholders, which specifically includes the following sub-steps:

[0028] S301, inputting the basic semantic representation of the word units, the basic semantic representation of the global placeholders and the basic semantic representation of the conjunction word placeholders into the multi-head attention mechanism sub-layer of the first layer of hybrid expert enhanced Transformer layer for processing to obtain the word unit level semantic representation, the semantic representation of the global placeholders and the semantic representation of the conjunction word placeholders output by the multi-head attention mechanism sub-layer of the first layer of hybrid expert enhanced Transformer layer;

[0029] S302, inputting the word unit level semantic representation output by the multi-head attention mechanism sub-layer of the first layer of hybrid expert enhanced Transformer layer into the word unit level hybrid expert network composed of word unit level experts, and the word unit level hybrid expert network dynamically calculates the weights of each word unit level expert according to the input information, and selects word unit level experts with the largest weights to participate in calculation to obtain the word unit level semantic representation output by the word unit level hybrid expert network, and there is the following relationship in the corresponding process:

[0030] ;

[0031] wherein, represents the weight vector before normalization in the word unit level hybrid expert network, and both represent learnable parameters in the word unit level hybrid expert network, represents the word unit level semantic representation output by the multi-head attention mechanism sub-layer of the first layer of hybrid expert enhanced Transformer layer, and both represent the index of the weight in , represents the processing after the TopK function, represents the number of selected word unit level experts, and both represent the selected normalized weight value corresponding to the selected and unnormalized weight value corresponding to the selected unnormalized weight value corresponding to the selected after Softmax function processing token-level semantic representation output by the token-level hybrid expert network normalized weight value corresponding to the selected normalized weight value corresponding to the selected selected token-level expert, which is usually composed of multiple nonlinear fully connected layers;

[0032] It should be noted that the total number of token-level experts is positively correlated with the size of the training data, that is, when the amount of training data is large, the number of token-level experts can be appropriately increased, for example, set to 30; and the number of token-level experts participating in calculation each time should not be too large, preferably 2-4.

[0033] S303, sequentially performing normalization processing and residual connection on the token-level semantic representation output by the token-level hybrid expert network to obtain the token-level semantic representation output by the first layer of hybrid expert enhanced Transformer layer, and there is the following relationship formula in the corresponding process:

[0034] ;

[0035] wherein, token-level semantic representation output by the first layer of hybrid expert enhanced Transformer layer, after normalization processing

[0036] S304, inputting the semantic representation of the global placeholder output by the multi-head attention mechanism sub-layer in the first layer of hybrid expert enhanced Transformer layer into an instance-level hybrid expert network composed of instance-level experts, the instance-level hybrid expert network dynamically calculates the weights of each instance-level expert according to the input information, and selects instance-level experts with the largest weights to participate in calculation to obtain the semantic representation of the global placeholder output by the instance-level hybrid expert network, and there is the following relationship formula in the corresponding process:

[0037] ;

[0038] wherein, unnormalized weight vector in the instance-level hybrid expert network, and learnable parameters in the instance-level hybrid expert network, semantic representation of global placeholder output by the first layer of mixed expert enhanced Transformer layer, and both represent the index of weight in represents the number of selected instance-level experts, and both represent the normalized weight value corresponding to the selected instance-level experts, and both represent the weight value before normalization corresponding to the selected instance-level experts, semantic representation of global placeholder output by the instance-level mixed expert network, normalized weight value corresponding to the instance-level expert, selected instance-level expert, usually composed of multiple nonlinear fully connected layers;

[0039] S305, sequentially performing normalization processing and residual connection on the semantic representation of global placeholder output by the instance-level mixed expert network to obtain the semantic representation of global placeholder output by the first layer of mixed expert enhanced Transformer layer, and the following relationship exists in the corresponding process:

[0040]

[0041] wherein, semantic representation of global placeholder output by the first layer of mixed expert enhanced Transformer layer;

[0042] S306, repeating the steps of S304 to S305 on the semantic representation of conjunction placeholder output by the multi-head attention mechanism sub-layer in the first layer of mixed expert enhanced Transformer layer to obtain the semantic representation of conjunction placeholder output by the first layer of mixed expert enhanced Transformer layer

[0043] It should be noted that although only the word-level mixed expert network and the instance-level mixed expert network are described in the embodiment, the present application can also be extended to more granular mixed expert networks in a similar manner in specific implementation, such as phrase-level or argument-level mixed expert networks.

[0044] repeating the word-level semantic representation, the semantic representation of global placeholder and the semantic representation of conjunction placeholder output by the first layer of mixed expert enhanced Transformer layer in an iterative form​​​ Steps S301 to S306 are used to obtain the final semantic representation of the global placeholder and the final semantic representation of the connector placeholder. The following relationship exists in the correspondence process:

[0045] ;

[0046] in, Indicates the first The semantic representation of global placeholders in the output of the Transformer layer enhanced by layer hybrid expert. Indicates the first The semantic representation corresponding to the first lexical unit output by the layered hybrid expert-enhanced Transformer layer. Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first Semantic representation of connective placeholders in the output of the Transformer layer enhanced by layered hybrid expert. Indicates the first Layered hybrid expert-enhanced Transformer layers, This represents the index of the Transformer layer enhanced by hybrid experts, and ; Indicates the first The semantic representation of global placeholders in the output of the Transformer layer enhanced by layered hybrid experts. Indicates the first The semantic representation corresponding to the first lexical unit output by the layer hybrid expert-enhanced Transformer layer. Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first Semantic representation of connective placeholders in the output of the Transformer layer enhanced by layered hybrid expert;

[0047] It should be noted that the multi-granularity hybrid expert coding module is composed of... A multi-granularity hybrid expert enhanced Transformer layer is stacked.

[0048] Step 4, processing the final semantic representation of the global placeholder and the final semantic representation of the conjunction placeholder by using the two-way prediction module to obtain a discourse relation recognition result.

[0049] In step 4, the final semantic representation of the global placeholder and the final semantic representation of the conjunction placeholder are processed by using the two-way prediction module to obtain a discourse relation recognition result, which includes the following sub-steps:

[0050] The final semantic representation of the conjunction placeholder is predicted and derived by using the conjunction-aware prediction network to obtain a prediction result of the conjunction, and there is a relationship formula in the corresponding process as follows:

[0051] ;

[0052] wherein, represents the prediction result of the conjunction, represents the final semantic representation of the conjunction placeholder, and both represent learnable parameters in the conjunction-aware prediction network;

[0053] The discourse relation class is derived based on the prediction result of the conjunction to obtain a probability distribution of the discourse relation class output by the conjunction-aware prediction network, and there is a relationship formula in the corresponding process as follows:

[0054] ;

[0055] wherein, represents the probability distribution of the discourse relation class output by the conjunction-aware prediction network, and both represent learnable parameters;

[0056] The final semantic representation of the global placeholder is predicted and derived by using the argument semantic direct prediction network to obtain a probability distribution of the discourse relation class output by the argument semantic direct prediction network, and there is a relationship formula in the corresponding process as follows:

[0057] ;

[0058] wherein, represents the probability distribution of the discourse relation class output by the argument semantic direct prediction network, represents the final semantic representation of the global placeholder, and both represent learnable parameters in the argument semantic direct prediction network;

[0059] The probability distribution of the discourse relation class output by the connecting word perception prediction network is adaptively fused with the probability distribution of the discourse relation class output by the argument semantic direct prediction network by using a gating mechanism to obtain a discourse relation recognition result. There are the following relationships in the corresponding process:

[0060] ;

[0061] wherein, represents a gating coefficient, represents a Sigmoid function, and both represent learnable parameters in the gating mechanism, represents a vector splicing operation, represents a discourse relation recognition result.

[0062] In this embodiment, the gating coefficient can be in scalar or vector form. The scalar indicates which prediction result is more trusted as a whole, and the vector indicates that different fusion weights are used for different relation types.

[0063] Step 5, pre-training the cross-domain implicit discourse relation recognition model using the multi-domain explicit discourse relation dataset, fine-tuning the implicit discourse relation dataset, and introducing a cross-entropy loss function constructed based on the discourse relation recognition result and a balanced loss function constructed based on the weights in the token-level hybrid expert network and the instance-level hybrid expert network, to obtain the trained cross-domain implicit discourse relation recognition model; the trained cross-domain implicit discourse relation recognition model is used to predict the category of the input instance.

[0064] In step 5, the cross-domain implicit discourse relation recognition model is pre-trained using the multi-domain explicit discourse relation dataset, fine-tuned using the implicit discourse relation dataset, and a cross-entropy loss function constructed based on the discourse relation recognition result and a balanced loss function constructed based on the weights in the token-level hybrid expert network and the instance-level hybrid expert network are introduced, to obtain the trained cross-domain implicit discourse relation recognition model, which specifically includes the following sub-steps:

[0065] The cross-domain implicit discourse relation recognition model is pre-trained using the multi-domain explicit discourse relation dataset, and a cross-entropy loss function constructed based on the discourse relation recognition result and a balanced loss function constructed based on the weights in the token-level hybrid expert network and the instance-level hybrid expert network are introduced, to obtain the pre-trained cross-domain implicit discourse relation recognition model. There are the following relationships in the corresponding process:

[0066] ;

[0067] wherein, represents a balanced loss function, This represents the average weight of each word-level expert network in the current training batch. Indicates the first The weights of each word-level expert in the current training batch. This represents the average weight of each instance-level expert network in the current training batch. Indicates the first The weights of each instance-level expert in the current training batch. This represents the total cost function during the pre-training phase. This represents a dataset of explicit textual relations across multiple domains. express The number of samples in the middle Indicates the training batch. This represents the training samples in the explicit discourse relation dataset. This represents the one-hot vector representation of the connectives in the training samples. This represents the one-hot vector representation of the discourse relation categories obtained by automatic mapping of conjunctions. This represents the calculation of the cross-entropy between two probability distributions. and Each represents a weighting coefficient for the corresponding cost;

[0068] The pre-trained cross-domain implicit discourse relation recognition model was fine-tuned using an implicit discourse relation dataset. A cross-entropy loss function based on discourse relation recognition results and an equilibrium loss function based on the weights in the word-level hybrid expert network and the instance-level hybrid expert network were introduced to obtain the trained cross-domain implicit discourse relation recognition model. The trained cross-domain implicit discourse relation recognition model is used to predict the category of the input instance.

[0069] It should be noted that during the pre-training phase, by minimizing The training phase of the pre-training stage is completed when the value of the total cost function in the fine-tuning stage converges. The fine-tuning stage is based on a manually labeled implicit discourse relation dataset. The definition of the total cost function in the fine-tuning stage is similar to that in the pre-training stage, but it does not include the cross-entropy cost for connector word classification, which will not be elaborated here. By minimizing the value of the total cost function in the fine-tuning stage until convergence, the training phase of the fine-tuning stage is completed, thus obtaining the trained cross-domain implicit discourse relation recognition model.

[0070] It should be understood that, while the steps of the flowcharts of the embodiments of the present application are shown in a sequence following the arrows, the steps are not necessarily executed in the order following the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in order, and the steps can be executed in other orders. Moreover, at least some of the steps in the embodiments can include a plurality of sub-steps or a plurality of stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the sub-steps or stages is not necessarily sequential, but can be round-robin or alternately executed with at least some of the other steps or sub-steps or stages of the other steps.

[0071] It should be understood that the various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, as in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0072] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0073] The above-described embodiments only express several implementation manners of the present application, which are described in a more specific and detailed manner, but cannot be understood as a limitation on the patent scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, which are all within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.< / mask> < / cls> < / sep> < / mask> < / sep> < / sep> < / cls>

Claims

1. A cross-domain implicit discourse relation identification method, characterized in that, The method includes the following steps: Step 1: Introduce a word-level hybrid expert network and an instance-level hybrid expert network into the Transformer layer to obtain a hybrid expert-enhanced Transformer layer. Construct a multi-granularity hybrid expert coding module based on the multi-layer hybrid expert-enhanced Transformer layer. Construct a connection word-aware prediction network and an argument semantic direct prediction network based on the Softmax function and linear transformation mechanism, respectively. Construct a dual-path prediction module based on the connection word-aware prediction network, the argument semantic direct prediction network, and a gating mechanism. Use the multi-granularity hybrid expert coding module and the dual-path prediction module to form a cross-domain implicit discourse relation recognition model. Step 2: Encode the implicit discourse relation instances using a pre-trained language model to obtain a basic semantic representation containing lexical units, global placeholders, and conjunction placeholders; Step 3: Use the multi-granularity hybrid expert coding module to perform multiple rounds of iterative enhancement processing on the basic semantic representation containing lexical units, global placeholders and connector placeholders, so as to obtain the final semantic representation of global placeholders and the final semantic representation of connector placeholders; Step 4: Process the final semantic representations of global placeholders and connector placeholders using the dual-path prediction module to obtain the discourse relation recognition results. This includes the following sub-steps: A connector-aware prediction network is used to predict and deduce the final semantic representation of connector placeholders in order to obtain the prediction results of connectors. The text relationship categories are derived based on the prediction results of the connective words, so as to obtain the probability distribution of the text relationship categories output by the connective word perception prediction network. The final semantic representation of global placeholders is predicted and derived using the argument semantic direct prediction network to obtain the probability distribution of the discourse relation categories output by the argument semantic direct prediction network. By using a gating mechanism, the probability distribution of the discourse relation categories output by the connective word perception prediction network and the probability distribution of the discourse relation categories output by the argument semantic direct prediction network are adaptively fused to obtain the discourse relation recognition result. Step 5: Pre-train the cross-domain implicit text relation recognition model using a multi-domain explicit text relation dataset, then fine-tune it using the implicit text relation dataset, and introduce a cross-entropy loss function based on the text relation recognition results and an equilibrium loss function based on the weights in the word-level hybrid expert network and the instance-level hybrid expert network to obtain the trained cross-domain implicit text relation recognition model; the trained cross-domain implicit text relation recognition model is used to predict the category of the input instance.

2. The cross-domain implicit discourse relation identification method according to claim 1, characterized in that, In step 2, implicit discourse relation instances are encoded using a pre-trained language model to obtain basic semantic representations. The following relation exists in the correspondence process: ； in, This represents an instance of implicit discourse relations. Argument 1 represents an instance of an implicit discourse relation. Argument 2 represents an instance of implicit discourse relation. The basic semantic representation of global placeholders. The basic semantic representation of the first lexical unit. Indicates the first The basic semantic representation of each word element Indices representing lexical elements, Indicates the first The basic semantic representation of each word element Indicates the number of lexical units. The basic semantic representation of conjunction placeholders. This indicates that the language has been encoded using a pre-trained language model. This represents a natural language prompt template.

3. The cross-domain implicit discourse relationship identification method according to claim 2, characterized in that, In step 3, a multi-granularity hybrid expert coding module is used to perform multiple rounds of iterative enhancement processing on the basic semantic representation containing lexical units, global placeholders, and connector placeholders to obtain the final semantic representation of the global placeholders and the final semantic representation of the connector placeholders. Specifically, this includes the following sub-steps: S301. Input the basic semantic representation of the word, the basic semantic representation of the global placeholder, and the basic semantic representation of the connector placeholder into the multi-head attention mechanism sub-layer of the first layer of the hybrid expert-enhanced Transformer layer for processing, so as to obtain the word-level semantic representation, the semantic representation of the global placeholder, and the semantic representation of the connector placeholder output by the multi-head attention mechanism sub-layer of the first layer of the hybrid expert-enhanced Transformer layer. S302. Input the lexical-level semantic representation output from the multi-head attention mechanism sublayer in the first layer of the hybrid expert-enhanced Transformer layer into the input... In a word-level hybrid expert network composed of word-level experts, the network dynamically calculates the weights of each word-level expert based on the input information and selects the expert with the highest weight. Each lexer-level expert participates in the computation to obtain the lexer-level semantic representation output by the lexer-level hybrid expert network; S303. The word-level semantic representation output by the word-level hybrid expert network is normally processed and connected to the residual in sequence to obtain the word-level semantic representation output by the first layer of the hybrid expert-enhanced Transformer layer. S304. Input the semantic representation of the global placeholders output from the multi-head attention mechanism sublayer in the first layer of the hybrid expert-enhanced Transformer layer into the input... In an instance-level hybrid expert network composed of instance-level experts, the network dynamically calculates the weights of each instance-level expert based on the input information and selects the expert with the largest weight. Each instance-level expert participates in the computation to obtain the semantic representation of the global placeholders output by the instance-level hybrid expert network; S305. The semantic representation of the global placeholders output by the instance-level hybrid expert network is normalized and connected to the residual in sequence to obtain the semantic representation of the global placeholders output by the first layer of the hybrid expert-enhanced Transformer layer. S306. Replace the semantic representation of the global placeholder output by the multi-head attention mechanism sub-layer in the first layer of the hybrid expert-enhanced Transformer layer with the semantic representation of the connector placeholder output by the multi-head attention mechanism sub-layer in the first layer of the hybrid expert-enhanced Transformer layer as input and repeat steps S304 to S305 to obtain the semantic representation of the connector placeholder output by the first layer of the hybrid expert-enhanced Transformer layer. The semantic representations of the word level, global placeholders, and connector placeholders output from the first layer of the hybrid expert-enhanced Transformer layer are used as inputs to the next layer in an iterative manner. Steps 301 to S306 are executed in each layer, and this iterative process is repeated. Next, to obtain the final semantic representation of the global placeholder and the final semantic representation of the conjunction placeholder; in, This indicates the number of layers in the Transformer layer with hybrid expert enhancement.

4. The cross-domain implicit discourse relation identification method according to claim 3, characterized in that, The lexical-level semantic representation output from the multi-head attention mechanism sublayer in the first-layer hybrid expert-enhanced Transformer layer is input to the... In a word-level hybrid expert network composed of word-level experts, the network dynamically calculates the weights of each word-level expert based on the input information and selects the expert with the highest weight. In the step of having multiple word-level experts participate in the computation to obtain the word-level semantic representation output by the word-level hybrid expert network, the following relationship exists: ； in, This represents the unnormalized weight vector in a word-level hybrid expert network. and Each represents a learnable parameter in a word-level hybrid expert network. This represents the word-level semantic representation of the output of the multi-head attention mechanism sublayer in the first layer of the hybrid expert-enhanced Transformer layer. and All indicate Medium-weighted indexes This indicates that the data has been processed by the TopK function. This indicates the number of lexical-level experts selected. and Both indicate selection The normalized weight values ​​corresponding to each word-level expert. and Both indicate selection The unnormalized weight values ​​corresponding to each word-level expert. This indicates that the data has been processed using the Softmax function. This represents the word-level semantic representation output by the word-level hybrid expert network. Indicates the first The normalized weight values ​​corresponding to each word-level expert. Indicates the selected first Meta-level expert; In the process of sequentially normalizing and connecting the word-level semantic representation output by the word-level hybrid expert network to obtain the word-level semantic representation output by the first layer of the hybrid expert-enhanced Transformer layer, the following relationship exists: ； in, This represents the word-level semantic representation output of the first layer of the Transformer layer, which is enhanced by hybrid experts. After normalization.

5. The cross-domain implicit discourse relation identification method according to claim 4, characterized in that, The semantic representation of the global placeholder output from the multi-head attention mechanism sublayer in the first-layer hybrid expert-enhanced Transformer layer is input to the... In an instance-level hybrid expert network composed of instance-level experts, the network dynamically calculates the weights of each instance-level expert based on the input information and selects the expert with the largest weight. In the step of having instance-level experts participate in the computation to obtain the semantic representation of the global placeholders output by the instance-level hybrid expert network, the following relationship exists: ； in, This represents the unnormalized weight vector in an instance-level hybrid expert network. and All of these represent learnable parameters in an instance-level hybrid expert network. This represents the semantic representation of the global placeholders output by the multi-head attention mechanism sublayer in the first-layer hybrid expert-enhanced Transformer layer. and All indicate Medium-weighted indexes This indicates the number of instance-level experts selected. and Both indicate selection The normalized weight values ​​corresponding to each instance-level expert and Both indicate selection The unnormalized weight values ​​corresponding to each instance-level expert The semantic representation of global placeholders in the output of instance-level hybrid expert networks. Indicates the first The normalized weight values ​​corresponding to each instance-level expert Indicates the selected first Instance-level experts; In the step of normalizing and concatenating the semantic representation of the global placeholders output by the instance-level hybrid expert network to obtain the semantic representation of the global placeholders output by the first-layer hybrid expert-enhanced Transformer layer, the following relationship exists: ； in, This represents the semantic representation of the global placeholders in the output of the Transformer layer with hybrid expert enhancement at layer 1.

6. The cross-domain implicit discourse relation identification method according to claim 5, characterized in that, The semantic representations of the word level, global placeholders, and connector placeholders output from the first layer of the hybrid expert-enhanced Transformer layer are used as inputs to the next layer in an iterative manner. Steps 301 to S306 are executed in each layer, and this iterative process is repeated. In the steps of obtaining the final semantic representation of the global placeholder and the final semantic representation of the connector placeholder, the following relationship exists: ； in, Indicates the first The semantic representation of global placeholders in the output of the Transformer layer enhanced by layered hybrid experts. Indicates the first The semantic representation corresponding to the first lexical unit output by the layer hybrid expert-enhanced Transformer layer. Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first Semantic representation of connective placeholders in the output of the Transformer layer enhanced by layered hybrid expert. Indicates the first Layered hybrid expert-enhanced Transformer layers, This represents the index of the Transformer layer enhanced by hybrid experts, and ; Indicates the first The semantic representation of global placeholders in the output of the Transformer layer enhanced by layered hybrid experts. Indicates the first The semantic representation corresponding to the first lexical unit output by the layer hybrid expert-enhanced Transformer layer. Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first The output of the layer hybrid expert-enhanced Transformer layer is the first The semantic representation of each word element Indicates the first Semantic representation of connector placeholders in the output of the Transformer layer enhanced by layer hybrid expert.

7. The cross-domain implicit discourse relation identification method according to claim 6, characterized in that, In the step of using a connector-aware prediction network to predict and derive the final semantic representation of connector placeholders to obtain the prediction results of connectors, the following relationship exists: ； in, Indicates the prediction result for the conjunction. The final semantic representation of the conjunction placeholder. and All of these represent learnable parameters in the connective word awareness prediction network; In the step of deriving the text relation categories based on the prediction results of connectives to obtain the probability distribution of the text relation categories output by the connective-aware prediction network, the following relationship exists: ； in, This represents the probability distribution of the discourse relation categories output by the connective word perception prediction network. and All represent learnable parameters; In the step of using the argument semantic direct prediction network to predict and derive the final semantic representation of global placeholders, so as to obtain the probability distribution of the discourse relation categories output by the argument semantic direct prediction network, the following relationship exists: ； in, This represents the probability distribution of argument semantics directly predicting the discourse relation categories in the network output. The final semantic representation of the global placeholder. and Both indicate that argument semantics directly predict learnable parameters in the network; In the step of adaptively fusing the probability distribution of the discourse relation categories output by the connector word perception prediction network and the probability distribution of the discourse relation categories output by the argument semantic direct prediction network using a gating mechanism to obtain the discourse relation recognition result, the following relationship exists: ； in, Indicates the gating coefficient. This represents the Sigmoid function. and All of these represent learnable parameters in the gating mechanism. This represents a vector concatenation operation. This indicates the results of the text relationship identification.

8. The cross-domain implicit discourse relation identification method according to claim 7, characterized in that, In step 5, the cross-domain implicit text relation recognition model is pre-trained using a multi-domain explicit text relation dataset, then fine-tuned using the implicit text relation dataset. A cross-entropy loss function based on the text relation recognition results and an equilibrium loss function based on the weights in the word-level hybrid expert network and the instance-level hybrid expert network are introduced to obtain the trained cross-domain implicit text relation recognition model. Specifically, this includes the following sub-steps: A cross-domain implicit text relation recognition model is pre-trained using a multi-domain explicit text relation dataset. A cross-entropy loss function based on the text relation recognition results and an equilibrium loss function based on the weights in the word-level hybrid expert network and the instance-level hybrid expert network are introduced to obtain the pre-trained cross-domain implicit text relation recognition model. We fine-tuned the pre-trained cross-domain implicit discourse relation recognition model using an implicit discourse relation dataset, and introduced a cross-entropy loss function based on discourse relation recognition results and an equilibrium loss function based on the weights in the word-level hybrid expert network and the instance-level hybrid expert network to obtain the trained cross-domain implicit discourse relation recognition model.

Images