An English-Chinese entity relation extraction method, system and device

Abstract

The application discloses a kind of Chinese-English entity relation extraction method, system and equipment, then entity extraction result is combined with prompt fine-tuning template input Chinese-English entity relation extraction network, and English entity relation extraction is carried out;The large-scale language model used in the application uses supervised training, and can fully adapt to large-scale data set.The prompt template designed in the application is different from the existing prompt template design concept, because the application pays more attention to the emphasis of model on entity information, and most of the existing prompt templates only focus on single task, i.e.the prompt template of the application is still feasible for single language information extraction.At the same time, in the data preprocessing of training model, bilingual word embedding mapping is not needed, which greatly reduces the mapping error caused by different grammar in different languages, and solves the problem of low training data quality.The method of the application has the characteristics of strong universality, low cost and easy updating.

Description

Technical Field

[0001] This invention belongs to the fields of computer science, natural language processing and machine learning technology, and relates to a method, system and device for extracting entity relations in Chinese and English texts. Specifically, it relates to a supervised method, system and device for extracting entity relations in Chinese and English texts based on deep learning and prompting fine-tuning. Background Technology

[0002] Relation extraction is one of the fundamental tasks of information extraction, aiming to identify target relationships between entities in text. Relation extraction is crucial for knowledge base construction and text understanding, particularly beneficial for natural language processing tasks such as question answering and text understanding. Furthermore, as a core component of knowledge graph construction, relation extraction is an indispensable step. Relation extraction models originate from feature-based statistical models. Recent research has focused on monolingual entity relation extraction. However, with the rapid development of information technology and the increasing number of channels for accessing information in diverse languages, it is essential to extract information from source language texts and convert it into a target language that readers can understand. Cross-lingual relation extraction is a vital task in information extraction, enabling rapid understanding of relationships between different entities in texts of different languages, thus accelerating text comprehension. It also benefits many web applications, such as information retrieval, web question answering, and data mining.

[0003] However, when models are extended to more languages, obtaining entity relation annotations becomes difficult. This has led most current cross-lingual entity relation extraction methods to focus on zero-shot learning, which trains the model in one language and then tests it in another. For example, Wasi Uddin Ahmad and Nanyun Peng et al. used general dependency parsing of sentences to add dependency information to a graph convolutional network, achieving good performance on the ACE05 dataset for cross-lingual entity relation extraction. This demonstrates the effectiveness of cross-lingual entity relation extraction, but it can be cumbersome in data processing and faces high overhead during model training.

[0004] Besides the methods mentioned above that utilize general dependency information of sentences and graph convolutional neural networks, Jian Ni and Radu Florian proposed a bilingual word embedding mapping method for cross-lingual entity relation extraction. This method maps source language word embeddings to target language word embeddings, allowing models trained in the source language to be easily applied to the target language. However, this method is not suitable for high-resource scenarios and is only applicable to low-resource scenarios. Furthermore, the grammar and vocabulary of each country differ significantly, making accurate and consistent mapping of words from different countries an extremely difficult task.

[0005] Currently, increasingly complex algorithms have been designed for cross-lingual relation extraction, ranging from combinations of different neural network models (CNN, LSTM, Bi-LSTM) to combinations of different language models. Zero-shot learning is unsupervised learning, while relation extraction is a classification task; supervised learning is better suited for classification. The problems stem from the lack of suitable supervised training datasets for cross-lingual relation extraction models. Unsupervised learning may be effective on small datasets, but it doesn't perform well on large datasets. In summary, the main problems with cross-lingual relation extraction are: 1. The reliance on zero-shot learning makes supervised training and testing difficult, hindering its ability to adapt to large-scale data. 2. The lack of suitable supervised datasets for training and testing. 3. The cumbersome preprocessing of sentences and feature extraction for each language significantly increase time costs. Therefore, to address these issues, we need new algorithms and technologies. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a supervised method, system, and device for extracting entity relationships from Chinese and English texts based on deep learning and prompting fine-tuning.

[0007] The technical solution adopted by the method of the present invention is: a method for extracting entity relations from Chinese and English texts, comprising the following steps:

[0008] Step 1: Preprocess the user's input sentence;

[0009] Step 2: Extract entities from the preprocessed sentences in Step 1;

[0010] Step 3: Combine the results of Step 2 with the prompts to fine-tune the template and input it into the Chinese and English text entity relation extraction network to perform Chinese and English text entity relation extraction;

[0011] The English-Chinese entity relation extraction network includes a SentencePiece processing layer, an encoding layer, a decoding layer, and a linear classification layer. The SentencePiece processing layer generates a numerical vector; the numerical vector generated from the English text is input into the encoding layer to generate an encoded vector; the numerical vector generated from the Chinese text and the encoded vector generated by the encoding layer are then input into the decoding layer to obtain the final relation output type; alternatively, the numerical vector generated from the English text is input into the decoding layer to generate an encoded vector; the numerical vector generated from the Chinese text and the encoded vector generated by the decoding layer are then input into the encoding layer to obtain the final relation output type. The linear classification layer, composed of a Linear layer and a Softmax layer, is used to classify entity relations in the sentence.

[0012] The prompt for fine-tuning includes:

[0013] Template 1: Input to the encoding layer: <s>The sentence: " sentence(English) ” include entity1 (English arts) and entity2 (English) < / s> ;

[0014] Decoding layer input: <s>This sentence Sentence Examples (Chinese) "middle Entity 1 (Chinese) and Entity 2 (Chinese) What is the relationship between them?< / s> ;

[0015] Template 2: Input for the encoding layer: <s>What is the type of relationship between entity1 (English) and entity2 (English) ?< / s> ;

[0016] Decoding layer input: <s>This sentence Sentence Examples (Chinese) "Including these two entities" Entity 1 (Chinese) "and" Reality Body 2 (Chinese) "< / s> ;

[0017] Template 3: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ;

[0018] Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0019] Template 4: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ;

[0020] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0021] Template 5: Input for the encoding layer: <s>The sentence: " sentence(English) ” includes entity1 (English arts) [Seg_ment_ation] entity2 (English) < / s> ;

[0022] Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0023] Template 6: Input for the encoding layer: <s>The sentence of sentence(English) includeentity1 (English) [Seg_ment_ation] entity2 (English) < / s> ;

[0024] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0025] Among them, [Seg_ment_ation] is the segmentation identifier, used to separate sentence instances from entities; <s> For sentence start markers,< / s>The template is for sentence end markers, entity1 is the input English entity 1, entity2 is the input English entity 2, entity1 is the input Chinese entity 1, entity2 is the input Chinese entity 2, sentence instance (Chinese) is the entire sentence containing Chinese entities, and sentence (English) is the entire sentence containing English entities; the underlined parts in the template are the input data, and the ununderlined parts are the prompt templates, (Chinese) indicates that the input is Chinese, and (English) indicates that the input is English.

[0026] The technical solution adopted by the system of the present invention is: a Chinese and English text entity relation extraction system, comprising the following modules:

[0027] Module 1 is used to preprocess the user's input sentence;

[0028] Module 2 is used to extract entities from the preprocessed sentences in Module 1;

[0029] Module 3 is used to combine the results of Module 2 with the prompts and fine-tuning templates and input them into the Chinese and English text entity relation extraction network to extract Chinese and English text entities relations.

[0030] The English-Chinese entity relation extraction network includes a SentencePiece processing layer, an encoding layer, a decoding layer, and a linear classification layer. The SentencePiece processing layer generates a numerical vector; the numerical vector generated from the English text is input into the encoding layer to generate an encoded vector; the numerical vector generated from the Chinese text and the encoded vector generated by the encoding layer are then input into the decoding layer to obtain the final relation output type; alternatively, the numerical vector generated from the English text is input into the decoding layer to generate an encoded vector; the numerical vector generated from the Chinese text and the encoded vector generated by the decoding layer are then input into the encoding layer to obtain the final relation output type. The linear classification layer, composed of a Linear layer and a Softmax layer, is used to classify entity relations in the sentence.

[0031] The prompt for fine-tuning includes:

[0032] Template 1: Input to the encoding layer: <s>The sentence: " sentence(English) ” include entity1 (English arts) and entity2 (English) < / s> ;

[0033] Decoding layer input: <s>This sentence Sentence Examples (Chinese) "middle Entity 1 (Chinese) and Entity 2 (Chinese) What is the relationship between them?< / s> ;

[0034] Template 2: Input for the encoding layer: <s>What is the type of relationship between entity1 (English) and entity2 (English) ?< / s> ;

[0035] Decoding layer input: <s>This sentence Sentence Examples (Chinese) "Including these two entities" Entity 1 (Chinese) "and" Reality Body 2 (Chinese) "< / s> ;

[0036] Template 3: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ;

[0037] Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0038] Template 4: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ;

[0039] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0040] Template 5: Input at the encoding layer: The sentence: “ sentence(English) "includes" entity1 (in English) [Segmentation] entity2 (English) ;

[0041] Decoding layer input: Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) ;

[0042] Template 6: Input for the encoding layer: <s>The sentence of sentence(English) includes entity1 (English arts) [Seg_ment_ation] entity2 (English) < / s> ;

[0043] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0044] Where [Seg_ment_ation] is the segmentation identifier, used to separate sentence instances from entities; <s> For sentence start markers,< / s> The template is defined as follows: entity1 is the input English entity 1, entity2 is the input English entity 2, entity1 is the input Chinese entity 1, entity2 is the input Chinese entity 2, sentence instance (Chinese) is the entire sentence containing Chinese entities, and sentence (English) is the entire sentence containing English entities. The underlined parts in the template represent the input data, while the ununderlined parts represent the prompt template. (Chinese) indicates that the input is in Chinese, and (English) indicates that the input is in English.

[0045] The technical solution adopted by the device of the present invention is: a device for extracting entity relationships in Chinese and English texts, comprising:

[0046] One or more processors;

[0047] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the Chinese and English text entity relation extraction method.

[0048] The core innovations of this invention include:

[0049] (1) Design of a dataset for entity relation extraction from Chinese and English parallel corpora and data preprocessing techniques.

[0050] This includes subordinate innovation point 1.1: screening of parallel corpus datasets.

[0051] Since there is currently no relation extraction dataset based on parallel corpora worldwide, and existing entity relation extraction datasets are limited to monolingual languages ​​and cannot be well applied to cross-lingual entity relation extraction, this embodiment extracts some data from the WMT17 English-Chinese parallel corpus dataset for processing. To avoid excessively long sentences that could reduce the accuracy of data processing, this embodiment filters the sentences, selecting only Chinese sentences with no more than 70 characters for processing. To further improve accuracy, if a filtered sentence contains a period, it is split into two sentences. This embodiment then summarizes the sentences filtered using the above method for further processing.

[0052] This includes subordinate innovation point 1.2: creation of a dataset for extracting entity relations from parallel corpora in English and Chinese texts.

[0053] The dataset selected in Subordinate Innovation Point 1.1 only contains synonymous Chinese and English sentences, without labeling the relationships between entities within the sentences. Therefore, this embodiment uses the PredPatt predicate-parameter extraction model to extract entities and connecting predicates from the sentences. For example, in the sentence "Xiaoming manufactured a boat," the extracted entities are "Xiaoming" and "a boat," and the connecting predicate is "manufactured." The predicate "manufactured" effectively represents the "creator and created" relationship between "Xiaoming" and "a boat." This embodiment categorizes sentences by extracting predicates, selecting sentences with over 50,000 predicates in each category. Then, 12 relationship labels are summarized from the categorized predicates, and sentences containing these 12 predicates are selected. Thus, this embodiment obtains a parallel corpus dataset of 941,000 sentences with entities and entity relationship labels. It is also the first Chinese-English entity relationship extraction dataset based on the WMT17 Chinese-English parallel corpus, and it is noteworthy that this is also the first large-scale dataset used for cross-language relationship extraction.

[0054] Including subordinate innovation point 1.3: data preprocessing technology. When this embodiment needs to use the model to predict the relationship type, this embodiment can use the method in subordinate innovation point 1.1 to segment the data into sentences, and then use the method in subordinate innovation point 1.2 to extract entities. The preprocessed data is then input into the model for relationship prediction.

[0055] (2): Create a supervised model for extracting entity relations from Chinese and English texts.

[0056] This includes subordinate innovation point 2.1: design of a model for extracting entity relationships between Chinese and English texts.

[0057] This embodiment designs a cross-language large-scale model for extracting entity relationships between Chinese and English text. This embodiment is implemented using an end-to-end structure, which includes an encoder layer and a decoder layer. The method first processes all preprocessed Chinese and English data using a SentencePiece model to generate numeric vectors. Then, the numeric vectors generated from English are input into the encoder layer, and the numeric vectors generated from Chinese are input into the decoder layer for training. When the input to the encoder layer is Chinese, the input to the decoder layer is English, thus obtaining a relationship classification model in this embodiment.

[0058] Subordinate innovation point 2.2: Supervised training of the Chinese and English entity relationship extraction model.

[0059] Current cross-language entity relation extraction methods lack supervised training due to the absence of parallel corpus relation extraction datasets, resulting in mostly unsupervised fine-tuning. However, by utilizing the dataset constructed using the core innovation point 1 of this embodiment, supervised training of the Chinese-English entity relation extraction model designed in this embodiment can be performed. This embodiment uses the relation labels from the parallel corpus entity relation extraction dataset created using core innovation point 1 as the prediction target. During each batch of training, the model calculates a loss function to optimize the prediction accuracy for the next iteration, and this iterative operation is repeated until the model is fully trained. The trained model is then tested.

[0060] (3): Hints on fine-tuning the template design.

[0061] Subordinate innovation point 3.1: Three prompt template designs.

[0062] Hard-prompt design involves using specific Chinese or English words as prompts. These are human-readable prompts, requiring design and creation based on experience. By combining source data with these prompts as training data and inputting them into the model, the language model's sentence comprehension ability is leveraged to improve training performance.

[0063] This embodiment designs the following hard-signal template:

[0064] Template 1. Encoding Layer Input: <s> The sentence: "sentence (English)" includeentity1 (English) and entity2 (English)< / s> ;

[0065] Decoding layer input: <s> What is the relationship between entity 1 (Chinese) and entity 2 (Chinese) in this sentence example (Chinese)?< / s> ;

[0066] Template 2. Input for the encoding layer: <s> What is the type of relationship between entity1 (English) and entity2 (English)?< / s> ;

[0067] Decoding layer input: <s> This sentence example (in Chinese) includes two entities: "Entity 1 (in Chinese)" and "Entity 2 (in Chinese)".< / s>

[0068] Soft-prompt template design involves automatically optimizing the model's vector space. These prompts can directly provide feedback within the model's embedding space. Therefore, the template vocabulary is no longer limited to human-readable language and may include labeled symbols or task-specific vectors. This embodiment designs specific symbols to insert into the original data, labeling different entities to create soft prompts that are then input into the model. During training, the model automatically optimizes parameters based on these prompts to improve training effectiveness. However, the design of soft prompts also requires human experience and creativity.

[0069] This embodiment designs the following soft prompt template:

[0070] Template 3. Encoding Layer Input: <s> sentence (English) [Seg_ment_ation] entity1 (English) [Seg_ment_ation] entity2 (English)< / s> ;

[0071] Decoding layer input: <s> Sentence Example (Chinese) [Seg_ment_ation] Entity 1 (Chinese) [Seg_ment_ation] Entity 2 (Chinese)< / s> ;

[0072] Template 4. Encoding Layer Input: <s>sentence (English) [Seg_ment_ation] entity1 (English) [Seg_ment_ation] entity2 (English)< / s> ;

[0073] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese)< / s>

[0074] [Seg_ment_ation] is a segmentation identifier used to separate sentence instances from entities.

[0075] The hybrid hard and soft cue template fine-tuning combines the original data with hard cue templates, inserting special markers between different entities in different input data. This improves the language model's ability to understand sentences and allows the language model to automatically optimize parameters in the vector space, thereby enhancing the model training effect.

[0076] This embodiment designs the following soft and hard prompt hybrid fine-tuning template:

[0077] Template 5. Input at the encoding layer: The sentence: “sentence (English)” include entity1 (English) [Segmentation] entity2 (English);

[0078] Decoding layer input: Sentence instance (Chinese) [Seg_ment_ation] Entity 1 (Chinese) [Seg_ment_ation] Entity 2 (Chinese);

[0079] Template 6. Encoding Layer Input: <s> The sentence of sentence (English) includeentity1 (English) [Seg_ment_ation] entity2 (English)< / s> ;

[0080] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese)< / s> .

[0081] Subordinate innovation point 3.2: Provide prompts for fine-tuning the entire model.

[0082] Currently, most prompts for fine-tuning are applied to downstream tasks after model training is complete. However, this embodiment aims to perform prompts for fine-tuning during model training. Therefore, the prompt template in this embodiment is not only applied to downstream tasks, but also used to fine-tune the model during training, with the prompt template being input into the model as training data.

[0083] Compared to existing technologies: This invention utilizes a large-scale language model trained under supervised conditions, making it fully adaptable to large-scale datasets. The prompt template designed in this invention differs from existing prompt templates in its design philosophy, as this invention emphasizes the model's focus on entity information, while most existing prompt templates only focus on a single task. In other words, the prompt template of this invention is still feasible for monolingual information extraction. Furthermore, in terms of data preprocessing for training the model, bilingual word embedding mapping is not required, significantly reducing mapping errors caused by grammatical differences between languages ​​and solving the problem of low training data quality. The method of this invention is characterized by strong versatility, low cost, and ease of updating. Attached Figure Description

[0084] Figure 1 This is a block diagram illustrating the principle of the method according to an embodiment of the present invention;

[0085] Figure 2 This is a diagram of the network structure for extracting entity relationships in both Chinese and English texts according to an embodiment of the present invention.

[0086] Figure 3 This is a diagram of the coding layer structure according to an embodiment of the present invention;

[0087] Figure 4 This is a diagram of the decoding layer structure according to an embodiment of the present invention;

[0088] Figure 5 This is a block diagram illustrating the principle of data preprocessing and dataset construction in an embodiment of the present invention.

[0089] Figure 6 This is a block diagram illustrating the training principle of the Chinese and English text entity relation extraction network according to an embodiment of the present invention.

[0090] Figure 7 This is a schematic diagram showing the comparison of the F1 value between the method of this invention and different models. Detailed Implementation

[0091] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0092] With the rapid development of information technology, people have access to an increasing number of resources and information, resulting in a wide variety of languages. To help people quickly and clearly extract entity information from texts in different languages, it is essential to extract information from the source language text and convert it into a target language that readers can understand. Cross-language relation extraction is a crucial task in information extraction. It helps people quickly identify the relationships between different entities in texts of different languages, thus accelerating the understanding of text content. It also benefits many web applications, such as information retrieval, web question answering, and data mining.

[0093] This embodiment proposes a method combining a large-scale pre-trained language model with a designed general prompt-tuning template to improve the effectiveness of cross-linguistic relation extraction. Currently, there is no cross-linguistic relation dataset based on parallel corpora. Therefore, this embodiment designs a method based on general language dependency analysis to improve existing parallel corpora and create a cross-linguistic relation dataset based on parallel corpora.

[0094] Please see Figure 1 The present invention provides a method for extracting entity relations from Chinese and English texts, comprising the following steps:

[0095] Step 1: Preprocess the user's input sentence;

[0096] In this embodiment, sentences whose length exceeds a preset value are first discarded; then the sentences are segmented. If a sentence contains a period and there is still a sentence after the period, the sentence is divided into multiple smaller sentences using the period as the cutting point.

[0097] Step 2: Extract entities from the preprocessed sentences in Step 1;

[0098] This embodiment uses the PredPatt predicate-parameter extraction model to identify the predicate entity structure in a sentence;

[0099] The data extracted using existing technologies cannot be used directly because the extracted entities and predicates are still quite ambiguous. This embodiment needs to define the extracted linear form. This embodiment requires traversing the entity and predicate nodes extracted by PredPatt to obtain a new linear form, and then labeling the words of each node: p represents the predicate label, a represents the entity label, ph represents the predicate header label, and ah represents the entity header label. This embodiment inserts parentheses at the beginning and end of entities, and square brackets at the beginning and end of predicates. This embodiment provides an example for reference: [(Chris:ah) wants:ph [(Chris:ah) build:ph (a:aboat:ah)]]. To recover the predicate-parameter structure, this embodiment only needs to recursively construct it from the outermost brackets. At each level of this linear structure, parentheses help recover the parameter nodes. The labels ah and ph help identify the predicate and parameter header labels, respectively. This embodiment defines that if an automatically generated linearized PredPatt has mismatched parentheses or curly braces, or if a predicate (or parameter) has zero or more header tags, then it is malformed and cannot be used.

[0100] Step 3: Combine the results of Step 2 with the prompts to fine-tune the template and input it into the Chinese and English text entity relation extraction network to perform Chinese and English text entity relation extraction;

[0101] Please see Figure 2 The entity relation extraction network for Chinese and English texts in this embodiment includes a SentencePiece processing layer, an encoding layer, a decoding layer, and a linear classification layer. The SentencePiece processing layer generates a numerical vector; the numerical vector generated from English text is input into the encoding layer to generate an encoded vector; the numerical vector generated from Chinese text and the encoded vector generated by the encoding layer are input into the decoding layer to obtain the final relation output type; or the numerical vector generated from English text is input into the decoding layer to generate an encoded vector; the numerical vector generated from Chinese text and the encoded vector generated by the decoding layer are input into the encoding layer to obtain the final relation output type. The linear classification layer, composed of a Linear layer and a Softmax layer, is used to classify entity relations in the sentence.

[0102] In this embodiment, the encoding layer consists of 12 identical transformer encoders stacked together, and the decoding layer also consists of 12 identical transformer decoders stacked together, with each transformer decoder having a cross-attention module. A single transformer encoder includes a self-attention mechanism module, a first residual block, a feedforward layer, and a second residual block; a single transformer decoder includes a masked self-attention module, a third residual block, a second self-attention mechanism module, a fourth residual block, a second feedforward layer, and a fifth residual block; the linear classification layer includes a linear layer and a softmax layer.

[0103] Please see Figure 3 In the coding layer of this embodiment, in the model of a single transformer encoder, the input data processed by the self-attention mechanism module and the SentencePiece processing layer are connected to the first residual block, and the input data is also connected to the self-attention mechanism module; the feedforward layer is connected to the first residual block and the second residual block, and the first residual block is connected to the second residual block; thus, a single transformer encoder is formed, and each transformer encoder connected and stacked together constitutes the coding layer of the network.

[0104] Please see Figure 4 In this embodiment, within a single transformer decoder in the decoding layer, the input data processed by the Masked Self-Attention module and the SentencePiece processing layer are both connected to the third residual block. The input data processed by the SentencePiece processing layer is also connected to the Masked Self-Attention module. The final output data of the encoding layer is connected to the third residual block and the second self-attention mechanism module. Both the third residual block and the second self-attention mechanism module are connected to the fourth residual block. The fourth residual block is connected to the second feedforward layer, and both of the above are connected to the fifth residual block. This constitutes a single transformer decoder, and each transformer decoder is connected to a Cross-Attention module, which is also connected to the encoder. This results in the decoding layer of the network.

[0105] The prompt for fine-tuning in this embodiment includes:

[0106] Template 1: Input to the encoding layer: <s>The sentence: " sentence(English) ” include entity1 (English arts) and entity2 (English) < / s> ;

[0107] Decoding layer input: <s>This sentence Sentence Examples (Chinese) "middle Entity 1 (Chinese) and Entity 2 (Chinese) What is the relationship between them?< / s> ;

[0108] Template 2: Input for the encoding layer:<s>What is the type of relationship between entity1 (English) and entity2 (English) ?< / s> ;

[0109] Decoding layer input: <s>This sentence Sentence Examples (Chinese) "Including these two entities" Entity 1 (Chinese) "and" Reality Body 2 (Chinese) "< / s> ;

[0110] Template 3: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ;

[0111] Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0112] Template 4: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ;

[0113] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0114] Template 5: Input for the encoding layer: <s>The sentence: " sentence(English) ” includes entity1 (English arts) [Seg_ment_ation] entity2 (English) < / s> ;

[0115] Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0116] Template 6: Input for the encoding layer: <s>The sentence of sentence(English) includeentity1 (English) [Seg_ment_ation] entity2 (English) < / s> ;

[0117] Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ;

[0118] Among them, [Seg_ment_ation] is the segmentation identifier, used to separate sentence instances from entities; <s> For sentence start markers,< / s> The template is for sentence end markers, entity1 is the input English entity 1, entity2 is the input English entity 2, entity1 is the input Chinese entity 1, entity2 is the input Chinese entity 2, sentence instance (Chinese) is the entire sentence containing Chinese entities, and sentence (English) is the entire sentence containing English entities; the underlined parts in the template are the input data, and the ununderlined parts are the prompt templates, (Chinese) indicates that the input is Chinese, and (English) indicates that the input is English.

[0119] In this embodiment, the encoded output is input to the decoding layer, and the final output is the final relation type.

[0120] For cross-language extraction tasks, the task is defined as follows: ,in It is a language pair for input that has already had prompt templates added. It is the output relation type, for each input instance. , For entities, 1≤ i ≤ n The two entities used to predict entity relationships are denoted as follows: and ,in Main entity, For object entities, the cross-language relation extraction task is to predict the main entity. and target entity Relationship ; Let M be the entity relation extraction network in both Chinese and English texts. Add " before and after" <s> "and"< / s> "get" <s> < / s> "The input is the encoding layer, which has a self-attention mechanism. The module first calculates the query vector of the word." (Query) (Key) and weighted vector (Value):

[0121] (Query)= ;

[0122] (Key)= ;

[0123] (Value) = ;

[0124] in , , Given a random matrix, the final output of the self-attention mechanism module is obtained:

[0125] ;

[0126] in Let be the dimension of the matrix; () indicates that the normalized exponential function takes the value (0, 1);

[0127] Will Perform residual connection to obtain new = LayerNorm ( );

[0128] Use two layers of linear mapping and activate using the GeLU activation function:

[0129] ;

[0130] The two Linear() functions represent two linear mappings; the above formula can also be written in the following form:

[0131] ;

[0132] in" "for" A randomly generated matrix, "for" A randomly generated matrix, "and" "Different bias values ​​initialized randomly"

[0133] Perform residual connection ,get:

[0134] = LayerNorm ( )= ;

[0135] The above process goes through 12 layers to obtain the final result. ; after that, “ <s> < / s> "and Input decoding layer ;

[0136] The decoding layer has a Masked Self-Attention module before the self-attention mechanism module of the encoding layer, and each layer of the decoding layer performs a Cross-Attention operation on the final output of the encoding layer; the Masked Self-Attention module will... <s> < / s> The result is obtained by regenerating the input into the Masked Self-Attention module:

[0137] (Query)= ;

[0138] (Key)= ;

[0139] Among them (Query) and The (Key) dot product matrix transforms some values ​​into 0 or negative infinity, making the matrix a lower triangular matrix. Then, each layer of the matrix is ​​normalized using the Softmax function to obtain the values ​​required in the decoding layer's self-attention module. The Q and K matrices in the self-attention mechanism module of the decoding layer are the final outputs of the encoding layer. The generation process involves using the V matrix as the output of the Masked Self-Attention module. Subsequent operations are the same as in the encoding layer, but after each layer in the decoding layer outputs its output, the output of the encoding layer is again processed. Perform a Cross-Attention operation; if the output of a certain layer in the decoding layer is , i For the first layer inside the decoding layer iIf there are multiple layers, the following sequence is obtained:

[0140] ;

[0141] In the above formula, " "for" "a random matrix", " "for" "random matrix" "for" The purpose of these random matrices is to generate " ". This then becomes the input to the next layer in the decoding layer. The Cross-Attention operation is repeated through 12 layers until the final result is obtained. ,after The predicted relation type is obtained after passing through the Linear layer and Softmax layer. Probability:

[0142]

[0143] in, W This represents the random initialization matrix that needs to be optimized, and b is a random bias value.

[0144] Please see Figure 5 The entity relation extraction network for Chinese and English texts in this embodiment is a pre-trained entity relation extraction network for Chinese and English texts; its training process includes the following steps:

[0145] (1) Construct a dataset for entity relation extraction from parallel Chinese and English corpora;

[0146] This embodiment uses the WMT17 English-Chinese parallel corpus dataset obtained from the internet for preprocessing. First, long sentences in the corpus are segmented. If a sentence contains a period followed by another sentence, this embodiment will divide it into multiple smaller sentences using the period as the cutting point. Sentences exceeding 70 characters are filtered out. The sentences retained using the above method are further processed. The PredPatt tool is used to obtain the entities and predicates from the retained sentences. This embodiment extracts predicates and classifies the sentences, selecting predicates with more than 50,000 sentences in each predicate category. Then, 12 relation labels are summarized from the classified predicates, covering most relationships in contemporary society. Thus, this embodiment obtains the labels for the relation types of sentences in each parallel corpus. In this embodiment, sentences, entities within sentences, and relation tags are encapsulated, and an average of 941,000 data points are selected from parallel corpora of 12 relation types as the entity relation extraction dataset for this embodiment. The training set, test set, and validation set sizes are 888,000, 21,200, and 31,800, respectively. It is worth noting that this is already a large-scale dataset.

[0147] (2) Combine the data results in the training set with the prompts to fine-tune the template, generate a digital vector through the SentencePiece processing layer, input the digital vector into the Chinese and English entity relationship extraction network for training, and turn a machine translation model into a relationship classification model.

[0148] Please see Figure 6 This embodiment designs a cross-language large-scale model for extracting entity relations between Chinese and English, using the mBART model, an end-to-end structure for machine translation, as an example. Because this embodiment considers translation tasks to be cross-language tasks, and machine translation models perform well on parallel corpus datasets, they are well-suited to the dataset designed in this embodiment. The end-to-end structure includes an encoder and a decoder. The method first processes all preprocessed Chinese and English data using a SentencePiece model to generate numeric vectors. Then, the numeric vectors generated from English are input into the encoder layer, and the numeric vectors generated from Chinese are input into the decoder layer for training. When the input to the encoder layer is Chinese, the input to the decoder layer is English; thus, this embodiment transforms a machine translation model into a relation classification model. For a language... Examples The model acquires monolingual knowledge during the pre-training phase by optimizing the following objectives:

[0149] ;

[0150] in This indicates the language set included in CC-25, which is a built-in language set of the mBART model and includes 25 different languages, used in the pre-training phase. This includes noise functions for sentence concealment and sentence ordering. When applying the above model to cross-language tasks, this embodiment provides a sentence pair. When inputting sentence pairs, language identifiers need to be added to the input sentences. For example, when the encoding layer inputs Chinese, the language identifier "[ZH]" needs to be added to the end of the sentence, and when the decoding layer inputs English, the language identifier "[EN]" needs to be added to the beginning of the sentence. For cross-language extraction tasks, this embodiment defines this task as... in It is a language pair for input that has already had prompt templates added. The output relation type is for each input instance. Since the above example contains many entities, there are also many entity tags. However, the tags used to predict entity relationships are mainly two of them. and These two tags are used to identify entities, among which Main entity, For the object entity, the remaining entity tags are irrelevant to the predicted relation, because the cross-language relation extraction task is to predict the main entity. and target entity Relationship This embodiment assumes that the cross-language relation extraction model is M. This embodiment will... Add " before and after" <s> "and"< / s> "get" <s> < / s> "The input to the encoding layer, and the output of the encoding layer, is the sequence of hidden vectors." Then “ <s> < / s> "and Input coding layer The following formula represents the predicted relation type. Conditional probability:

[0151] ;

[0152] W is a random matrix, b is a bias value, and in the next fine-tuning stage, this embodiment also needs to use a prompt template to improve the task effect. In this embodiment, the prompt template is assumed to be... . The template will represent additional input words, word count, and word position. A prompt mapped to an input Since model M has been trained, it can predict the relationship type. Therefore, this embodiment yields the following formula for predicting the relationship type:

[0153] ;

[0154] The dataset used in this embodiment includes a training set of 888,000 Chinese-English parallel texts, a validation set of 31,800 Chinese-English parallel texts, a test set of 21,200 Chinese-English parallel texts, and 12 types of relation labels.

[0155] This embodiment will conduct experimental comparisons of the following methods:

[0156] LSTM model;

[0157] Bi-LSTM model;

[0158] mBART classification model;

[0159] mBART classification model + hard cue template 1;

[0160] mBART classification model + hard cue template 2;

[0161] mBART classification model + soft cue template 3;

[0162] mBART classification model + soft cue template 4;

[0163] mBART classification model + soft and hard cue combination template 5;

[0164] mBART classification model + soft and hard cue combination template 6;

[0165] For ease of observation, this embodiment will be written in the form of mBART+template x in the following articles or figures. For example, "mBART classification model + hard cue template 1" will be written as "mBART+template 1".

[0166] Parameter settings: The activation function is the Gelu function, and the parameter is initialized to N(0, 0.2).

[0167] Both the encoding and decoding layers have 12 layers, and the learning rate is... The epoch is 10 and the batch size is 128.

[0168] To verify the method proposed in this embodiment, this embodiment will compare the F1 scores of different algorithms.

[0169] Figure 7As can be seen in this embodiment, the combination of the mBART classification model and the prompt templates performs significantly better than the mBART model alone. The best-performing mBART+template 4 achieves an F1 score of 65.4, followed by mBART+template 3 with an F1 score of 64.8. This embodiment shows that soft prompts are the most effective, and the other templates also significantly improve the model's relation extraction. Comparing LSTM, Bi-LSTM, and the standalone mBART model, this embodiment demonstrates that the reconstructed mBART-based classification model also achieves good relation extraction performance, outperforming the two widely used language models mentioned earlier. This embodiment uses a large-scale dataset... Figure 7 As can be seen from this, the model in this embodiment still has a good effect when extracting relations from large-scale datasets.

[0170] for Figure 6 Based on the results, it can be considered that the parallel corpus of English and Chinese text entity relation extraction dataset constructed by the data preprocessing method designed in this embodiment can be used as a training dataset. This is because experiments on different models have shown that the English and Chinese text entity relation extraction dataset constructed in this embodiment still has normal performance.

[0171] This embodiment uses English as the source language and Chinese as the target language. It addresses the following problems in current cross-language relation extraction: 1. The reliance on zero-shot learning prevents supervised training for model learning, training, and testing, hindering its ability to adapt to large-scale data. 2. A lack of suitable supervised datasets for model training and testing. 3. Extensive preprocessing of sentences and feature extraction for each language are overly complex, significantly increasing time costs. This embodiment designs a cross-language large-scale model-based entity relation extraction technique for both Chinese and English texts and proposes a large-scale supervised dataset for training. This fills the gaps in large-scale training and the lack of supervised datasets in cross-language relation extraction, and also provides a data preprocessing method. Furthermore, experiments in this embodiment verify the usability of this dataset and the effectiveness of the proposed Chinese-English text relation extraction technique.

[0172] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for extracting entity relations from Chinese and English texts, characterized in that, Includes the following steps: Step 1: Preprocess the user's input sentence; Step 2: Extract entities from the preprocessed sentences in Step 1; Step 3: Combine the results of Step 2 with the prompts to fine-tune the template and input it into the Chinese and English text entity relation extraction network to perform Chinese and English text entity relation extraction; The Chinese and English entity relationship extraction network includes a SentencePiece processing layer, an encoding layer, a decoding layer, and a linear classification layer. The SentencePiece processing layer generates a numeric vector; the numeric vector generated in English is input into the encoding layer to generate an encoded vector; and the numeric vector generated in Chinese and the encoded vector generated in the encoding layer are input into the decoding layer to obtain the final relationship output type. Alternatively, the numeric vector generated in English is input into the decoding layer to generate an encoded vector; and the numeric vector generated in Chinese and the encoded vector generated in the decoding layer are input into the encoding layer to obtain the final relationship output type. The linear classification layer, consisting of a Linear layer and a Softmax layer, is used to classify entity relationships in a sentence. The prompt for fine-tuning includes: Template 1: Input to the encoding layer: <s>The sentence: " sentence(English) ” include entity1 (in English) and entity2 (English) < / s> ; Decoding layer input: <s>This sentence Sentence Examples (Chinese) "middle Entity 1 (Chinese) and Entity 2 (Chinese) What is the relationship between them?< / s> ; Template 2: Input for the encoding layer: <s>What is the type of relationship between entity1 (English arts) and entity2 (English) ?< / s> ; Decoding layer input: <s>This sentence Sentence Examples (Chinese) "Including these two entities" Entity 1 (Chinese) "and" Entity 2 (Chinese) "< / s> ; Template 3: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ; Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ; Template 4: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ; Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ; Template 5: Input for the encoding layer: <s>The sentence: " sentence(English) ” includes entity1 (in English) [Seg_ment_ation] entity2 (English) < / s> ; Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ; Template 6: Input for the encoding layer: <s>The sentence of sentence(English) includes entity1 (in English) [Seg_ment_ation] entity2 (English) < / s> ; Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ; Among them, [Seg_ment_ation] is the segmentation identifier, used to separate sentence instances from entities; <s> For sentence start markers,< / s> The template is for sentence end markers, entity1 is the input English entity 1, entity2 is the input English entity 2, entity1 is the input Chinese entity 1, entity2 is the input Chinese entity 2, sentence instance (Chinese) is the entire sentence containing Chinese entities, and sentence (English) is the entire sentence containing English entities; the underlined parts in the template are the input data, and the ununderlined parts are the prompt templates, (Chinese) indicates that the input is Chinese, and (English) indicates that the input is English.

2. The method for extracting entity relations from Chinese and English texts according to claim 1, characterized in that: In step 1, the preprocessing first discards sentences whose length exceeds a preset value; then the sentences are segmented. If a sentence contains a period and there is still a sentence after the period, the sentence is divided into multiple smaller sentences using the period as the cutting point.

3. The method for extracting entity relations from Chinese and English texts according to claim 1, characterized in that: In step 2, the PredPatt predicate-parameter extraction model is used to identify the predicate entity structure in the sentence; First, define the extracted linear form. Traverse the entity and predicate nodes extracted by PredPatt to obtain a new linear form. Then, mark the words of each node, where p represents a predicate marker, a represents an entity marker, ph represents a predicate header marker, and ah represents an entity header marker. Insert parentheses at the beginning and end of entities, and square brackets at the beginning and end of predicates. If an automatically generated linearized PredPatt has mismatched parentheses or curly braces, or if a predicate has zero or more header markers, it is an unusable incorrect format.

4. The method for extracting entity relations from Chinese and English texts according to claim 1, characterized in that: The preprocessed data in step 3 is input into our trained model, and the final relation type is output. For cross-language extraction tasks, the task is defined as follows: ,in It is a language pair for input that has already had prompt templates added. It is the output relation type, for each input instance. , For entities, 1≤ i ≤ n The two entities used to predict entity relationships are denoted as follows: and ,in Main entity, For object entities, the cross-language relation extraction task is to predict the main entity. and target entity Relationship ; Let M be the entity relation extraction network in both Chinese and English texts. Add before and after " <s>"and"< / s> "get" <s> < / s> "The input encoding layer, which has a self-attention mechanism, first calculates the query vector of the word." (Query) (Key) and weighted vector (Value): (Query)= ; (Key)= ; (Value)= ; in , , Given a random matrix, the final output of the self-attention mechanism module is obtained: ； in Let be the dimension of the matrix; () indicates that the normalized exponential function takes the value (0, 1); Will Perform residual connection to obtain new = LayerNorm ( ); Use two layers of linear mapping and activate using the GeLU activation function: ； The two Linear() functions represent two linear mappings; the above equation can be further written in the following form: ； in" "for" The randomly generated matrix of "". "for" The randomly generated matrix of "". "and" "Different bias values ​​initialized randomly" Perform residual connection ,get: = LayerNorm ( )= ； The above encoding process goes through 12 layers to obtain the final result. ; after that, " <s> < / s> "and Input decoding layer ; The decoding layer has a Masked Self-Attention module before the self-attention mechanism module of the encoding layer, and each layer of the decoding layer performs a Cross-Attention operation on the final layer output of the encoding layer; wherein the Masked Self-Attention module will... <s> < / s> The result is obtained by regenerating the input into the Masked Self-Attention module: (Query)= ; (Key)= ; Among them and The matrix obtained by dot product has some values ​​that become 0 or negative infinity, making it a lower triangular matrix. Then, the matrix is ​​normalized using the Softmax function at each layer to obtain the values ​​required in the self-attention module of the decoding layer. The Q and K matrices in the self-attention mechanism module of the decoding layer are the final outputs of the encoding layer. The generation process involves using the V matrix as the output of the Masked Self-Attention module. Subsequent operations are the same as in the encoding layer, but after each layer in the decoding layer outputs its output, the output of the encoding layer is again processed. Perform a Cross-Attention operation; if the output of a certain layer in the decoding layer is , i For the first layer inside the decoding layer i If there are multiple layers, the following sequence is obtained: ； In the above formula, " "for" "a random matrix, " "for" "a random matrix" "for" The purpose of these random matrices is to generate " "; then it becomes the input to the next layer in the decoding layer, and the above Cross-Attention operation is repeated through 12 layers until the final result is obtained. ,after The predicted relation type is obtained after passing through the Linear layer and Softmax layer. Probability: in, W This represents the random initialization matrix that needs to be optimized, and b is a random bias value.

5. The method for extracting entity relations from Chinese and English texts according to claim 1, characterized in that: In step 1, the encoding layer consists of 12 identical transformer encoders stacked together; the transformer encoder includes a self-attention mechanism module, a first residual block, a feedforward layer, and a second residual block. The self-attention mechanism module and the input data processed by the SentencePiece processing layer are both connected to the first residual block, and the input data is also connected to the self-attention module; the feedforward layer is connected to the first residual block and the second residual block, and the first residual block is connected to the second residual block.

6. The method for extracting entity relations from Chinese and English texts according to claim 1, characterized in that: In step 1, the decoding layer consists of 12 identical transformer decoders stacked together, and each transformer decoder has a cross-attention module between them; The transformer decoder includes a Masked Self-Attention module, a third residual block, a second self-attention mechanism module, a fourth residual block, a second feedforward layer, and a fifth residual block; The input data processed by the Masked Self-Attention module and the SentencePiece processing layer are both connected to the third residual block, and the input data processed by the SentencePiece processing layer are also connected to the MaskedSelf-Attention module; the final output data of the encoding layer is connected to the third residual block and the second self-attention mechanism module, and the third residual block and the second self-attention mechanism module are both connected to the fourth residual block; the fourth residual block is connected to the second feedforward layer, and both of the above are connected to the fifth residual block.

7. The method for extracting entity relations from Chinese and English texts according to any one of claims 1-6, characterized in that: The Chinese and English text entity relation extraction network is a pre-trained Chinese and English text entity relation extraction network; Its training process includes the following steps: (1) Construct a dataset for entity relation extraction from parallel Chinese and English corpora; Several data points were extracted from the WMT17 English-Chinese parallel corpus dataset and processed. The processing first discarded sentences whose word length exceeded a preset value. Then, the sentences were segmented. If a sentence contained a period and there was still a sentence after the period, the sentence was divided into multiple smaller sentences using the period as the cutting point. The PredPatt predicate-parameter extraction model is used to extract entities and connection predicates from sentences. By extracting predicates, sentences are classified, and sentences with more than M predicates under each predicate category are selected. Then, N relation labels are summarized from the classified predicates, and sentences with these N relation labels are selected to obtain a number of parallel corpus datasets with entities and entity relation labels. Here, M and N are preset values. The data in the parallel corpus datasets are divided into training set, test set and validation set. (2) Combine the data results in the training set with the prompts to fine-tune the template, generate a digital vector through the SentencePiece processing layer, input the digital vector into the Chinese and English entity relationship extraction network for training, and turn a machine translation model into a relationship classification model. For a language Examples The Chinese and English text entity relation extraction network acquires monolingual knowledge during the pre-training phase by optimizing the following objectives: in This refers to the language set included in CC-25 within the entity relation extraction network for both Chinese and English texts. Noise functions including sentence concealment and sentence ordering.

8. A system for extracting entity relations from Chinese and English texts, characterized in that, Includes the following modules: Module 1 is used to preprocess the user's input sentence; Module 2 is used to extract entities from the preprocessed sentences in Module 1; Module 3 is used to combine the results of Module 2 with the prompts and fine-tuning templates and input them into the Chinese and English text entity relation extraction network to extract Chinese and English text entities relations. The Chinese and English entity relationship extraction network includes a SentencePiece processing layer, an encoding layer, a decoding layer, and a linear classification layer. The SentencePiece processing layer generates a numeric vector; the numeric vector generated in English is input into the encoding layer to generate an encoded vector; and the numeric vector generated in Chinese and the encoded vector generated in the encoding layer are input into the decoding layer to obtain the final relationship output type. Alternatively, the numeric vector generated in English is input into the decoding layer to generate an encoded vector; and the numeric vector generated in Chinese and the encoded vector generated in the decoding layer are input into the encoding layer to obtain the final relationship output type. The linear classification layer, consisting of a Linear layer and a Softmax layer, is used to classify entity relationships in a sentence. The prompt for fine-tuning includes: Template 1: Input to the encoding layer: <s>The sentence: " sentence(English) ” include entity1 (in English) and entity2 (English) < / s> ; Decoding layer input: <s>This sentence Sentence Examples (Chinese) "middle Entity 1 (Chinese) and Entity 2 (Chinese) What is the relationship between them?< / s> ; Template 2: Input for the encoding layer: <s>What is the type of relationship between entity1 (English arts) and entity2 (English) ?< / s> ; Decoding layer input: <s>This sentence Sentence Examples (Chinese) "Including these two entities" Entity 1 (Chinese) "and" Entity 2 (Chinese) "< / s> ; Template 3: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ; Decoding layer input: <s> Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ; Template 4: Input for the encoding layer: <s> sentence(English) [Segmentation] entity1 (in English) [Segmentation] entity2 (English) < / s> ; Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ; Template 5: Input at the encoding layer: The sentence: " sentence(English) "includes" entity1 (in English) [Segmentation] entity2 (English) ; Decoding layer input: Sentence Examples (Chinese) [Segmentation] Entity 1 (Chinese) [Segmentation] entity 2 (Chinese) ; Template 6: Input for the encoding layer: <s>The sentence of sentence(English) includes entity1 (in English) [Seg_ment_ation] entity2 (English) < / s> ; Decoding layer input: <s> Entity 1 (Chinese) [Segmentation] Entity 2 (Chinese) < / s> ; Where [Seg_ment_ation] is the segmentation identifier, used to separate sentence instances from entities; <s> For sentence start markers,< / s> The template is defined as follows: entity1 is the input English entity 1, entity2 is the input English entity 2, entity1 is the input Chinese entity 1, entity2 is the input Chinese entity 2, sentence instance (Chinese) is the entire sentence containing Chinese entities, and sentence (English) is the entire sentence containing English entities. The underlined parts in the template represent the input data, while the ununderlined parts represent the prompt template. (Chinese) indicates that the input is in Chinese, and (English) indicates that the input is in English.

9. A device for extracting entity relationships from Chinese and English texts, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the Chinese and English text entity relation extraction method as described in any one of claims 1 to 7.

Images