Text information extraction method and device, named entity recognition method and device and relation extraction method and device

Abstract

The invention provides a text information extraction method and device, a named entity recognition method and device and a relation extraction method and device. According to the method, on the basis of a large-scale or small-scale information extraction model, the to-be-processed text information is analyzed to obtain the grammar semantic information (such as syntactic structure information and semantic association information) of the text information, and the grammar semantic information of the text information is injected into the information extraction model; according to the method, the information extraction model better learns language semantic knowledge, integrates grammar semantic information into text representation of the text information, obtains enhanced representation of the text information, and performs information extraction according to the enhanced representation to obtain an information extraction result, so that the missing of key information can be effectively reduced or even avoided, and the information extraction accuracy is improved.

Description

Technical Field

[0001] The present application relates to computer technology, and in particular to a method and device for text information extraction, named entity recognition, and relationship extraction. Background Art

[0002] The information extraction task refers to extracting structured key information from unstructured text, including named entity recognition, relationship extraction, etc. Although large models shine in various downstream tasks in the field of natural language processing, they do not perform well in information extraction tasks because they lack fine-grained language and semantic knowledge that is important for information extraction tasks.

[0003] Current information extraction models generally extract information based on the embedding vectors of the input text, which cannot obtain the language and semantic knowledge that are very important for information extraction tasks, resulting in the problem of missing key information in the information extraction results. Summary of the invention

[0004] The present application provides a method and device for text information extraction, named entity recognition, and relationship extraction, so as to solve the problem that key information is missing in the information extraction results.

[0005] In a first aspect, the present application provides a method for extracting text information, comprising: analyzing the text information to be processed to obtain grammatical and semantic information of the text information; inputting the grammatical and semantic information of the text information into an information extraction model, integrating the grammatical and semantic information into a text representation of the text information through the information extraction model to obtain an enhanced representation of the text information, and extracting information based on the enhanced representation to obtain an information extraction result.

[0006] In a second aspect, the present application provides a method for named entity recognition, comprising: obtaining text information to be processed in response to a call request from an end-side device to a named entity recognition interface; analyzing the text information to be processed to obtain grammatical and semantic information of the text information; inputting the text information and the grammatical and semantic information of the text information into a named entity recognition model, and integrating the grammatical and semantic information into a text representation of the text information through the named entity recognition model to obtain an enhanced representation of the text information; performing named entity recognition based on the enhanced representation to obtain the named entities contained in the text information; and returning the named entities contained in the text information to the end-side device.

[0007] In a third aspect, the present application provides a relationship extraction method, comprising: in response to a call request from an end-side device to a relationship extraction interface, obtaining text information to be processed and position information of two text fragments whose relationship is to be determined in the text information; analyzing the text information to be processed to obtain grammatical and semantic information of the text information; inputting the text information, the position information of the two text fragments, and the grammatical and semantic information of the text information into a relationship extraction model, and integrating the grammatical and semantic information into a text representation of the text information through the relationship extraction model to obtain an enhanced representation of the text information; performing relationship classification according to the enhanced representation to obtain a relationship category between the two text fragments; and returning the relationship category between the two text fragments to the end-side device.

[0008] In a fourth aspect, the present application provides a cloud server, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method described in any of the preceding aspects.

[0009] The methods and devices for text information extraction, named entity recognition, and relationship extraction provided in the present application obtain the grammatical and semantic information of the text information, which contains the language semantic knowledge of the text information. By integrating the grammatical and semantic information of the text information into the information extraction model, the information extraction model can better learn the language semantic knowledge, integrate the grammatical and semantic information into the text representation of the text information, obtain the enhanced representation of the text information, and realize information extraction, which can effectively reduce or even avoid the loss of key information and improve the accuracy of information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0011] Figure 1 A schematic diagram of an example network architecture applicable to the present application;

[0012] Figure 2 A flowchart of a text information extraction method provided for an exemplary embodiment of the present application;

[0013] Figure 3 A schematic diagram of a dependency syntax analysis tree provided for an exemplary embodiment of the present application;

[0014] Figure 4 A schematic diagram of a semantic dependency analysis tree provided for an exemplary embodiment of the present application;

[0015] Figure 5 Based on Figure 3A schematic diagram of the knowledge representation matrix of syntactic dependencies generated by the dependency syntactic analysis tree shown;

[0016] Figure 6 Based on Figure 4 Schematic diagram of the knowledge representation matrix of semantic dependency relations generated by the semantic dependency analysis tree shown;

[0017] Figure 7 A framework diagram of an information extraction method provided for an exemplary embodiment of the present application;

[0018] Figure 8 A flow chart of a training method for an information extraction model provided for an exemplary embodiment of the present application;

[0019] Fig. 9 A framework diagram of information extraction model training provided for an exemplary embodiment of the present application;

[0020] Fig.10 A flowchart of a method for named entity recognition provided for an exemplary embodiment of the present application;

[0021] Fig.11 A flow chart of a relationship extraction method provided for an exemplary embodiment of the present application;

[0022] Fig.12 A schematic diagram of the structure of a cloud server provided in an embodiment of the present application.

[0023] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0025] First, the terms involved in this application are explained:

[0026] Transformer: It is a deep learning architecture widely used in natural language processing tasks, which can effectively solve the problem of modeling sequence data. Its core is the attention mechanism.

[0027] BERT: The full name is Bidirectional Encoder Representations from Transformer. It is a bidirectional encoder representation model based on Transformer and a pre-trained language representation model.

[0028] Attention: A deep learning algorithm that allows the model to focus on the parts related to the current task and ignore the parts that are not related to the task when processing input data. It is usually implemented through a special layer called "attention layer". This layer can calculate the importance of each input and use these importances as weights to weight the inputs, so that the model pays more attention to the important parts. This mechanism can improve the performance and efficiency of the model, especially when processing long sequence data.

[0029] Self-Attention (SA) mechanism: A type of attention mechanism. There are three important input matrices in the self-attention mechanism: the query matrix Q (query), the key matrix K (key), and the value matrix V (value). These three matrices are obtained by different linear transformations of the input sequence (using three different weight matrices). Then, the product of the query matrix Q and the key matrix K is normalized by a softmax function to obtain a probability distribution with the same length as the input sequence, which is the attention weight. Finally, the attention weight is multiplied by the value matrix V to obtain the self-attention representation (vector / matrix) output by the self-attention mechanism, which represents the result of weighted averaging of the value of each element (value).

[0030] Named Entity Recognition (NER): Identify named entities with specific meanings from text.

[0031] Relation Extraction (RE): Identify the relationships between text fragments from the text.

[0032] Conditional Random Field (CRF): is a typical discriminant model that models the target sequence based on the observation sequence, focusing on solving the problem of serialization labeling.

[0033] Multimodal tasks: refers to downstream tasks whose input and output data involve multiple modal data such as images and text, such as visual question answering tasks, image description tasks, visual implication tasks, referential expression and understanding tasks, image generation tasks, etc.

[0034] Multimodal pre-trained model: refers to a pre-trained model whose input and output data involve multiple modal data such as images and text. After fine-tuning and training, it can be applied to multimodal task processing.

[0035] Pre-trained language model: A pre-trained model obtained by pre-training a large-scale language model (LLM).

[0036] A large model refers to a deep learning model with large-scale model parameters, usually containing hundreds of millions, tens of billions, or even hundreds of billions of model parameters. A large model can also be called a foundation model. It is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as large language models (LLM) and multi-modal pre-training models.

[0037] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0038] Although large models shine in various downstream tasks in the field of natural language processing, they do not perform well in information extraction tasks because they lack fine-grained linguistic and semantic knowledge that is important for information extraction tasks.

[0039] Current information extraction models, whether small-scale models or large-scale / ultra-large-scale models, mostly extract information based on the embedding vectors of the input text. They are unable to obtain the language and semantic knowledge that is very important for information extraction tasks, and generally lead to the problem of missing key information.

[0040] The present application provides a text information extraction method, which analyzes the text information to be processed to obtain the grammatical semantic information of the text information; inputs the grammatical semantic information of the text information into an information extraction model, and integrates the grammatical semantic information into the text representation of the text information through the information extraction model to obtain an enhanced representation of the text information; extracts information based on the enhanced representation to obtain an information extraction result. The method of the present application obtains the grammatical semantic information of the text information, and the grammatical semantic information includes the language semantic knowledge of the text information. By injecting the grammatical semantic information of the text information into the information extraction model, the information extraction model can better learn the language semantic knowledge, integrate the grammatical semantic information into the text representation of the text information, obtain the enhanced representation of the text information, and realize information extraction, which can effectively reduce or even avoid the loss of key information and improve the accuracy of information extraction.

[0041] Among them, the grammatical semantic information of the text information includes the syntactic structure information and / or semantic association information of the text information. The syntactic structure information of the text information includes the relationship between each grammatical component (text fragment) in the text information, reveals the syntactic structure of the text information, and is obtained by analyzing the grammatical components such as subject, predicate, object, attributive, adverbial, and complement in the text, and analyzing the relationship between each grammatical component. The semantic association information of the text information includes the semantic association relationship between the text fragments in the text information, and is obtained by analyzing the semantic relationship between the content words and the content words in the text information. This semantic relationship is a factual or logical relationship, and only exists when the words enter the sentence.

[0042] Figure 1 FIG. 1 is a schematic diagram of an example network architecture applicable to this application. Figure 1 As shown, the network architecture includes: a terminal device and a server. There is a communication link between the server and the terminal device, which can realize the communication connection between the server and the terminal device.

[0043] Among them, the server can be a server cluster deployed in the cloud, or a local device with computing power. The server runs an information extraction model for implementing information extraction functions based on the information extraction model. The server can run one or more information extraction models applied to different information extraction tasks, including but not limited to named entity recognition models and relationship extraction models. The information extraction model can be implemented based on a large model or a small-scale transformer-based information extraction model. Specifically, it can be obtained by improving the attention module of any transformer-based information extraction model, so that the improved attention module has the ability to integrate the language and semantic knowledge of text information into text representation.

[0044] The terminal device is an electronic device used by the user, which can be a hardware device with network communication function, computing function and information display function, including but not limited to smart phones, tablet computers, desktop computers, server devices, etc.

[0045] When extracting information, the user sends the text information to be processed to the server through the end-side device. The server receives the text information to be processed sent by the end-side device, analyzes the text information to be processed, and obtains the grammatical and semantic information of the text information; inputs the grammatical and semantic information of the text information into the information extraction model, and integrates the grammatical and semantic information into the text representation of the text information through the information extraction model to obtain an enhanced representation of the text information; extracts information based on the enhanced representation to obtain an information extraction result. Furthermore, the server returns the information extraction result to the end-side device.

[0046] Exemplarily, the server may provide an information extraction service interface for implementing information extraction tasks, such as a named entity recognition interface for implementing named entity recognition, a relationship extraction interface for implementing relationship extraction, etc. The end-side device may submit the text information to be processed to the server by calling the interface to trigger the server to execute the corresponding information extraction task and return the information extraction result. In addition, in other embodiments, the end-side device may send an execution request for the information extraction task to the server, carry the text information to be processed in the execution request and send it to the server to trigger the server to execute the corresponding information extraction task and return the information extraction result.

[0047] In an example scenario, the method of the present application can implement a named entity recognition task, and the server runs a named entity recognition model. When performing named entity recognition, the user sends a call request to the named entity recognition interface to the server through the terminal device, and the call request contains the text information to be processed. In response to the call request of the terminal device to the named entity recognition interface, the server obtains the text information to be processed, analyzes the text information to be processed, and obtains the grammatical and semantic information of the text information; inputs the grammatical and semantic information of the text information into the named entity recognition model, and integrates the grammatical and semantic information into the text representation of the text information through the named entity recognition model to obtain an enhanced representation of the text information; performs named entity recognition based on the enhanced representation to obtain the named entities contained in the text information. Furthermore, the server returns the named entities contained in the text information to the terminal device.

[0048] The named entity recognition solution implemented in this application can be specifically applied to application scenarios such as text key element extraction, machine translation, question-answering system, and knowledge graph construction.

[0049] In another example scenario, the method of the present application can be applied to a relationship extraction task, and the server runs a relationship extraction model. When performing relationship extraction, the user sends a call request for the relationship extraction interface to the server through the terminal device, and the call request contains the text information to be processed and the position information of the two text fragments whose relationship is to be determined in the text information. In response to the call request of the terminal device for the relationship extraction interface, the server obtains the text information to be processed and the position information of the two text fragments whose relationship is to be determined in the text information; analyzes the text information to be processed to obtain the grammatical semantic information of the text information; inputs the text information, the position information of the two text fragments, and the grammatical semantic information of the text information into the relationship extraction model, and integrates the grammatical semantic information into the text representation of the text information through the relationship extraction model to obtain an enhanced representation of the text information; classifies the relationship according to the enhanced representation to obtain the relationship category between the two text fragments. Further, the server returns the relationship class between the two text fragments to the terminal device.

[0050] The relationship extraction scheme implemented in this application can be widely used in finance, medical care, social network analysis, public opinion monitoring, public opinion analysis and other fields. For example, in the financial industry, relationship extraction technology can be used to discover the relationship between customers and companies, so as to better carry out market positioning and customer management. In the medical field, relationship extraction technology can be used to mine information such as efficacy evaluation and drug interactions from medical literature to help clinical decision-making and disease diagnosis.

[0051] In a specific application scenario, the method of the present application can be applied to construct a knowledge graph based on a text corpus. The server runs a named entity recognition model and a relationship extraction model. Based on the solution of the first example scenario, named entities are identified from the text corpus, and based on the solution of the second example scenario, the relationship between any named entities is extracted from the text corpus. Furthermore, based on the extracted named entities and the relationship between the named entities, a knowledge graph is constructed.

[0052] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0053] Figure 2 This is a flowchart of a text information extraction method provided by an exemplary embodiment of the present application. The execution subject of this embodiment is the server in the aforementioned system architecture. Figure 2 As shown, the specific steps of this method are as follows:

[0054] Step S201: Analyze the text information to be processed to obtain grammatical and semantic information of the text information.

[0055] The grammatical and semantic information of the text information includes the syntactic structure information and / or semantic association information of the text information.

[0056] The syntactic structure information of text information includes the relationship between various grammatical components (text fragments, also called word units, words) in the text information, and reveals the syntactic structure of the text information. It is obtained by analyzing the grammatical components such as subject, predicate, object, attributive, adverbial, and complement in the text, and analyzing the relationship between each grammatical component. Text information can be represented as a character sequence formed by the arrangement of multiple characters. Based on the syntactic structure information of the text information, the text information is divided into multiple word units, and the syntactic structure information includes the syntactic dependency relationship between each word unit.

[0057] In this step, the server can perform syntactic analysis on the text information to obtain the syntactic structure information of the text information, or perform semantic analysis on the text information to obtain the semantic association information of the text information, or perform syntactic analysis and semantic analysis on the text information respectively to obtain the syntactic structure information and semantic association information of the text information.

[0058] Optionally, when the server performs syntactic analysis on the text information to obtain the syntactic structure information of the text information, it can perform dependency syntactic analysis on the text information to obtain a dependency syntactic analysis tree (Dependency Parse Tree, DEPT for short), which contains the syntactic structure information of the text information. Specifically, the server can generate a corresponding character sequence based on the text information, and the character sequence is composed of the characters contained in the text information (repeated characters are regarded as multiple characters) arranged in the order in which they appear in the text information. The character sequence of the text information is input into an existing dependency syntactic analysis tool, and the tool is used to perform dependency syntactic analysis on the character sequence to obtain a dependency syntactic analysis tree. The dependency syntactic analysis tree reveals the syntactic structure of the input text information.

[0059] In the dependency syntactic analysis tree, each word (word unit) in the text information corresponds to a node, and a root node (Root) is inserted at the head. All nodes are connected by directed arcs to form a tree, and the following three conditions are met: First, except for the root node, any node has and only has one incoming edge; second, except for the leaf node, any node has at least one outgoing edge, and the root node has only one outgoing edge, and the corresponding arc points to the core word that dominates the entire sentence; third, all arcs cannot cross. In the dependency syntactic tree, an arc represents a dependency relationship (syntactic dependency relationship), and the direction of the arc represents the dominance order. A directed arc always starts from a parent node and points to its child node. This is called the parent node dominating the child node, and the child node depends on the parent node. Common syntactic dependency relationships include: Attributive-Central Relationship (ATT), which indicates that the attributive is dependent on the central word; Adverbial-Central Relationship (ADV), which indicates that the adverbial is dependent on the central word; Parallel Relationship (COO), which indicates that the noun dominates (is parallel to) the noun; Subject-Predicate Relationship (SBV), which indicates that the subject is dependent on the predicate; Verb-Object Relationship (VOB), which indicates that the predicate dominates the object; Core Relationship (HED), which indicates that the Root dominates the core word of the sentence; Dependency Clause (DC), which indicates that the core word of the main clause dominates the core word of the clause; Punctuation (PUNC), which indicates that the core word of the sentence dominates the punctuation. In addition, the syntactic dependency tree can also contain other syntactic dependency relationships. Depending on the different dependency syntactic analysis tools used, the types and names of the syntactic dependency relationships may be different, which does not affect the use of this solution.

[0060] For example, taking the text information "West Lake is located in Hangzhou." as an example, its character sequence can be represented as: {"West", "Lake", "Location", "Yu", "Hangzhou", "State", "."}. The dependency parse tree corresponding to the character sequence is as follows: Figure 3 As shown in the figure, the character sequence is divided into four text segments (word units): "located", "West Lake", "Hangzhou", and ".". The syntactic dependency relationship between the root node (Root) and "located" is ②, referring to "HED", indicating that "located" is the core word of the text information; the syntactic dependency relationship between "located" and "West Lake" is ①, referring to "SBV", indicating that the two are in a subject-predicate relationship; the syntactic dependency relationship between "located" and "Hangzhou" is ③, referring to "VOB", indicating that the two are in a verb-object relationship; the syntactic dependency relationship between "located" and "." is ④, referring to "PUNC", indicating that "." is a punctuation mark.

[0061] In addition, in this step, the server can also use probability-based context-free grammars (PCFG) methods, binary word affinity models, maximum spanning tree syntactic analysis, support vector machines, decision trees, syntactic analysis methods based on deep learning, etc. to perform syntactic analysis on the text information to obtain the syntactic structure information of the text information. This embodiment does not specifically limit the specific implementation method for obtaining the syntactic structure information of the text information.

[0062] In this embodiment, the semantic association information of the text information includes the semantic association relationship between text fragments in the text information, which is obtained by analyzing the semantic relationship between content words in the text information. This semantic relationship is a factual or logical relationship and only exists when the words enter a sentence.

[0063] Optionally, in this step, when the server performs semantic analysis on the text information to obtain semantic association information of the text information, it can perform semantic dependency analysis on the text information to obtain a semantic dependency analysis tree (Semantic Dependency Parse Tree, referred to as SDPT), which contains the semantic association information of the text information. Specifically, the server can generate a corresponding character sequence based on the text information, and the character sequence is composed of the characters contained in the text information (repeated characters are regarded as multiple characters) arranged in the order in which they appear in the text information. The character sequence of the text information is input into an existing semantic dependency analysis tool, and the tool is used to perform semantic dependency analysis on the character sequence to obtain a semantic dependency analysis tree. The semantic dependency analysis tree reveals the semantic relationship between each word in the input text information.

[0064] The structure of the semantic dependency parse tree is similar to that of the dependency syntactic parse tree. The difference is that the directed arcs in the semantic dependency parse tree represent semantic dependency relations, while the directed arcs in the dependency syntactic parse tree represent syntactic dependency relations. The semantic dependency relations are classified into several categories, including semantic roles, event relations, and relation markers. Among them, event relations include but are not limited to: agent relations (Agt), such as I gave him a gift (I←send); party relations (Exp), such as I run fast (run→I); feeling relations (Aft), such as I miss my hometown (missing→I); .... Semantic roles include but are not limited to: belonging roles (Belg), such as Lao Wang has a son (Lao Wang←have); comparison roles (Comp), such as his grades are better than mine (he→I); intention roles (Int), such as he worked very hard for the gold medal (gold medal←work hard); time roles (Time); space roles (Loc)... Relationship tags include but are not limited to: parallel relationship (eSelt), cause relationship (eCau), result relationship (eResu), conditional relationship (eCond), purpose relationship (eMetd), conjunction tag (mConj), word tag (mAux), tone tag (mTone), time tag (mTime), negation tag (mNeg), punctuation tag (mPunc)... In addition, the semantic dependency analysis tree can also contain other semantic dependency relationships, only some of which are listed here. Depending on the different semantic dependency analysis tools used, the types and names of semantic dependency relationships may be different, which does not affect the use of this solution.

[0065] For example, taking the text information "West Lake is located in Hangzhou." as an example, its character sequence can be represented as: {"West", "Lake", "Location", "Yu", "Hangzhou", "State", "."}. The semantic dependency analysis tree corresponding to the character sequence is as follows: Figure 4 As shown, the character sequence is divided into four text segments (word units): "located", "West Lake", "Hangzhou", "." The semantic dependency relationship between the root node (Root) and "located" is A , refers to "head", indicating that "located" is the core word of the text information; the relationship between "located" and "West Lake" is B , refers to "Parties (Exp)", the relationship between "Located in" and "Hangzhou" is C , refers to "spatial role (Loc)"; the relationship between "located" and "." is D , which stands for "punctuation mark (mPunc)", means that "." is a punctuation mark.

[0066] In addition, in this step, the server can also perform semantic dependency analysis on the text information by using a semantic dependency analysis tool to obtain a semantic dependency analysis graph, and use the semantic dependency analysis graph as the semantic association information of the text information. In addition, the server can also use a text topic model, such as latent semantic analysis (Latent Dirichlet Allocation, LDA), probabilistic latent semantic analysis (Probabilistic Latent Semantic Analysis, PLSA), etc., or a semantic analysis model based on deep learning, etc., to perform semantic analysis on the text information and obtain the semantic association information of the text information, which is not specifically limited in this embodiment.

[0067] Step S202: input the grammatical and semantic information of the text information into the information extraction model, integrate the grammatical and semantic information into the text representation of the text information through the information extraction model, obtain an enhanced representation of the text information, and extract information based on the enhanced representation to obtain an information extraction result.

[0068] In this embodiment, the server can obtain any one or two kinds of knowledge information of the syntactic structure information and the semantic association information of the text information as the grammatical semantic information of the text information, which can be specifically configured according to the needs of the actual application scenario. In step S202, the knowledge information obtained in the previous step is injected into the information extraction model so that the information extraction model can learn the injected knowledge information to achieve more accurate information extraction.

[0069] It should be noted that the input of the information extraction model also includes the embedding vector of the text information as the initial text representation of the text information. For information extraction tasks such as relation extraction (RE), the input of the information extraction model also includes the embedding vector and position information (usually position encoding) of the text information. The embedding vector and position information of the text information are concatenated as the initial text representation of the text information.

[0070] In a preferred embodiment, the server simultaneously obtains the syntactic structure information and semantic association information of the text information, thereby obtaining the language knowledge and semantic knowledge of the text information, and subsequently injects the obtained language knowledge and semantic knowledge into the information extraction model, so that the information extraction model can learn the language and semantic knowledge to achieve more accurate information extraction.

[0071] Specifically, the server can input the grammatical semantic information of the text information into the information extraction model. In the process of representing the text information, the information extraction model will learn the grammatical semantic information of the text information, and integrate the learned knowledge into the text representation of the text information to obtain an enhanced representation. The enhanced representation has a stronger representation ability and contains richer language and language information. Further, through the output layer of the information extraction model, information is extracted based on the enhanced representation, and the information extraction result is output.

[0072] Optionally, in this step, when implementing the text representation of integrating grammatical semantic information into text information, the embedding vector of the grammatical semantic information can be spliced ​​with the embedding vector of the text information, and the information extraction model characterizes the splicing result to obtain an enhanced representation. For example, if the grammatical semantic information contains both syntactic structure information and semantic association information, the syntactic structure information and the semantic association information can be mapped to embedding vectors respectively, and the embedding vectors of the syntactic structure information and the semantic association information can be spliced ​​with the embedding vector of the text information, and the information extraction model characterizes the splicing result to obtain an enhanced representation.

[0073] Optionally, in this step, when implementing the integration of grammatical semantic information into the text representation of text information, a key matrix for attention calculation can be generated according to the representation of grammatical semantic information through the attention module of the information extraction model, and the text representation of the text information is subjected to attention calculation according to the key matrix to obtain an enhanced representation of the text information. The language semantic knowledge contained in the grammatical semantic information is integrated into the text representation of the text information through attention calculation to enhance the representation of the text information. There are three input matrices in the self-attention (SA) mechanism: query matrix Q (query), key matrix K (key) and value matrix V (value). The key matrix K in the traditional self-attention mechanism is obtained by linear transformation of the input sequence. In this embodiment, the attention module is a variant of the traditional self-attention mechanism, and also includes three input matrices: query matrix Q (query), key matrix K (key) and value matrix V (value), and the functions and meanings of these three matrices are the same as those in the traditional self-attention mechanism. The difference is that in the attention module (variant of the self-attention mechanism) of this embodiment, the key matrix K used for attention calculation is generated according to the knowledge representation of grammatical semantic information.

[0074] Furthermore, when information extraction is performed based on the enhanced representation of text information to obtain an information extraction result, the enhanced representation of the text information is input into the output layer of the information extraction model, and information extraction is performed based on the enhanced representation of the text information through the output layer to output the information extraction result.

[0075] Exemplarily, for the information extraction model (named entity recognition model) for named entity recognition, its output layer can use a conditional random field (CRF) model. The conditional random field (CRF) model is used to mark the start position and end position of the named entity in the character sequence of the input text information. For example, the RIOES annotation set can be used to annotate each character in the character sequence, with B indicating that the character is at the beginning (Begin) position of an entity, I indicating that the character is inside (Inside) of an entity, O indicating that the character is outside (Outside) of an entity, E indicating that the character is at the end (End) position of an entity, and S indicating that the character itself can form an entity (Single).

[0076] In addition, the output layer of the named entity recognition model can also be implemented using an output layer structure of any existing model for named entity recognition that generates named entity recognition results based on text representation, and no specific limitation is made here.

[0077] Exemplarily, for an information extraction model (relation extraction model) for relation extraction, its output layer can be implemented using a relation classifier. The relation classifier is used to classify and identify the relation category between two text segments specified in the character sequence of the input text information. The relation classifier can be a classifier for performing multiple classifications, and can be implemented using a classifier used in any existing relation extraction model, which is not specifically limited here.

[0078] The method of this embodiment obtains grammatical and semantic information of text information, which includes language semantic knowledge. By injecting the grammatical and semantic information of text information into the information extraction model, the information extraction model can better learn the language semantic knowledge, integrate the grammatical and semantic information into the text representation of the text information, obtain the enhanced representation of the text information, and realize information extraction, which can effectively reduce or even avoid the loss of key information and improve the accuracy of information extraction.

[0079] In an optional embodiment, the user is supported to adjust and confirm the grammatical and semantic information of the obtained text information to improve the accuracy of the syntactic structure information and semantic association information of the text information. Specifically, after obtaining the grammatical and semantic information of the text information, the server can also output the grammatical and semantic information of the text information through the front-end interface. The user can adjust the word units divided in the syntactic structure information and the syntactic dependency relationship between any two word units through the front-end interface, and can also adjust the word units divided in the semantic association information and the semantic dependency relationship between any two word units. Furthermore, in response to the editing operation of the grammatical and semantic information in the front-end interface, the server updates the grammatical and semantic information.

[0080] Exemplarily, the grammatical semantic information of the acquired text information may include syntactic structure information, and the syntactic structure information may be a dependency syntactic analysis tree, and the server may visually output the dependency syntactic analysis tree through the front-end interface. The user may edit the nodes (corresponding to word units) and the arcs between any two nodes (representing the syntactic dependency relationship between the two nodes) in the dependency syntactic analysis tree. The server updates the dependency syntactic analysis tree according to the editing operation of the dependency syntactic analysis tree on the front-end interface.

[0081] Exemplarily, the grammatical semantic information of the acquired text information may include semantic association information, and the semantic association information may be a semantic dependency analysis tree, and the server may visually output the semantic dependency analysis tree through a front-end interface. The user may edit the nodes (corresponding to word units) and the arcs between any two nodes (representing the semantic dependency relationship between the two nodes) in the semantic dependency analysis tree. The server updates the semantic dependency analysis tree according to the editing operation of the semantic dependency analysis tree on the front-end interface.

[0082] In an optional embodiment, in the aforementioned step S202, the grammatical and semantic information of the text information is input into the information extraction model, which can be specifically implemented in the following manner: the grammatical and semantic information of the text information is represented to obtain the corresponding knowledge representation; and the knowledge representation is input into the information extraction model.

[0083] Exemplarily, the grammatical and semantic information of the text information may include one of syntactic structure information and semantic association information. In the aforementioned step S202, the syntactic structure information or the semantic association information included in the grammatical and semantic information is represented, and the obtained knowledge representation is input into the information extraction model.

[0084] Exemplarily, the grammatical semantic information of the text information may include both syntactic structure information and semantic association information. In the aforementioned step S202, the syntactic structure information of the text information is represented to obtain a syntactic structure knowledge representation; the semantic association information of the text information is represented to obtain a semantic association knowledge representation; and the syntactic structure knowledge representation and the semantic association knowledge representation are input into the information extraction model.

[0085] In this embodiment, different relationships in the obtained syntactic structure information and semantic association information play different roles in the text, and the semantic knowledge contained in the relationship itself is very rich and important. By performing meaningful relationship representations on the obtained syntactic structure information and semantic association information respectively, a good representation enables the information extraction model to learn the rich language and semantic knowledge of the text information, thereby enhancing the text representation of the text information and obtaining an enhanced representation with stronger expressive ability.

[0086] Optionally, when representing the syntactic structure information of the text information and obtaining the syntactic structure knowledge representation, the syntactic structure information of the text information can be directly mapped to an embedding vector (embedding), and the embedding vector of the syntactic structure information can be used as the syntactic structure knowledge representation; or, the syntactic structure information of the text information can be mapped to an embedding vector, and the embedding vector of the syntactic structure information can be encoded using a coding model to obtain the syntactic structure knowledge representation.

[0087] When representing the semantic association information of text information and obtaining the semantic association knowledge representation, the semantic association information of the text information can be directly mapped to an embedding vector, and the embedding vector of the semantic association information can be used as the semantic association knowledge representation; or, the semantic association information of the text information can be mapped to an embedding vector, and the embedding vector of the semantic association information can be encoded using an encoding model to obtain the semantic association knowledge representation. The encoding model used can be a BERT model or other models commonly used for text encoding, which is not specifically limited in this embodiment.

[0088] Optionally, the syntactic structure information of the text information is represented to obtain the syntactic structure knowledge representation, which can also be achieved by the following steps a1-a3:

[0089] Step a1: Based on the syntactic structure information of the text information, for any two word units with a syntactic dependency relationship, encode the concatenated text of the text information and the two word units with a syntactic dependency relationship to obtain a knowledge representation of the syntactic dependency relationship.

[0090] Among them, based on the syntactic structure information of the text information, the text information is divided into multiple word units, and the syntactic structure information includes the syntactic dependency relationship between each word unit. According to the syntactic structure of the text information, some word units have syntactic dependency relationships, while some word units do not have syntactic dependency relationships. Any two word units with syntactic dependency relationships can be expressed as {ent p ,ent q ,r dep}, where ent p ,ent q Represents two word units, r dep Represents two word units ent p andent q C represents the original character sequence of text information.

[0091] In step a1, the character sequence C of the text information is concatenated with any two word units having a syntactic dependency relationship to obtain a concatenated text (denoted as C prompt ): C prompt =[CLS]+C+[SEP]+entp +[SEP]+ent q , where [CLS] and [SEP] are two predefined markers, [CLS] indicates the beginning of a text (sentence or document), and [SEP] indicates the end of a text (sentence or document), which are used to segment different texts. prompt Input the encoding model, and concatenate the text C through the encoding model prompt Encode and obtain the syntactic dependency r dep A knowledge representation of ). This knowledge representation is a sentence-level knowledge representation.

[0092] The encoding model used can be a BERT model, which uses the BERT model to encode the character sequence C of the text information and the concatenation text C of any two word units with syntactic dependencies. prompt Encoding is essentially the concatenation of text C prompt The process of further feature extraction can better represent the syntactic structure information. In addition, other models commonly used for text encoding can also be used to concatenate the text C prompt Encoding is performed. In this embodiment, the encoding model used is not specifically limited.

[0093] In step a1, based on the syntactic structure information of the text information, word units with different syntactic dependencies are processed to obtain knowledge representations corresponding to different syntactic dependencies. Multiple groups of word units with the same syntactic dependency are processed to obtain multiple knowledge representations corresponding to the same syntactic dependency.

[0094] Step a2: Determine the relational knowledge representation of the syntactic dependency relationship based on the knowledge representation of the same syntactic dependency relationship.

[0095] Optionally, based on the knowledge representations of each syntactic dependency relationship obtained in the above step a1, the knowledge representations of the same syntactic dependency relationship are averaged as the relational knowledge representation of the syntactic dependency relationship (which can be expressed as ), the quality of relational knowledge representation can be improved. In addition, based on the knowledge representations of each syntactic dependency relationship obtained in the aforementioned step a1, the knowledge representations of the same syntactic dependency relationship are concatenated / summed up as the relational knowledge representation of the syntactic dependency relationship.

[0096] Step a3: Based on the relational knowledge representations of various syntactic dependencies contained in the text information, a knowledge representation matrix of syntactic dependencies is constructed to obtain a syntactic structure knowledge representation.

[0097] Based on the knowledge representation of each syntactic dependency relationship obtained in step a2, step a3 can help the information extraction model better understand and learn syntactic structure knowledge by integrating the relational knowledge representation of various syntactic dependency relationships contained in the text information into a knowledge representation matrix.

[0098] Specifically, we first construct a full Matrix M dep Then, based on the syntactic structure information of the text information, for any two word units {ent p ,ent q ,r dep},when as well as Sometimes, there is That is, the matrix M dep The value at position (i,j) is set to in, and Respectively represent word units ent p The starting and ending positions in the character sequence of the text message, and Respectively represent word units ent q The starting and ending positions in the character sequence of the text information. p and word unit ent q The syntactic dependency relationship between dep The relational knowledge representation of is integrated into the knowledge representation matrix M of the syntactic dependency relationship dep The relational knowledge representation is extended from the word unit level to the token level.

[0099] For example, the syntactic structure information of the text information is a dependency syntactic parse tree (DEPT) as an example, the dependency syntactic parse tree (T dep It can be expressed as: T dep ={Root dep ,Node dep ,Edge dep}, where Root dep Represents the root of the dependency parse tree. dep A node set representing a dependency parsing tree, where each node is a word unit (text fragment), and a word unit may contain more than one character. dep Represents the edge set of the dependency parse tree, each edge can be abstractly represented as {ent p ,ent q ,r dep}, indicating the word unit ent p and word unit entq There is a syntactic dependency relationship between dep . r dep ∈R dep , R dep represents the set of all possible syntactic dependencies. In this step, the dependency syntactic analysis tree T based on the text information dep , for each edge {ent p ,ent q ,r dep}, generate syntactic dependency r dep The corresponding sentence-level knowledge representation It is possible to obtain sentence-level knowledge representations corresponding to multiple different edges.

[0100] Furthermore, the same syntactic dependency r dep The corresponding knowledge representation of all sentences Find the average and get the syntactic dependency r dep Relational knowledge representation

[0101] Furthermore, building a Matrix M dep , dependency syntactic analysis tree T based on text information dep , for any edge {ent p ,ent q ,r dep},when as well as Sometimes, there is That is, the matrix M dep The value at position (i,j) is set to in, and Respectively represent word units ent p The starting and ending positions in the character sequence of the text message, and Respectively represent word units ent q The starting and ending positions in the character sequence of the text information. p and word unit ent q The syntactic dependency relationship between dep The relational knowledge representation of is integrated into the knowledge representation matrix M of the syntactic dependency relationship dep middle.

[0102] For example, based on Figure 3 The dependency parse tree shown in Figure 5 The knowledge representation matrix M of the syntactic dependency relationship shown dep . Figure 3In the parse tree, ①②③④ are used to represent the syntactic dependency relationship corresponding to each edge in the dependency syntactic analysis tree. Taking the edge {"West Lake", "located", ①} as an example, the starting and ending positions of "West Lake" in the text information are 1 and 2 respectively, and the starting and ending positions of "located" in the text information are 3 and 4 respectively. In the knowledge representation matrix M of the syntactic dependency relationship, dep The positions of the 1st row and the 3rd-4th columns of the 2nd row are set as the relational knowledge representation corresponding to the syntactic dependency relation ①, Figure 5 In the 1st row and the 3rd to 4th column of the 2nd row, ① is filled in, indicating that the corresponding position is filled with the relational knowledge representation corresponding to the syntactic dependency relation ①. For the edge {“Hangzhou”, “located in”, ③}, the starting and ending positions of “Hangzhou” in the text information are 5 and 6 respectively. In the knowledge representation matrix M dep The positions of the 3rd and 4th columns of the 5th and 6th rows are set as the relational knowledge representation corresponding to the syntactic dependency relation ③. Figure 5 In the 5th and 6th rows, the 3rd and 4th columns are filled with ③, indicating that the corresponding positions are filled with the relational knowledge representation corresponding to the syntactic dependency relation ③. For the edge {“.”, “located at”, ④}, the starting and ending positions of “.” in the text information are both 7, and in the knowledge representation matrix M dep The positions of the 3rd and 4th columns of the 7th row are set as the relational knowledge representation corresponding to the syntactic dependency relation ④. Figure 5 In the figure, the position between the 3rd and 4th columns of the 7th row is filled with ④, indicating that the corresponding position is filled with the relational knowledge representation corresponding to the syntactic dependency relation ④.

[0103] It should be noted that the method of this embodiment can be applied not only to Chinese text information extraction, but also to text information extraction in other languages ​​such as English. For example, for the text information "The West Lake is located in Hangzhou.", the generated dependency syntactic analysis tree divides the text information into word units corresponding to the word unit division of the corresponding Chinese text, including the word units "The West Lake", "is located in", "Hangzhou" and "."

[0104] Optionally, the semantic association information of the text information is represented to obtain the semantic association knowledge representation, which can also be implemented by the following steps b1-b3:

[0105] Step b1: Based on the semantic association information of the text information, for any two word units with a semantic dependency relationship, encode the concatenated text of the text information and the two word units with a semantic dependency relationship to obtain a knowledge representation of the semantic dependency relationship.

[0106] Among them, based on the semantic association information of the text information, the text information is divided into multiple word units, and the semantic association information includes the semantic dependency relationship between each word unit. According to the semantic relationship of the text information, some word units have semantic dependency relationship, while some word units do not have semantic dependency relationship. Any two word units with semantic dependency relationship can be expressed as {node p ,node q ,r sdp}, where node p ,node q Represents two word units, r sdp Represents two word units node p and node q The semantic dependency relationship between them exists. C is used to represent the original character sequence of the text information.

[0107] In step a1, the character sequence C of the text information is concatenated with any two word units having a semantic dependency relationship to obtain a concatenated text (denoted as C p ′ rompt ): C p ′ rompt =[CLS]+C+[SEP]+node p +[SEP]+node q , where [CLS] and [SEP] are two predefined markers, [CLS] indicates the beginning of a text (sentence or document), and [SEP] indicates the end of a text (sentence or document), which are used to segment different texts. p ′ rompt Input the encoding model, and concatenate the text C through the encoding model p ′ rompt Encode and obtain the semantic dependency relationship r sdp A knowledge representation of ). This knowledge representation is a sentence-level knowledge representation.

[0108] The encoding model used can be a BERT model, which uses the BERT model to encode the character sequence C of the text information and the concatenated text C of any two word units with semantic dependencies. p ′ rompt Encoding is essentially the concatenation of text C p ′ rompt The process of further feature extraction can better represent the semantic association information. In addition, other models commonly used for text encoding can also be used to concatenate the text Cp ′ rompt Encoding is performed. In this embodiment, the encoding model used is not specifically limited.

[0109] In step a1, based on the semantic association information of the text information, word units with different semantic dependencies are processed to obtain knowledge representations corresponding to different semantic dependencies. Multiple groups of word units with the same semantic dependency are processed to obtain multiple knowledge representations corresponding to the same semantic dependency.

[0110] Step b2: Determine the relational knowledge representation of the semantic dependency relationship based on the knowledge representation of the same semantic dependency relationship.

[0111] Optionally, based on the knowledge representations of each semantic dependency relationship obtained in the above step b1, the knowledge representations of the same semantic dependency relationship are averaged as the relational knowledge representation of the semantic dependency relationship (which can be expressed as ), the quality of relational knowledge representation can be improved. In addition, based on the knowledge representations of the various semantic dependency relationships obtained in the aforementioned step b1, the knowledge representations of the same semantic dependency relationship are concatenated / summed up as the relational knowledge representation of the semantic dependency relationship.

[0112] Step b3: construct a knowledge representation matrix of semantic dependencies based on the relational knowledge representations of various semantic dependencies contained in the text information to obtain semantic association knowledge representations.

[0113] Based on the knowledge representation of each semantic dependency relationship obtained in step b2, step b3 can help the information extraction model better understand and learn semantic knowledge by integrating the relational knowledge representation of various semantic dependency relationships contained in the text information into a knowledge representation matrix.

[0114] Specifically, we first construct a full Matrix M sdp Then, based on the semantic association information of the text information, for any two word units {node p ,node q ,r sdp},when as well as Sometimes, there is That is, the matrix M sdp The value at position (i,j) is set to in, and Respectively represent word units node p The starting and ending positions in the character sequence of the text message, and Respectively represent word units node q The starting and ending positions in the character sequence of the text information. p And word unit node q The semantic dependency relationship between sdp The relational knowledge representation of is integrated into the knowledge representation matrix M of the semantic dependency relationship sdp The relational knowledge representation is extended from the word unit level to the token level.

[0115] For example, the semantic association information of text information is a semantic dependency parsing tree (SDPT) as an example, the semantic dependency parsing tree (T sdp It can be expressed as: T sdp ={Root sdp ,Node sdp ,Edge sdp}, where Root sdp Represents the root of the semantic dependency parse tree. sdp A set of nodes representing a semantic dependency analysis tree, where each node is a word unit (text fragment), and a word unit may contain more than one character. sdp Represents the edge set of the semantic dependency analysis tree. Each edge can be abstractly represented as {node p ,node q ,r sdp}, indicating word unit node p And word unit node q There is a semantic dependency relationship between sdp . r sdp ∈R sdp , R sdp represents the set of all possible semantic dependency relationships. In this step, the semantic dependency analysis tree T based on the text information sdp , for each edge {node p ,node q ,r sdp}, generate semantic dependency relationship r sdp The corresponding sentence-level knowledge representation It is possible to obtain sentence-level knowledge representations corresponding to multiple different edges.

[0116] Furthermore, the same semantic dependency r sdp The corresponding knowledge representation of all sentences Find the average and get the semantic dependency r sdp Relational knowledge representation

[0117] Furthermore, building a Matrix M sdp , semantic dependency analysis tree T based on text information sdp , for any edge {node p ,node q ,r sdp},when as well as Sometimes, there is That is, the matrix M sdp The value at position (i,j) is set to in, and Respectively represent word units node p The starting and ending positions in the character sequence of the text message, and Respectively represent word units node q The starting and ending positions in the character sequence of the text information. p And word unit node q The semantic dependency relationship between sdp Relational knowledge representation The knowledge representation matrix M that incorporates semantic dependencies sdp middle.

[0118] For example, based on Figure 4 The semantic dependency analysis tree shown in Figure 6 The knowledge representation matrix M of the semantic dependency relationship shown sdp . Figure 4 Medium A B C D The semantic dependency relationship corresponding to each edge in the semantic dependency analysis tree is represented by the edge {“West Lake”, “Located in”, B}, the starting and ending positions of "West Lake" in the text information are 1 and 2 respectively, and the starting and ending positions of "located in" in the text information are 3 and 4 respectively. sdp The first row and the 3rd-4th columns of the second row are set as semantic dependencies. B The corresponding relational knowledge representation, Figure 6 Fill in the 3rd to 4th columns of row 1 and row 2 B , indicating that the corresponding position is filled with semantic dependencies B Corresponding relational knowledge representation. For the edge {"Hangzhou","located in", C}, the starting and ending positions of "Hangzhou" in the text information are 5 and 6 respectively. sdp The positions of the 5th row and the 3rd-4th column of the 6th row are set as semantic dependencies.C The corresponding relational knowledge representation, Figure 6 Fill in the 3rd to 4th columns of rows 5 and 6 C , indicating that the corresponding position is filled with semantic dependencies C Corresponding relational knowledge representation. For the edge {“.”, “located in”, D}, "." in the text information starts and ends at position 7, in the knowledge representation matrix M sdp The 3rd and 4th columns of the 7th row are set as the semantic dependency D The corresponding relational knowledge representation, Figure 6 Fill in the 3rd to 4th columns of row 7 D , indicating that the corresponding position is filled with semantic dependencies D Corresponding relational knowledge representation.

[0119] Compared with directly mapping syntactic structure information and / or semantic association information into embedded vectors, the method of this embodiment performs meaningful knowledge representation of the syntactic structure information and / or semantic association information of text information, and the obtained syntactic structure knowledge representation contains finer-grained syntactic structure knowledge of the text information, and the semantic association knowledge representation contains finer-grained semantic association knowledge of the text information, which can well represent the syntactic structure knowledge and semantic association knowledge of the text information.

[0120] In an optional embodiment, in the aforementioned step S202, the grammatical and semantic information is integrated into the text representation of the text information through the information extraction model to obtain an enhanced representation of the text information, which can be specifically implemented in the following manner: through the attention module of the information extraction model, a key matrix for self-attention calculation is determined according to the knowledge representation of the grammatical and semantic information, and self-attention calculation is performed on the text representation of the text information according to the key matrix to obtain an enhanced representation of the text information.

[0121] Optionally, three different weight matrices are used to perform a linear transformation on the text embedding of the text information to obtain a query matrix Q1, a key matrix K1, and a value matrix V1; self-attention calculation is performed based on the query matrix Q1, the key matrix K1, and the value matrix V1 to obtain a first self-attention representation.

[0122] If only the syntactic structure knowledge representation of the text information is obtained, and the semantic association knowledge representation is not obtained, the syntactic structure knowledge representation is used as the key matrix K2, and self-attention calculation is performed based on the query matrix Q1, the key matrix K2 and the value matrix V1 to obtain the second self-attention representation. The first self-attention representation and the second self-attention representation are weighted and summed to obtain an enhanced representation of the text information. In addition, the first self-attention representation and the second self-attention representation can also be directly summed to obtain an enhanced representation of the text information. Among them, the weight coefficients corresponding to the first self-attention representation and the second self-attention representation can be configured according to the needs of the actual application scenario, and are not specifically limited here.

[0123] If only the semantic association knowledge representation of the text information is obtained, and the syntactic structure knowledge representation is not obtained, the semantic association knowledge representation is used as the key matrix K3, and self-attention calculation is performed based on the query matrix Q1, the key matrix K3 and the value matrix V1 to obtain the third self-attention representation. The first self-attention representation and the third self-attention representation are weighted and summed to obtain an enhanced representation of the text information. In addition, the first self-attention representation and the third self-attention representation can also be directly summed to obtain an enhanced representation of the text information. Among them, the weight coefficients corresponding to the first self-attention representation and the third self-attention representation can be configured according to the needs of the actual application scenario, and are not specifically limited here.

[0124] If the syntactic structure knowledge representation and semantic association knowledge representation of the text information are obtained at the same time, the syntactic structure knowledge representation is used as the key matrix K2, and self-attention calculation is performed based on the query matrix Q1, the key matrix K2 and the value matrix V1 to obtain the second self-attention representation; the semantic association knowledge representation is used as the key matrix K3, and self-attention calculation is performed based on the query matrix Q1, the key matrix K3 and the value matrix V1 to obtain the third self-attention representation; the first self-attention representation is weighted and summed with the second self-attention representation and the third self-attention representation to obtain the enhanced representation of the text information. In addition, the first self-attention representation can also be directly summed with the second self-attention representation and the third self-attention representation to obtain the enhanced representation of the text information. Among them, the weight coefficients corresponding to the first self-attention representation and the third self-attention representation can be configured according to the needs of the actual application scenario, and are not specifically limited here.

[0125] For example, the process of self-attention calculation based on the self-attention mechanism can be expressed as: Attn = softmax(Q·K T )·V. Based on the self-attention mechanism, the first self-attention representation Attn is calculated based on the query matrix Q1, key matrix K1 and value matrix V1 determined according to the text information: Attn = softmax(Q1·K1 T)·V1. The syntactic structure knowledge representation is used as the key matrix K2, and the second self-attention representation Attn is calculated based on the query matrix Q1, the key matrix K2 and the value matrix V1. dep :Attn dep =softmax(Q1·K2 T )·V1. The semantic association knowledge representation is used as the key matrix K3, and the third self-attention representation Attn is calculated based on the query matrix Q1, the key matrix K3 and the value matrix V1. sdp :Attn sdp =softmax(Q1·K3 T )·V1. Further, based on the weight vector W, by Attn * =W·[Attn,Attn sdp ,Attn dep ] T , the first self-attention representation Attn and the second self-attention representation Attn dep and the third self-attention representation Attn sdp Weighted summation to obtain the enhanced representation Attn * . Where W is a three-dimensional vector composed of three weight coefficients, and W is a learnable parameter of the information extraction model, which is determined after training optimization.

[0126] In a preferred embodiment, when calculating the second self-attention representation according to the syntactic structure knowledge representation and calculating the third self-attention representation according to the semantic association knowledge representation, a weighted self-attention machine (a variant of the self-attention mechanism) can be used. The weighted self-attention calculation process of the weighted self-attention machine (a variant of the self-attention mechanism) can be expressed as: Attn ′ =softmax((Q+v)·K T )·V, where v is the weight parameter, which is a learnable parameter.

[0127] For example, the syntactic structure knowledge is represented as M dep As the key matrix K2, when performing self-attention calculation based on the query matrix Q1, the key matrix K2 and the value matrix V1, a weighted self-attention mechanism (a variant of the self-attention mechanism) can be used to perform weighted self-attention calculation based on the query matrix Q1, the key matrix K2 and the value matrix V1 to obtain a second self-attention representation. Specifically, the method of calculating the second self-attention representation using the weighted self-attention machine (a variant of the self-attention mechanism) is: in, M is the syntactic structure knowledge representation dep The transposed matrix, v dep It is a weight parameter related to syntactic structure knowledge and is a learnable parameter that is determined after training and optimization.

[0128] For example, the semantic association knowledge is represented as sdp As the key matrix K3, when performing self-attention calculation based on the query matrix Q1, the key matrix K3 and the value matrix V1, a weighted self-attention mechanism (a variant of the self-attention mechanism) can be used to perform weighted self-attention calculation based on the query matrix Q1, the key matrix K3 and the value matrix V1 to obtain a third self-attention representation. Specifically, the method of calculating the third self-attention representation using the weighted self-attention machine (a variant of the self-attention mechanism) is: in, is the knowledge representation M of the semantic dependency relationship sdp The transposed matrix, v sdp It is a weight parameter related to semantic knowledge and is a learnable parameter that is determined after training and optimization.

[0129] In this preferred embodiment, three different weight matrices are used to perform a linear transformation on the text embedding of the text information to obtain a query matrix Q1, a key matrix K1, and a value matrix V1; a self-attention calculation is performed based on the query matrix Q1, the key matrix K1, and the value matrix V1 to obtain the first self-attention representation: Attn = softmax (Q1 K1 T )·V1. The syntactic structure knowledge representation is used as the key matrix K2, and a weighted self-attention calculation is performed based on the query matrix Q1, the key matrix K2, and the value matrix V1 to obtain the second self-attention representation: The semantic association knowledge representation is used as the key matrix K3, and the weighted self-attention calculation is performed based on the query matrix Q1, the key matrix K3 and the value matrix V1 to obtain the third self-attention representation: The first self-attention representation is weighted summed with the second self-attention representation and the third self-attention representation to obtain the enhanced representation: Attn * =W·[Attn,Attn sdp ,Attn dep ] T .

[0130] For example, Figure 7 A framework diagram of an information extraction method provided for an exemplary embodiment of the present application. Figure 7 In the given text information "West Lake is located in Hangzhou.", the overall framework of the information extraction method is illustrated as follows: Figure 7As shown in the figure, for a given text information "West Lake is located in Hangzhou.", it can be represented as the following character sequence: {"West", "Lake", "Location", "Yu", "Hangzhou", "State", "."}. The character sequence is mapped into an embedding vector (Embedding) and input into the information extraction model. Based on the character sequence of the text information, the dependency syntactic analysis tree DEPT and the semantic dependency analysis tree SDPT of the text information are generated. Based on the dependency syntactic analysis tree DEPT, the syntactic dependency relationship is represented to obtain the syntactic structure knowledge representation M dep , based on the semantic dependency analysis tree SDPT, the semantic dependency relationship is represented to obtain the semantic association knowledge representation M sdp . Further, based on the self-attention mechanism, the original text representation (embedding vector) of the character sequence is self-attention calculated to obtain the first self-attention representation. Based on the weighted self-attention mechanism (variant self-attention mechanism, Variant Self-Attention), the syntactic structure knowledge representation M dep The text embedding of the text information is used to perform weighted self-attention calculation to obtain the second self-attention representation. Based on the weighted self-attention mechanism (variant self-attention mechanism, Variant Self-Attention), the semantic association knowledge representation M sdp The text embedding of the text information is used to perform weighted self-attention calculation to obtain the third self-attention representation. The first self-attention representation is fused with the second self-attention representation and the third self-attention representation (weighted summation) to obtain the enhanced representation H. The enhanced representation H is used as the input of the output layer, and information is extracted according to the enhanced representation H through the output layer to obtain the information extraction result. Among them, for the named entity recognition (NER) task, the output layer can use the CRF (conditional random field) model, and use CRF to implement named entity recognition to obtain the named entity recognition result. For the relation extraction (RE) task, the output layer can use a relation classifier, and use the relation classifier for classification and recognition to obtain the relation extraction result.

[0131] The method of this embodiment provides a knowledge fusion framework for information extraction. On the basis of a transformer-based information extraction model (such as a large model or a small-scale model), the language and semantic knowledge obtained from text information can be better represented and injected into the information extraction model. The relational knowledge representation is integrated into the text representation of the text information through the variant's self-attention mechanism to obtain an enhanced representation. The information extraction result is output based on the enhanced representation through the output layer, so that the information extraction model can better complete the information extraction task and improve the accuracy of information extraction.

[0132] In an optional embodiment, when training the information extraction model, for any input text, the syntactic dependency between any two characters in the input text predicted by the information extraction model is determined through back propagation based on the enhanced representation of the input text; a first loss is calculated based on the syntactic dependency between any two characters predicted by the model and the syntactic dependency between any two characters contained in the syntactic structure knowledge representation of the input text. The first loss is used to optimize the parameters of the information extraction model, which can verify whether the model has truly captured the injected syntactic dependency knowledge, forcing the information extraction model to learn the relational knowledge related to the injected syntactic structure.

[0133] In an optional embodiment, when training the information extraction model, for any training text, the semantic dependency between any two characters in the training text predicted by the information extraction model is determined through back propagation based on the enhanced representation of the input text; based on the semantic dependency between any two characters predicted by the model and the semantic dependency between any two characters contained in the semantic association knowledge representation of the training text, a second loss is calculated. The second loss is used to optimize the parameters of the information extraction model, which can verify whether the model has truly captured the injected semantic dependency knowledge, forcing the information extraction model to learn the injected semantically related relationship knowledge.

[0134] Figure 8 The following is a flow chart of a training method for an information extraction model provided by an exemplary embodiment of the present application. Figure 8 As shown in Figure 1, the training process of the information extraction model is as follows:

[0135] Step S801: Analyze the training texts in the training set to obtain grammatical and semantic information of the training texts.

[0136] The grammatical and semantic information of the training text includes at least one of the syntactic structure information and semantic association information of the training text.

[0137] In an example usage scenario, only the syntactic structure information of the training text can be obtained and injected into the information extraction model. After the model training is completed, when performing information extraction, for a given text information to be processed, in the aforementioned information extraction method embodiment, only the syntactic structure information of the text information is obtained, and in the subsequent steps, the syntactic structure information of the text information is input into the information extraction model, and the syntactic structure information of the text information is integrated into the text representation of the text information to obtain an enhanced representation.

[0138] In another example usage scenario, only the semantic association information of the training text can be obtained and injected into the information extraction model. After the model training is completed, when performing information extraction, for a given text information to be processed, in the aforementioned information extraction method embodiment, only the semantic association information of the text information is obtained, and in the subsequent steps, the semantic association information of the text information is input into the information extraction model, and the semantic association information of the text information is integrated into the text representation of the text information to obtain an enhanced representation.

[0139] In another example usage scenario, the syntactic structure information and semantic association information of the training text can be obtained at the same time, and the syntactic structure information and semantic association information can be injected into the information extraction model. After the model training is completed, when performing information extraction, for a given text information to be processed, in the aforementioned information extraction method embodiment, the syntactic structure information and semantic association information of the text information are obtained, and in subsequent steps, the syntactic structure information and semantic association information of the text information are input into the information extraction model, and the syntactic structure information and semantic association information of the text information are integrated into the text representation of the text information to obtain an enhanced representation. This is a preferred usage scenario, which can simultaneously integrate language and semantic knowledge into the information extraction model, which can improve the accuracy of information extraction.

[0140] The specific implementation method of this step is similar to the implementation method of the aforementioned step S201. Please refer to the relevant content in the aforementioned embodiment for details, and will not be repeated here.

[0141] Step S802: input the grammatical and semantic information of the training text into the information extraction model, integrate the grammatical and semantic information into the text representation of the training text through the information extraction model, obtain an enhanced representation of the training text, extract information based on the enhanced representation, and obtain an information extraction result.

[0142] The specific implementation method of this step is similar to the implementation method of step S202. Please refer to the relevant content in the above embodiment for details, which will not be repeated here.

[0143] Step S803: According to the enhanced representation of the training text, determine the syntactic dependency between any two characters in the training text predicted by the information extraction model through back propagation.

[0144] In this embodiment, by back propagation, according to the enhanced representation H of the obtained training text, a linear transformation is performed on the enhanced representation H (multiplied by two different preset linear transformation matrices) to obtain H head and H tail , H head Indicates the information of the first word unit (text fragment) in the syntactic dependency relationship, H tail Represents the information of the second word unit (text fragment) in the syntactic dependency. Get the reproduction knowledge representation matrix generated after the model learns to inject knowledge Among them, the two different preset linear transformation matrices can be set according to the needs and experience of actual application scenarios, and are not specifically limited here.

[0145] Furthermore, for syntactic structure information, by reproducing the knowledge representation matrix A linear transformation is implemented on the syntactic dependency relationship (multiplied by the preset second linear transformation matrix) to obtain the reproduction knowledge representation matrix of the syntactic dependency relationship. Among them, the value at any position (i,j) The syntactic dependency relationship between the i-th character and the j-th character learned by the model is represented. The preset second linear transformation matrix can be set according to the needs and experience of the actual application scenario, and is not specifically limited here.

[0146] Step S804: Calculate a first loss based on the syntactic dependency between any two characters predicted by the model and the syntactic dependency between any two characters included in the syntactic structure knowledge representation of the training text.

[0147] In this step, the knowledge representation matrix based on syntactic dependencies is reproduced The value at any position (i,j) represents the syntactic dependency between the i-th character and the j-th character learned by the model. The actual syntactic dependency between the i-th character and the j-th character is r dep .Will Through multi-classification using the softmax classifier, we can find that there is a syntactic dependency relationship r between the i-th character and the j-th character. dep The estimated probability of (i,j) (r dep |C). Among them, there is a syntactic dependency relationship r between the i-th character and the j-th character in the training text. dep , we can represent the knowledge matrix M based on the syntactic dependency of the training text dep Sure.

[0148] Furthermore, the first loss L is calculated as follows: dep (Θ): n represents the number of characters in the training text. (i,j) (r dep |C), use other variant formulas to calculate the first loss L dep (Θ), the specific formula for calculating the first loss is not specifically limited here.

[0149] Step S805: According to the enhanced representation of the training text, the semantic dependency between any two characters in the training text predicted by the information extraction model is determined through back propagation.

[0150] For semantically related information, by reproducing the knowledge representation matrix A linear transformation is implemented on the y-values ​​(multiplied by the preset third linear transformation matrix) to obtain the reproduction knowledge representation matrix of the semantic dependency relationship. Among them, the value at any position (i,j) The semantic dependency relationship between the i-th character and the j-th character learned by the representation model. The preset third linear transformation matrix and the second linear transformation matrix are different linear transformation matrices, which can be set according to the needs and experience of the actual application scenario and are not specifically limited here.

[0151] Step S806: Calculate the second loss based on the semantic dependency relationship between any two characters predicted by the model and the semantic dependency relationship between any two characters included in the semantic association knowledge representation of the training text.

[0152] In this step, the reproduction knowledge representation matrix based on semantic dependencies The value at any position (i,j) represents the semantic dependency relationship between the i-th character and the j-th character learned by the model. The actual semantic dependency relationship between the i-th character and the j-th character is r sdp .Will Through multi-classification using the softmax classifier, we can find that there is a semantic dependency relationship r between the i-th character and the j-th character. sdp The estimated probability of (i,j) (r sdp |C). Among them, there is a semantic dependency relationship r between the i-th character and the j-th character in the training text. sdp , we can represent the knowledge matrix M based on the semantic dependency relationship of the training text sdp Sure.

[0153] Furthermore, the second loss L is calculated as follows: sdp (Θ): n represents the number of characters in the training text. (i,j) (r sdp |C), use other variant formulas to calculate the second loss L sdp (Θ), the specific formula for calculating the second loss is not specifically limited here.

[0154] Step S807: Calculate the task loss based on the information extraction results and annotation results of the training text.

[0155] In this step, the cross entropy loss is calculated based on the information extraction results and annotation results of the training text as the task loss of the information extraction task. In addition, this step can also be implemented by any method of calculating the information extraction task loss during the information extraction model training process, which is not specifically limited here.

[0156] Step S808: Optimize the parameters of the information extraction model according to the task loss, the first loss and the second loss.

[0157] In this step, the overall loss is calculated comprehensively based on the task loss, the first loss and the second loss. According to the overall loss, the parameters of the information extraction model are optimized to realize the training of the information extraction model.

[0158] Optionally, the sum of the task loss, the first loss and the second loss can be used as the overall loss. Optionally, the task loss, the first loss and the second loss can be weighted and summed according to preset weight coefficients corresponding to the respective losses to obtain the overall loss.

[0159] Exemplarily, the overall loss L(Θ) can be calculated as follows: L(Θ)=L task (Θ)+λ sdp ·L sdp (Θ)+λ dep ·L dep (θ). Where L task (Θ) represents the task loss. L sdp (Θ) represents the second loss determined for semantic dependencies, λ sdp Indicates the preset weight coefficient corresponding to the second loss. dep (Θ) represents the first loss determined for syntactic dependencies, λ dep Represents the preset weight coefficient corresponding to the first loss. sdp and λ dep It can be set according to the actual application scenario and experience value, such as λ sdp =e -5 ,λ dep =e -5 , this embodiment is not specifically limited here.

[0160] For example, Fig. 9 The framework diagram of the information extraction model training provided in this embodiment is as follows: Fig. 9 The training text "West Lake is located in Hangzhou" is used as an example to illustrate the overall framework of the information extraction method. Fig. 9As shown in the figure, the training text "West Lake is located in Hangzhou." can be represented as the following character sequence: {"West", "Lake", "Location", "Yu", "Hangzhou", "State", "."}. The character sequence is mapped into an embedding vector (Embedding) and input into the information extraction model. Based on the character sequence of the training text, the dependency syntactic analysis tree DEPT and semantic dependency analysis tree SDPT of the training text are generated. Based on the dependency syntactic analysis tree DEPT, the syntactic dependency relationship is represented to obtain the syntactic structure knowledge representation M dep , based on the semantic dependency analysis tree SDPT, the semantic dependency relationship is represented to obtain the semantic association knowledge representation M sdp . Further, based on the self-attention mechanism, the original text representation (embedding vector) of the character sequence is self-attention calculated to obtain the first self-attention representation. Based on the weighted self-attention mechanism (variant self-attention mechanism, Variant Self-Attention), the syntactic structure knowledge representation M dep The text embedding of the training text is used to perform weighted self-attention calculation to obtain the second self-attention representation. Based on the weighted self-attention mechanism (variant self-attention mechanism, Variant Self-Attention), the semantic association knowledge representation M sdp The text embedding of the training text is used to perform weighted self-attention calculation to obtain the third self-attention representation. The first self-attention representation is weighted summed with the second self-attention representation and the third self-attention representation to obtain the enhanced representation H. The enhanced representation H is used as the input of the output layer, and information is extracted according to the enhanced representation H through the output layer to obtain the information extraction result. Further, two different linear transformations are performed on the enhanced representation H to obtain H head and H tail , and then calculate the reproduction knowledge representation matrix generated after the model learning injects knowledge In the reproduction of knowledge representation matrix Perform two different linear transformations on the syntactic dependency relationship to obtain the reproduction knowledge representation matrix and the reproduction knowledge representation matrix of semantic dependencies based on and M dep Calculate the first loss L dep ,based on and M sdp Calculate the second loss L sdp The first loss L dep and the second loss L sdp Used to optimize the parameters of the information extraction model.

[0161] It should be noted that, in this embodiment, the syntactic structure information and semantic association information of the training text are used simultaneously as an example for exemplary description. If only the syntactic structure information of the training text is used, the aforementioned steps S803-S804 are executed to calculate the first loss, and there is no need to execute the aforementioned steps S805-S806 to calculate the second loss. In step S808, the overall loss is determined according to the task loss and the first loss, and the parameters of the information extraction model are optimized based on the overall loss to achieve the training of the information extraction model. If only the semantic association information of the training text is used, the aforementioned steps S805-S806 are executed to calculate the second loss, and there is no need to execute the aforementioned steps S803-S804 to calculate the first loss. In step S808, the overall loss is determined according to the task loss and the second loss, and the parameters of the information extraction model are optimized based on the overall loss to achieve the training of the information extraction model.

[0162] Among them, for the named entity recognition (NER) task, the output layer can use the CRF (conditional random field) model, use CRF to implement named entity recognition, and obtain the named entity recognition result. For the relation extraction (RE) task, the output layer can use the relation classifier, use the relation classifier to perform classification and recognition, and obtain the relation extraction result.

[0163] In practical applications, the information extraction model can be iterated and optimized for multiple rounds based on multiple training texts in the training set until the convergence condition is met. The convergence condition can be that the number of iterations reaches a preset threshold, the performance of the model is relatively stable, etc. The convergence condition can be set and adjusted according to the actual application scenario and experience, and is not specifically limited here.

[0164] In this embodiment, by adding the first loss and the second loss, it is possible to test whether the model has truly captured the injected syntactic dependency-related knowledge and semantic dependency-related knowledge, forcing the information extraction model to learn the injected relational knowledge, thereby improving the expressive power of the information extraction model and improving the accuracy of information extraction.

[0165] Fig.10 This is a flowchart of a named entity recognition method provided by an exemplary embodiment of the present application. The execution subject of this embodiment is the server in the aforementioned system architecture. Fig.10 As shown, the specific steps of the method are as follows:

[0166] Step S1001: In response to a call request from a terminal device to a named entity recognition interface, obtain text information to be processed.

[0167] In this embodiment, the server runs a model for implementing the named entity recognition task, which is called a named entity recognition model. The server provides a service interface for implementing the named entity recognition task, which is called a named entity recognition interface. The user can trigger the server to perform the corresponding named entity recognition task and return the named entity recognition result by calling the named entity recognition interface through the terminal device.

[0168] The end-side device calls the named entity recognition interface, that is, the end-side device sends a call request for the named entity recognition interface to the server, and the call request carries the text information to be processed. The server receives the call request for the named entity recognition interface and obtains the text information to be processed.

[0169] Step S1002: Analyze the text information to be processed to obtain grammatical and semantic information of the text information.

[0170] The specific implementation method of this step is the same as the specific implementation method of the aforementioned step S201. Please refer to the relevant content in the aforementioned embodiment for details, and will not be repeated here.

[0171] Step S1003: input the text information and the grammatical semantic information of the text information into a named entity recognition model, integrate the grammatical semantic information into the text representation of the text information through the named entity recognition model, and obtain an enhanced representation of the text information; perform named entity recognition based on the enhanced representation to obtain the named entities contained in the text information.

[0172] In this step, the text information and the grammatical and semantic information of the text information are input into the named entity recognition model, and the grammatical and semantic information is integrated into the text representation of the text information through the named entity recognition model to obtain an enhanced representation of the text information. This is similar to the above-mentioned step S202 in which the grammatical and semantic information of the text information is input into the information extraction model, and the grammatical and semantic information is integrated into the text representation of the text information through the information extraction model to obtain an enhanced representation of the text information. The specific implementation method is similar, except that the information extraction model used is specifically a named entity recognition model. For the specific implementation, please refer to the relevant content of step S202 in the above-mentioned embodiment, which will not be repeated here.

[0173] In this step, the output layer of the named entity recognition model can use a conditional random field (CRF) model. The conditional random field (CRF) model is used to mark the start and end positions of the named entity in the character sequence of the input text information. For example, the RIOES annotation set can be used to annotate each character in the character sequence, with B indicating that the character is at the beginning (Begin) position of an entity, I indicating that the character is inside (Inside) of an entity, O indicating that the character is outside (Outside) of an entity, E indicating that the character is at the end (End) position of an entity, and S indicating that the character itself can form an entity (Single).

[0174] In addition, the output layer of the named entity recognition model can also be implemented using an output layer structure of any existing model for named entity recognition that generates named entity recognition results based on text representation, and no specific limitation is made here.

[0175] Step S1004: Return the named entities contained in the text information to the terminal device.

[0176] The method of this embodiment obtains grammatical and semantic information of text information, and the grammatical and semantic information includes language semantic knowledge of the text information. By injecting the grammatical and semantic information of the text information into the named entity recognition model, the named entity recognition model can better learn the language semantic knowledge, integrate the grammatical and semantic information into the text representation of the text information, obtain the enhanced representation of the text information, and realize named entity recognition, which can effectively reduce or even avoid the loss of key information and improve the accuracy of named entity recognition.

[0177] Fig.11 This is a flowchart of a relationship extraction method provided by an exemplary embodiment of the present application. The execution subject of this embodiment is the server in the aforementioned system architecture. Fig.11 As shown, the specific steps of the method are as follows:

[0178] Step S1101: In response to a call request from a terminal device to a relationship extraction interface, obtain text information to be processed and location information of two text segments whose relationship is to be determined in the text information.

[0179] In this embodiment, the server runs a model for implementing the relationship extraction task, which is called a relationship extraction model. The server provides a service interface for implementing the relationship extraction task, which is called a relationship extraction interface. The user can trigger the server to execute the corresponding relationship extraction task and return the relationship extraction result by calling the relationship extraction interface through the terminal device.

[0180] The end-side device calls the relationship extraction interface, that is, the end-side device sends a call request for the relationship extraction interface to the server, and the call request carries the text information to be processed and the location information of two text segments whose relationship is to be determined in the text information. The server receives the call request for the relationship extraction interface and obtains the text information to be processed and the location information of the two text segments whose relationship is to be determined in the text information.

[0181] Step S1102: Analyze the text information to be processed to obtain grammatical and semantic information of the text information.

[0182] The specific implementation method of this step is the same as the specific implementation method of the aforementioned step S201. Please refer to the relevant content in the aforementioned embodiment for details, and will not be repeated here.

[0183] Step S1103: input the text information, the position information of the two text fragments, and the grammatical semantic information of the text information into the relationship extraction model, and integrate the grammatical semantic information into the text representation of the text information through the relationship extraction model to obtain an enhanced representation of the text information; perform relationship classification based on the enhanced representation to obtain the relationship category between the two text fragments.

[0184] In this step, the text information, the position information of the two text segments, and the grammatical semantic information of the text information are input into the relation extraction model. Specifically, the embedding vector of the text information and the position encoding of the position information of the two text segments are concatenated as the initial text representation of the text information.

[0185] In this step, the grammatical and semantic information is integrated into the text representation of the text information through the relation extraction model to obtain an enhanced representation of the text information. This is similar to the above-mentioned step S202 in which the grammatical and semantic information of the text information is input into the information extraction model, and the grammatical and semantic information is integrated into the text representation of the text information through the information extraction model to obtain an enhanced representation of the text information. The specific implementation method is similar, except that the information extraction model used is specifically a relation extraction model. For the specific implementation, please refer to the relevant content of step S202 in the above-mentioned embodiment, which will not be repeated here.

[0186] In this step, the output layer of the relation extraction model can be implemented using a relation classifier. The relation classifier is used to classify and identify the relation category between two specified text segments in the character sequence of the input text information. The relation classifier can be a multi-classification classifier, and can be implemented using a classifier used in any existing relation extraction model, which is not specifically limited here.

[0187] Step S1104: Return the relationship category between the two text segments to the terminal device.

[0188] The method of this embodiment obtains grammatical and semantic information of text information, and the grammatical and semantic information includes language semantic knowledge of the text information. By injecting the grammatical and semantic information of the text information into the relationship extraction model, the relationship extraction model can better learn the language knowledge and / or semantic knowledge, integrate the grammatical and semantic information into the text representation of the text information, obtain the enhanced representation of the text information, and realize relationship extraction, which can effectively reduce or even avoid the loss of key information and improve the accuracy of relationship extraction.

[0189] Fig.12 A schematic diagram of the structure of a cloud server provided in an embodiment of the present application. Fig.12 As shown, the cloud server includes: a memory 1201 and a processor 1202. The memory 1201 is used to store computer-executable instructions and can be configured to store various other data to support operations on the cloud server. The processor 1202 is connected to the memory 1201 in communication and is used to execute the computer-executable instructions stored in the memory 1201 to implement the technical solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are similar and will not be repeated here.

[0190] Optional, such as Fig.12 As shown, the cloud server also includes: a firewall 1203, a load balancer 1204, a communication component 1205, a power supply component 1206 and other components. Fig.12 The following only shows some components, which does not mean that the cloud server only includes Fig.12 Components shown.

[0191] An embodiment of the present application also provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the technical solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0192] An embodiment of the present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium. At least one processor of the cloud server can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the cloud server executes the technical solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0193] The embodiment of the present application provides a chip, including: a processing module and a communication interface, the processing module can execute the technical solution of the cloud server in the aforementioned method embodiment. Optionally, the chip also includes a storage module (such as a memory), the storage module is used to store instructions, the processing module is used to execute the instructions stored in the storage module, and the execution of the instructions stored in the storage module enables the processing module to execute the technical solution provided by any of the aforementioned method embodiments.

[0194] The above-mentioned memory can be an object storage service (OSS). The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a mobile hotspot (WiFi), a second generation mobile communication system (2G), a third generation mobile communication system (3G), a fourth generation mobile communication system (4G) / long term evolution (LTE), a fifth generation mobile communication system (5G) and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies. The above-mentioned power supply component provides power to various components of the device where the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing and distributing power for the device where the power supply component is located.

[0195] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0196] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0197] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. Memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium. Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0198] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0199] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and are only used to distinguish between different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit "first" and "second" to different types. The meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0200] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0201] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A text information extraction method, characterized in that: include: Analyze the text information to be processed to obtain grammatical and semantic information of the text information; The grammatical and semantic information of the text information is input into the information extraction model, and the grammatical and semantic information is integrated into the text representation of the text information through the information extraction model to obtain an enhanced representation of the text information, and information extraction is performed according to the enhanced representation to obtain an information extraction result.

2. The method according to claim 1, characterized in that The step of analyzing the text information to be processed to obtain grammatical and semantic information of the text information includes: Performing syntactic analysis on the text information to obtain syntactic structure information of the text information; and / or, Perform semantic analysis on the text information to obtain semantic association information of the text information.

3. The method according to claim 2, characterized in that The performing syntactic analysis on the text information to obtain syntactic structure information of the text information includes: A dependency syntactic analysis is performed on the text information to obtain a dependency syntactic analysis tree, wherein the dependency syntactic analysis tree includes syntactic structure information of the text information.

4. The method according to claim 2, characterized in that: The performing semantic analysis on the text information to obtain semantic association information of the text information includes: A semantic dependency analysis is performed on the text information to obtain a semantic dependency analysis tree, wherein the semantic dependency analysis tree contains semantic association information of the text information.

5. The method according to any one of claims 1 to 4, characterized in that The step of inputting the grammatical and semantic information of the text information into an information extraction model comprises: Representing the grammatical and semantic information of the text information to obtain corresponding knowledge representation; The knowledge representation is input into the information extraction model.

6. The method according to claim 5, characterized in that The grammatical semantic information includes syntactic structure information, and the characterizing the grammatical semantic information of the text information to obtain the corresponding knowledge characterization includes: The text information includes a plurality of word units, and based on the syntactic structure information of the text information, for any two word units having a syntactic dependency relationship, a concatenated text of the text information and the two word units having a syntactic dependency relationship is encoded to obtain a knowledge representation of the syntactic dependency relationship; According to the knowledge representation of the same syntactic dependency relationship, determining the relational knowledge representation of the syntactic dependency relationship; According to the relational knowledge representations of various syntactic dependency relationships contained in the text information, a knowledge representation matrix of syntactic dependency relationships is constructed to obtain a syntactic structure knowledge representation.

7. The method according to claim 5, characterized in that The grammatical semantic information includes semantic association information, and the characterizing the grammatical semantic information of the text information to obtain the corresponding knowledge characterization includes: The text information includes a plurality of word units, and based on the semantic association information of the text information, for any two word units having a semantic dependency relationship, a concatenated text of the text information and the two word units having a semantic dependency relationship is encoded to obtain a knowledge representation of the semantic dependency relationship; According to the knowledge representation of the same semantic dependency relationship, determining the relational knowledge representation of the semantic dependency relationship; According to the relational knowledge representations of various semantic dependency relationships contained in the text information, a knowledge representation matrix of semantic dependency relationships is constructed to obtain semantic association knowledge representations.

8. The method according to claim 5, characterized in that The step of integrating the grammatical and semantic information into the text representation of the text information through the information extraction model to obtain an enhanced representation of the text information includes: Through the attention module of the information extraction model, a key matrix for self-attention calculation is determined according to the knowledge representation, and self-attention calculation is performed on the text representation of the text information according to the key matrix to obtain an enhanced representation of the text information.

9. The method according to claim 6, characterized in that Also includes: When training the information extraction model, for any input text, according to the enhanced representation of the input text, the syntactic dependency between any two characters in the input text predicted by the information extraction model is determined by back propagation; A first loss is calculated based on the syntactic dependency between any two characters predicted by the model and the syntactic dependency between any two characters contained in the syntactic structure knowledge representation of the input text, and the first loss is used to optimize the parameters of the information extraction model.

10. The method according to claim 7, characterized in that Also includes: When training the information extraction model, for any training text, according to the enhanced representation of the input text, the semantic dependency relationship between any two characters in the training text predicted by the information extraction model is determined by back propagation; A second loss is calculated based on the semantic dependency between any two characters predicted by the model and the semantic dependency between any two characters contained in the semantic association knowledge representation of the training text, and the second loss is used to optimize the parameters of the information extraction model.

11. The method according to any one of claims 1 to 4, characterized in that: After obtaining the grammatical and semantic information of the text information, the method further includes: Outputting grammatical and semantic information of the text information through a front-end interface; In response to an editing operation on the grammatical and semantic information in the front-end interface, the grammatical and semantic information is updated.

12. A method for named entity recognition, characterized in that: include: In response to a call request from the end-side device to the named entity recognition interface, obtain text information to be processed; Analyze the text information to be processed to obtain grammatical and semantic information of the text information; Inputting the text information and the grammatical semantic information of the text information into a named entity recognition model, and integrating the grammatical semantic information into the text representation of the text information through the named entity recognition model to obtain an enhanced representation of the text information; Performing named entity recognition according to the enhanced representation to obtain the named entities contained in the text information; The named entities contained in the text information are returned to the terminal device.

13. A relationship extraction method, characterized in that: include: In response to a call request from the terminal device to the relationship extraction interface, obtaining text information to be processed and position information of two text segments whose relationship is to be determined in the text information; Analyze the text information to be processed to obtain grammatical and semantic information of the text information; Inputting the text information, the position information of the two text segments, and the grammatical semantic information of the text information into a relation extraction model, and integrating the grammatical semantic information into the text representation of the text information through the relation extraction model to obtain an enhanced representation of the text information; Performing relationship classification according to the enhanced representation to obtain a relationship category between the two text segments; The relationship category between the two text segments is returned to the client device.

14. A cloud server, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 13.

Images