Entity recognition model training methods, entity recognition methods, devices and related equipment

Abstract

This application discloses an entity recognition model training method, entity recognition method, apparatus, and related equipment. By performing walk sampling from a knowledge graph, at least one walk path composed of a sequence of quintuples is obtained. A corresponding training sentence can be generated based on the quintuple sequence in the walk path. Entities, relation names, and / or concepts in the training sentence are masked to obtain a masked sentence. The language model is trained using the predicted masked characters in the masked sentence as the target. Furthermore, entity positions in the training sentence can be labeled, and the labeled training sentences are used as training data. This allows for the acquisition of a large amount of domain-specific labeled data without relying on manual annotation, while reducing noise such as mislabeling and omissions. The language model is then fine-tuned using the training data to obtain an adjusted entity recognition model for handling entity recognition tasks in the target domain, achieving cold start for entity recognition tasks.

Description

Technical Field

[0001] This application relates to the field of information extraction technology, and more specifically, to an entity recognition model training method, entity recognition method, device and related equipment. Background Technology

[0002] Currently, artificial intelligence technology is moving from perceptual intelligence to cognitive intelligence. Research on language understanding and knowledge mining is deepening, and the value of factual and common-sense knowledge is gradually becoming apparent. Named Entity Recognition (NER) is an indispensable foundational step in language understanding and knowledge extraction. Its goal is to identify named entities in natural language text and categorize them into predefined entity categories. The results of entity recognition can be used to support higher-level intelligent applications such as question answering, dialogue, and recommendation.

[0003] In the process of building entity recognition models in vertical domains, a large amount of labeled corpus is usually required as training samples for fine-tuning pre-trained language models such as BERT. Therefore, obtaining domain-labeled samples is a crucial prerequisite. Current technologies often employ manual annotation to acquire samples. This method requires significant human resources, cannot achieve rapid cold start of downstream systems, and is prone to noise such as mislabeling and omissions. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide an entity recognition model training method, entity recognition method, apparatus, and related equipment to solve the problems of existing entity recognition models relying on manual annotation of training samples, which consumes a lot of human resources, cannot achieve rapid cold start of downstream systems, and is prone to annotation noise. The specific solution is as follows:

[0005] Firstly, a method for training an entity recognition model is provided, including:

[0006] Obtain the knowledge graph of the target domain;

[0007] Walk sampling is performed from the knowledge graph to obtain at least one walk path. Each walk path contains a sequence of quintuples. Each quintuple in the sequence includes a head entity and its concept, a tail entity and its concept, and the relation name between the head entity and the tail entity.

[0008] For each traversal path, a corresponding training statement is generated based on the sequence of quintuples contained therein;

[0009] The entities, relation names, and / or concepts in the training statements are masked to obtain masked statements. The language model is trained with the goal of predicting the masked characters in the masked statements.

[0010] The entity locations in the training sentences are labeled, and the training sentences with labeled entity locations are used as training data to fine-tune the trained language model to obtain an adjusted entity recognition model, which is used to process entity recognition tasks in the target domain.

[0011] Secondly, an entity recognition method is provided, including:

[0012] Obtain the text to be processed within the target domain;

[0013] The text to be processed is input into the entity recognition model obtained by the above method, and the entities contained in the text to be processed are obtained from the model output.

[0014] Thirdly, an entity recognition model training device is provided, comprising:

[0015] The knowledge graph acquisition unit is used to acquire the knowledge graph of the target domain.

[0016] A sampling unit is used to perform walk sampling from the knowledge graph to obtain at least one walk path. Each walk path contains a sequence of quintuples. Each quintuple in the sequence includes a head entity and its concept, a tail entity and its concept, and the relation name between the head entity and the tail entity.

[0017] The training statement generation unit is used to generate corresponding training statements for each traversal path based on the sequence of 5-tuples contained therein.

[0018] The model pre-training unit is used to mask the entities, relation names and / or concepts in the training statement to obtain the masked statement, and to train the language model with the goal of predicting the masked characters in the masked statement.

[0019] The model fine-tuning unit is used to annotate the entity positions in the training sentences and use the annotated training sentences as training data to fine-tune the trained language model to obtain an adjusted entity recognition model, which is used to process entity recognition tasks in the target domain.

[0020] Fourthly, an entity recognition model training device is provided, including: a memory and a processor;

[0021] The memory is used to store programs;

[0022] The processor is used to execute the program to implement the various steps of the entity recognition model training method as described above.

[0023] Fifthly, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the various steps of the entity recognition model training method as described above.

[0024] By employing the aforementioned technical solution, this application incorporates a knowledge graph of the target domain into the training process of the entity recognition model. The knowledge graph contains semantic knowledge of the target domain. By performing walk sampling from the knowledge graph, at least one walk path consisting of a sequence of quintuples is obtained. Corresponding training statements can be generated based on the quintuple sequences in the walk path. The generated training statements can cover prior knowledge information of the target domain, and a large number of training statements can be quickly generated even in low-resource scenarios. Furthermore, entities, relation names, and / or concepts in the training statements are masked to obtain masked statements. The language model is trained by predicting the masked characters in the masked statements, enabling the model to explicitly model semantic units at the domain knowledge level, thus obtaining a knowledge-enhanced language model. Based on this, entity locations in training sentences can be labeled, and training sentences with labeled entity locations can be used as training data. This allows for the acquisition of a large amount of domain-labeled data without relying on manual labeling, reducing the cost of manual labeling and minimizing noise such as mislabeling and omissions. The language model can then be fine-tuned using the training data to obtain an adjusted entity recognition model for handling entity recognition tasks in the target domain.

[0025] Compared to existing technologies, this approach combines the knowledge graph of the target domain with the training of the entity recognition model, which can quickly start downstream entity recognition tasks and eliminates the need for users to manually annotate a large amount of training data, thus achieving a cold start for entity recognition tasks. Attached Figure Description

[0026] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0027] Figure 1 A flowchart illustrating the entity recognition model training method provided in this application embodiment;

[0028] Figures 2a-2c The training process of language models under entity mask, relation mask, and concept mask are illustrated respectively.

[0029] Figure 3 An example of a label matrix diagram is provided;

[0030] Figure 4 This example illustrates the overall process of entity recognition.

[0031] Figure 5A schematic diagram of an entity recognition model training device provided in an embodiment of this application;

[0032] Figure 6 This is a schematic diagram of the structure of the entity recognition model training device provided in the embodiments of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] This application provides a training scheme for an entity recognition model, capable of training a neural network model for handling entity recognition tasks. In training the entity recognition model, this application utilizes a knowledge graph, a graph data structure composed of nodes and directed edges. Nodes represent entities, and directed edges between nodes represent relationships between entities. A knowledge graph can be viewed as a semantic knowledge base, describing concepts and their relationships in the physical world in symbolic form, providing comprehensive, holistic, and relational references and support for domain knowledge research and development. This application addresses the dependence of supervised learning models on large amounts of labeled data by referencing a knowledge graph and leveraging its semantic knowledge to train the entity recognition model.

[0035] The proposed solution can be implemented based on a terminal with data processing capabilities, such as a mobile phone, computer, server, or cloud platform.

[0036] Next, combined Figure 1 The entity recognition model training method of this application may include the following steps:

[0037] Step S100: Obtain the knowledge graph of the target domain.

[0038] Specifically, for the entity recognition model to be created, the domain it applies to is defined as the target domain, meaning the entity recognition model is used to handle entity recognition tasks within the target domain. Therefore, this step first obtains the knowledge graph of the target domain. The knowledge graph contains semantic information about the target domain, which will be used to generate training data for the model training process.

[0039] In this step, the structure of the target knowledge graph can be either a directed instance graph without ontology constraints or a directed semantic web subject to ontology constraints. (Target domain knowledge graph) A knowledge graph can contain a certain number of entities ξ, with clear and explicit relationships and names between them. These relationships are represented as follows:

[0040] Step S110: Perform walk sampling from the knowledge graph to obtain at least one walk path.

[0041] Each walk path contains a sequence of quintuples, where each quintuple includes a head entity and its concept, a tail entity and its concept, and the relation name between the head and tail entities. For example:

[0042] From knowledge graph In the process, a set P = {p1, p2, p3, ..., p...} of several traversal paths is collected. n In set P, each walk path p i p is a sequence composed of several quintuples i ={o1, o2, o3, ..., o m A quintuple can be represented as e i and e j Representing the head entity and tail entity respectively, c i and c j The concept r represents the head entity and the tail entity. ij This is the name of the relationship between entities. Example: a quintuple (Liu Mouhua, singer, work, "Forget Love Potion", song), where the head entity is "Liu Mouhua" and the corresponding concept is "singer", the tail entity is ""Forget Love Potion" and the corresponding concept is "song", and the relationship between the head and tail entities is "work".

[0043] When performing walking sampling in a knowledge graph, a random walk approach can be used. Starting from the selected starting node, the user gradually walks and moves along the directed edges until the set walking termination condition is met, thus obtaining the walking path.

[0044] It is understandable that during the walking sampling, different walking strategies can be executed according to the user's settings to achieve sampling according to the user's set walking tendency. For example, the walking strategy may include: walking sampling according to the direction of the relationship between nodes, walking around the starting point according to the user's set tendency, or walking sampling in a direction that gradually moves away from the starting point, etc.

[0045] Step S120: For each traversal path, generate the corresponding training statement based on the quintuple sequence contained therein.

[0046] Specifically, each traversal path contains multiple quintuples, each quintuple containing a head and tail entity, the corresponding concept, and a relation name. Based on this, training statements can be generated using the sequence of quintuples. The training statements can include one or more of the following: head and tail entity, concept, and relation name.

[0047] Taking the aforementioned five-element combination (Liu Mouhua, singer, work, "Forget Love Potion", song) as an example:

[0048] The generated training statements can include: Liu Mouhua's works include "Forget Love Potion"; singer Liu Mouhua's works include a song called "Forget Love Potion", etc.

[0049] It is understandable that, since the training statements are generated with reference to the sequence of quintuples obtained from the knowledge graph, the training statements can contain knowledge semantic information from the knowledge graph. When training the model based on such training statements, semantic units at the domain knowledge level can be explicitly modeled to obtain a knowledge-enhanced language model.

[0050] Step S130: Mask the entities, relation names and / or concepts in the training statement to obtain a masked statement, and train the language model with the goal of predicting the masked characters in the masked statement.

[0051] Specifically, in this embodiment, the language model can be a pre-trained language model such as BERT or Transformer. To enable the language model to better learn domain knowledge semantics, this embodiment employs any one or more knowledge masking strategies at three granularities: entity masking, relation masking, and concept masking. This integrates relational phrases, entity-level knowledge, and concept-level knowledge into the language model, making the trained language model's representation of knowledge more accurate.

[0052] Taking a language model using Transformer as the basic encoder as an example, it can capture the contextual information of each mask in the input sentence through a self-attention mechanism and generate a series of contextual embeddings for reinforcement learning and training of entity, concept, and relational knowledge. It also maintains character-based input, so that the model does not need to rely on additional information such as word segmentation during downstream fine-tuning. Training strategies for language models can be found in [reference needed]. Figures 2a-2c As shown, examples of entity masks, relation masks, and concept masks are provided respectively.

[0053] Specifically, in this step, when masking entities in the training statement, a certain proportion of entity words in the training statement can be randomly replaced with mask tags. Of course, the head entities in the training statement can also be selectively masked, and the training objective is to predict the masked entity words.

[0054] In this step, when masking relation names in the training statement, a certain proportion of relation names in the training statement can be randomly replaced with mask tags. The training objective is to predict the masked relation names, so that the model can explicitly model the semantic relationships between entities.

[0055] In this step, when masking the concepts in the training statements, a certain proportion of the concepts in the training statements can be randomly replaced with mask tags. The training objective is to predict the masked concepts, so that the model can model the hierarchical concepts of entities.

[0056] Step S140: Label the entity positions in the training sentences, and use the training sentences labeled with entity positions as training data to fine-tune the trained language model to obtain the adjusted entity recognition model.

[0057] Understandably, the previous step explicitly models the prior knowledge in the knowledge graph, effectively utilizing the implicit semantics in the large-scale pre-trained language model, while enhancing the model's expressive power in domain knowledge semantics and its cross-domain transfer capabilities. However, the language model cannot be directly used for cold starts in specific entity recognition tasks. This step requires fine-tuning the language model using training sentences labeled with entity locations, i.e., adjusting the network parameters of the language model, so that the adjusted entity recognition model can be adapted to specific entity recognition task scenarios.

[0058] The entity recognition model obtained after fine-tuning in this step can be used for downstream tasks, that is, it can be used to process entity recognition tasks in the target domain.

[0059] In the training process of the entity recognition model, this application incorporates a knowledge graph of the target domain. This knowledge graph contains semantic knowledge of the target domain. By performing walk sampling from the knowledge graph, at least one walk path consisting of a sequence of quintuples is obtained. Training statements can be generated based on the quintuple sequences in the walk path. These generated training statements can cover prior knowledge information of the target domain, and even in low-resource scenarios, a large number of training statements can be generated quickly in this way. Furthermore, entities, relation names, and / or concepts in the training statements are masked to obtain masked statements. The language model is trained by predicting the masked characters in the masked statements, allowing the model to explicitly model semantic units at the domain knowledge level, resulting in a knowledge-enhanced language model. Based on this, entity positions in the training statements can be labeled, and these labeled training statements are used as training data. This allows for the generation of a large amount of domain-labeled data without relying on manual labeling, reducing manual labeling costs and minimizing noise such as mislabeling and omissions. The language model can then be fine-tuned using the training data to obtain an adjusted entity recognition model for handling entity recognition tasks in the target domain.

[0060] Compared to existing technologies, this approach combines the knowledge graph of the target domain with the training of the entity recognition model, which can quickly start downstream entity recognition tasks and eliminates the need for users to manually annotate a large amount of training data, thus achieving a cold start for entity recognition tasks.

[0061] In some embodiments of this application, the above step S110, which involves performing walk sampling from the knowledge graph to obtain at least one walk path, is described as an optional implementation.

[0062] In one alternative approach, several starting nodes can be randomly selected from the knowledge graph. Then, for each starting node, a random walk is performed to move from one node to another. When moving from the current node to the next, the probability of moving to any of the current node's neighboring nodes is the same, while the probability of moving to any non-neighboring node is 0. Each time a node undergoes a move, a quintuple is generated based on the two nodes before and after the move, along with the directed edge. This quintuple is added to the walk path corresponding to the starting node, until the walk path reaches a set length, thus obtaining the walk path corresponding to each starting node.

[0063] The above algorithm is relatively simple to implement, but because the transfer is completely random, it is easy for the path to contain duplicate entities, ambiguous semantics, and semantic irrelevance, resulting in low quality of the sampled path. It cannot well reflect the overall structure of the knowledge graph and the semantic relationship between entities. Furthermore, completely random sampling makes the collection method unadjustable and uncontrollable, and it is impossible to control the sampling bias in different domains.

[0064] Based on this, this embodiment further provides another optional implementation method, as follows:

[0065] S1. Select at least one node in the target knowledge graph as the starting node.

[0066] In this step, the structure of the target knowledge graph can be either a directed instance graph without ontology constraints or a directed semantic web subject to ontology constraints. Target Knowledge Graph A knowledge graph can contain a certain number of entities ξ, with clear and explicit relationships and names between them. These relationships are represented as follows:

[0067] To obtain a traversal path, the starting point of the path needs to be selected first. Therefore, starting nodes can be selected in the target knowledge graph. There can be one or more starting nodes. Each starting node will eventually result in a corresponding traversal path. That is, the number of selected starting nodes is consistent with the number of traversal paths obtained in the end.

[0068] Optionally, when selecting a starting node in the target knowledge graph, a random selection method can be used to randomly select a specified number of nodes as the starting node. Alternatively, the user can specify the starting node, or other strategies can be used to select the starting node.

[0069] S2. For each starting node, according to the sampling strategy, the nodes in the target knowledge graph are transferred along the directed edges with different probabilities. Each time a node undergoes a transfer, a quintuple is generated based on the two nodes before and after the transfer and the directed edges.

[0070] Specifically, the node before the transfer represents the head entity, and the node after the transfer represents the tail entity. The directed edge between the two represents the relation name between the two entities. Based on this, each time a node is transferred, a quintuple can be generated based on the two nodes before and after the transfer and the directed edge. For example: (Liu Mouhua, singer, work, "Forget Love Potion", song).

[0071] In this step, nodes are transferred along directed edges when sampling the walking path. That is, when transferring from the current node to the next node, it is only possible to transfer to a neighboring node of the current node.

[0072] Define the current node as v. Let V be the set of neighboring nodes of node v. Then, when node v moves to another node x in the graph, if x does not belong to the set of neighboring nodes of node v, the neighboring node will be determined by the set of neighboring nodes of node v. The corresponding transition probability is 0.

[0073] Furthermore, during each transition, the probability of transitioning from the current node v to any neighboring node x is determined based on the distance between the nodes previously traversed by the current node v (i.e., nodes preceding the current node v in the traversal path) and the neighboring node x, and / or the semantic relationship between the current node v and the neighboring node x. In other words, the probability of transitioning from the current node v to any neighboring node x can be influenced by two factors: the distance between the nodes previously traversed by the current node v and the neighboring node x, and the semantic relationship between the current node v and the neighboring node x.

[0074] It is understandable that different neighboring nodes may have different distances between the nodes that the current node v has previously traversed, and the semantic relationships between different neighboring nodes and the current node v may also be different. Therefore, when the current node v moves to different neighboring nodes, the corresponding transition probability may also be different. The specific transition probability can be controlled by a preset sampling strategy.

[0075] S3. Add the generated quintuple to the traversal path corresponding to the starting node until the length of the traversal path reaches the set length, and obtain the final traversal path corresponding to each starting node.

[0076] Specifically, starting from each starting node, a quintuple is generated after each transition. The generated quintuple is then added to the traversal path corresponding to the starting node until the length of the traversal path reaches the set length, thus obtaining the final traversal path corresponding to the starting node.

[0077] Ultimately from knowledge graph In the process, collect a set of several traversal paths P = {p1, p2, p3, ..., p...} m In set P, each walk path p i p is a sequence composed of several quintuples i ={o1, o2, o3, ..., o m A quintuple can be represented as... e i and e j Representing the head entity and tail entity respectively, c i and c j The concept r represents the head entity and the tail entity. ij The name of the relationship between entities.

[0078] Since the walking path moves along directed edges, in any two adjacent quintuples in the walking path, the tail entity in the first quintuple is the same as the head entity in the second quintuple, thus ensuring the continuity of the path.

[0079] The set length of the walking path can be a threshold number of quintuples contained in the walking path. Generally, the set length can be 3, 4, 5, etc.

[0080] The traversal path determination method provided in this application selects a starting node in the target knowledge graph, and then, according to a sampling strategy, moves the nodes in the target knowledge graph along directed edges with different probabilities. Each time a node undergoes a transfer, a quintuple is generated based on the two nodes before and after the transfer and the directed edge. During each transfer, the probability of the current node moving to any neighboring node is determined based on the distance between the nodes previously traversed by the current node and the neighboring node, and / or the semantic relationship between the current node and the neighboring node. Thus, it can be seen that the traversal sampling in the knowledge graph in this application is not completely random. Each time a node is transferred, the transfer probability can be determined by referring to the distance between the nodes previously traversed by the current node and the next neighboring node. Based on this, the sampling tendency can be controlled, such as controlling the tendency to traverse and sample around the starting node, or controlling the tendency to traverse and sample in a depth direction away from the starting node, thereby reflecting the affinity of the knowledge graph in terms of homogeneity and structure.

[0081] Furthermore, the semantic relationship between the current node and its next neighbor node can be used to determine the transition probability. Based on this, walk sampling can be performed along semantically explicit relationships to improve the quality of the walk path. For example, if the entity represented by the current node is "Liu Mouhua", most of its neighbor nodes have a "work relationship" with the current node, while a small number of neighbor nodes have a "birthplace population" relationship. If the current node is moved to a neighbor node corresponding to the "work relationship", the resulting quintuple, when converted into a training statement, might result in a statement like "When was the release date of Liu Mouhua's work?"; if the current node is moved to a neighbor node corresponding to the "birthplace population", the resulting quintuple, when converted into a training statement, might result in a statement like "How many people were born in Liu Mouhua's birthplace?". Therefore, fewer relationships mean a more explicit relationship, while more relationships mean a more ambiguous relationship. Training the model based on ambiguous statements can lead to difficulty in model convergence. Therefore, in this embodiment, the transition probability can be determined by referring to the semantic relationship between the current node and the next neighbor node. Based on this, walking sampling can be performed along semantically clear relationships to improve the quality of the walking path and ensure that the convergence speed of the model is accelerated when training the model by generating training sentences in the future.

[0082] In some embodiments of this application, the process of determining the probability of the current node moving to any neighboring node in S2 above is described. This process is based on the distance between the current node and the neighboring nodes that it has previously traversed, and / or the semantic relationship between the current node and the neighboring nodes.

[0083] In one possible implementation:

[0084] The transfer score for the current node to move to any neighboring node can be determined first based on the distance between the nodes previously visited by the current node and the neighboring nodes, and / or the semantic relationship between the current node and the neighboring nodes.

[0085] Furthermore, for any target neighbor node:

[0086] The probability of the current node moving to the target neighbor node is determined based on the transfer score of the current node moving to the target neighbor node and the sum of the transfer scores of the current node moving to each neighbor node.

[0087] The scheme described in this embodiment determines the transfer score of a current node transitioning to a neighboring node, and further converts the transfer score into a transfer probability. In one optional approach, the conversion of the transfer score into the transfer probability is as follows:

[0088]

[0089] Among them, t i Let represent the i-th transition, v be the current node in the knowledge graph, and p(x) be the probability of transitioning to another node x in the graph. Where π vx This represents the transition score between node v and node x. Let N be the set of neighboring nodes of node v. v Let V be the number of neighboring nodes of node v, and let V be a value.

[0090] Furthermore, the process of determining the transfer score for the current node to any neighboring node can be based solely on the distance between the nodes previously traversed by the current node and the neighboring node, or solely on the semantic relationship between the current node and the neighboring node. Of course, it can also be based on both of the above factors simultaneously.

[0091] Wherein, the nodes visited before the current node can be any one or more nodes in the traversal path that are located before the current node. In this embodiment, the previous node of the current node is used as an example for illustration. The process of determining the transfer score of the current node to any neighboring node may include:

[0092] S1. Determine the first transfer score for the current node to transfer to the neighboring node based on the distance between the previous node of the current node and the neighboring node.

[0093] The previous node of the current node v can be represented as node e, where e = t i-2 .

[0094] Optionally, the distance d between node e and node x ex It can be the shortest number of jumps between node e and node x, with a value range of {0,1,2}. Different shortest jump numbers correspond to different transition score settings. The transition score setting corresponding to the shortest jump number 0 is controlled by the first hyperparameter p, and the transition score setting corresponding to the shortest jump number 2 is controlled by the second hyperparameter q.

[0095] Based on the shortest number of jumps between the current node's previous node e and its neighbor node x, the corresponding transition score setting value is selected as the first transition score α for the current node v to transition to its neighbor node x. pq (e, x).

[0096] An example of different shortest jump counts d ex The correspondence between the transfer score setting and the corresponding transfer score setting is as follows:

[0097]

[0098] Therefore, the hyperparameter *p* controls the tolerance for a wandering path to re-enter a node it has already visited. A lower *p* value results in a higher first-transition score and a greater probability of re-entry, meaning the path is more likely to re-enter a previously visited node, causing it to tend to wander near the starting node. The hyperparameter *q* controls the bias in allowing a wandering path to move to nodes at greater depths. A lower *q* value results in a higher first-transition score and a greater probability of re-entry, meaning the path will tend to move away from the starting node.

[0099] By weighing the propensity of p and q, the tendency of the wandering path can be guided, thereby reflecting the affinity of the domain knowledge graph in terms of homogeneity and structure.

[0100] The method provided in this step allows users to control the tendency of the wandering path by setting different p and q values.

[0101] S2. Determine the second transfer score for the current node to transfer to the neighboring node based on the semantic relationship between the current node and the neighboring node.

[0102] Specifically, among the neighboring nodes of the current node, some have the same semantic relationship as the current node, while others have a different semantic relationship. For example, if the entity represented by the current node is "Liu Mouhua", then the relationships between its neighboring nodes A, B, and C and the current node are all "work relationship", while the relationship between node D and the current node is "birthplace population". The fewer the number of semantic relationships, the clearer the semantic relationship.

[0103] When moving from the current node to the next node, the selection of the transfer object from among the neighboring nodes can be based on semantically clear relationships, that is, a preference is given to transferring along relationship types with fewer semantic relationships. Using the example above, since the semantic relationship "birthplace population" has only one instance, while the semantic relationship "works relationship" has three instances, node D can be preferentially selected as the next transfer node. This means that the second transfer score for node D is set higher than that for nodes A, B, and C.

[0104] One possible implementation of step S2 to determine the second transition score is as follows:

[0105] For any target neighbor node of the current node:

[0106] S21. Determine the target relationship name between the current node and the target neighbor node.

[0107] Define the semantic relationship between the current node v and the target neighbor node x as r, that is, the target relationship name is r.

[0108] S22. Count the total number of neighboring nodes of the current node that have the target relation name with the current node.

[0109] Define the total number of neighboring nodes of the current node that have the target relation name r with the current node as follows:

[0110] S23, Based on the total number The number N of all neighboring nodes of the current node v And the third hyperparameter λ, to determine the second transition score of the current node v to the target neighbor node x.

[0111] Specifically, to ensure that nodes transition along semantically explicit relationships, a total number can be set. The larger the score, the higher the second transition score. The smaller the value, the better. Here, the hyperparameter λ is the penalty coefficient.

[0112] An optional method for calculating the second transfer score is as follows:

[0113]

[0114] Where λ≥0,

[0115] The larger the λ value, the more the walk path tends to move along semantically explicit relationships.

[0116] S3. Determine the total transfer score of the current node to the neighboring node based on the first transfer score and / or the second transfer score.

[0117] Specifically, the first transition score α can be directly applied. pq (e, x) represents the total transition score π for the current node v to move to its neighbor node x. vx Alternatively, the second transfer score can be directly transferred. The total transition score π for the current node v to move to its neighbor node x vx Of course, the first transition score α can also be considered. pq (e, x) and second transition score Determine the total transfer score π vx For example:

[0118] The first transition score α pq (e, x) and second transition score Multiply, and the result is the total transfer score π. vx :

[0119]

[0120] As can be seen from the above scheme, the total transfer score of the current node v to the neighbor node x can be varied by adjusting the three hyperparameters p, q, and λ. Therefore, different sampling strategies can be set by different combinations of p, q, and λ values ​​to control the sampling tendency of the walking path in knowledge graphs of different domains.

[0121] In some embodiments of this application, the above step S120 is described as an optional implementation of generating corresponding training statements based on the quintuple sequence contained in each traversal path.

[0122] This embodiment provides several optional generation strategies, which can be used to generate training statements.

[0123] The first method is path splicing.

[0124] Specifically, the head entity, relation name, and tail entity in each quintuple can be concatenated into a training statement containing a subject, verb, and object, according to the sorting order of each quintuple in the walk path.

[0125] or,

[0126] According to the sorting order of each quintuple in the walk path, the head entity and its concept, relation name, tail entity and its concept in each quintuple can be concatenated into a training statement containing subject, verb and object.

[0127] Alternatively, to ensure the fluency of the concatenated training sentences, the training sentences can be generated according to a preset regular expression pattern during the concatenation process, that is, some prepositions for coherence can be added at appropriate positions in the concatenated sentences.

[0128] For example, suppose that in a music knowledge graph, the following walking path is collected: p = {(Liu Mouhua, singer, work, "Forgetful Water", song), ("Forgetful Water", song, from the movie "Heaven and Earth", movie)}. Following the above path concatenation strategy, the generated training statements can include:

[0129] Liu Mouhua's work is "Forget Love Potion," which is based on the film "Heaven and Earth."

[0130] The singer Liu Mouhua's work is "Forget Love Potion," which is based on the movie "Heaven and Earth."

[0131] Path concatenation is the most basic generation strategy. Based on this, it can be combined with other generation strategies to better simulate the sentence expressions that might occur in real-world scenarios. Other optional generation strategies are as follows:

[0132] The second method is physical ablation.

[0133] The entity ablation strategy mainly involves omitting some entities from the statements generated by the aforementioned path concatenation strategy and converting them into implicit semantic representations to obtain the training statements.

[0134] Let's continue with the example above:

[0135] Following the entity ablation generation strategy, the entity "《Forget Love Potion》" can be omitted, resulting in the following training statement:

[0136] Liu Mouhua's work is based on the film "Heaven and Earth".

[0137] The third type is object fronting.

[0138] The object-fronting generation strategy mainly involves swapping the positions of the subject and object in sentences generated according to the path concatenation strategy, and adding pronouns to ensure the semantic consistency of the training sentences.

[0139] Let's continue with the example above:

[0140] Following the object-fronting generation strategy, the positions of the subject "Liu Mouhua" and the object "《Forgetful Water》" are swapped, and the pronoun positions of the object "《Heaven and Earth》" and the subject "《Forgetful Water》" are replaced. The resulting training sentences are as follows:

[0141] "Forget Love Potion" is a work by Liu Mouhua, and "Heaven and Earth" is its film adaptation.

[0142] Fourth, concept replacement.

[0143] The concept replacement generation strategy mainly replaces some entities in the statements generated according to the path splicing strategy with corresponding concepts or upper concepts to obtain training statements.

[0144] Still taking the above example for illustration:

[0145] According to the concept replacement generation strategy, replace "Forget-Me-Not Water" with the concept "song", and replace "Heaven and Earth" with the upper concept of the concept "movie": "film and television drama", and the obtained training statements are as follows:

[0146] The works of singer Liu Mouhua include a certain song, and the movie it comes from is a film and television drama.

[0147] Fifth, sentence pattern transformation.

[0148] The sentence pattern transformation generation strategy mainly omits some entities and relationship names in the statements generated according to the path splicing strategy, and converts the statements into interrogative sentences, rhetorical questions and / or rhetorical questions for the omitted parts to obtain the transformed training statements.

[0149] Still taking the above example for illustration:

[0150] According to the sentence pattern transformation generation strategy, omit the entity "Forget-Me-Not Water" and convert it into an interrogative sentence for "Forget-Me-Not Water", and the obtained training statements are as follows:

[0151] Which song in Liu Mouhua's works comes from the movie Heaven and Earth?

[0152] Alternatively, according to the sentence pattern transformation generation strategy, omit the relationship name "comes from the movie" and convert it into a rhetorical question for "comes from the movie", and the obtained training statements are as follows:

[0153] Is the relationship between Liu Mouhua's work Forget-Me-Not Water and Heaven and Earth that they come from the movie?

[0154] In this embodiment, several optional generation strategies are provided. This application can select one or several strategies from the various generation strategies in the above examples for combination to finally obtain the generated training statements. Since different generation strategies generate training statements from different perspectives, they can well simulate the expression methods of various input corpora that may appear in the actual scenario, that is, enrich the types of training statements. Based on this, the trained model has better robustness.

[0155] In some embodiments of this application, another optional implementation of the entity recognition model training method is provided. Specifically, in order to further enrich the training statements and make the training statements generated above more in line with actual application scenarios, this embodiment further adds a semantic expansion process to the training statements before executing steps S130 and S140 to obtain expanded training statements. Then, in subsequent steps S130 and S140, the entities, relation names and / or concepts in the expanded training statements can be masked, and the entity positions in the expanded training statements can be labeled.

[0156] This embodiment provides two optional semantic extension methods, one of which is:

[0157] Entity name perturbation.

[0158] The entity name perturbation involves randomly removing and repeating entities in the training statement at a set ratio to obtain the first expanded training statement.

[0159] By perturbing entity names, user input information in real-world scenarios can be simulated, improving the robustness of the model in downstream entity recognition tasks such as user question answering and voice input.

[0160] It should be noted that the first expanded training statement obtained by perturbing the entity name is mainly used as the processing object for entity location annotation in step S140, that is, as training data for the language model fine-tuning stage.

[0161] For example, the generated training statement is "Liu Mouhua's work is 'Forget Love Potion,' which is from the movie 'Heaven and Earth.'"

[0162] By perturbing the entity name, the entity "Liu Mouhua" can be repeated as "Liu Mouhua Hua", and "Heaven and Earth" can be deleted to "Heaven and Earth". The final first expanded training statement is "Liu Mouhua Hua's work is 'Forget Love Potion', which is from the movie 'Heaven and Earth'".

[0163] The second is:

[0164] Generate semantically similar sentences.

[0165] Semantic similarity sentence generation mainly involves generating sentences similar to the training sentences to obtain the second expanded training sentences.

[0166] Specifically, a similar sentence generation model can be used to generate similar sentences corresponding to the training sentences, which are then used as the second expanded training sentences; wherein, the similar sentence generation model is trained using similar sentence pairs as training data.

[0167] In this embodiment, a seq2seq structure can be used as the similar sentence generation model. The collected similar sentence pairs are concatenated and used as training corpus. For example, if SENT_A and SENT_B are similar sentences, they can be concatenated as [CLS]SENT_A[SEP]SENT_B or [CLS]SENT_B[SEP]SENT_A. The concatenated training corpus is then input into the similar sentence generation model for training. Finally, the trained similar sentence generation model generates sentences similar to the training sentences, resulting in the second expanded training sentence.

[0168] For example, the generated training statement is "Liu Mouhua's work is 'Forget Love Potion,' which is from the movie 'Heaven and Earth.'"

[0169] Based on the method of generating similar sentences, the generated similar sentences can include:

[0170] Liu Mouhua's signature song is "Forget Love Potion," which comes from the movie "Heaven and Earth."

[0171] In some embodiments of this application, the process of annotating the entity positions in the training statement to obtain training data for the language model fine-tuning stage in step S140 is described.

[0172] An alternative approach is to use sequence labeling to annotate the entity positions in the training statements to obtain training data.

[0173] In addition, considering that the training statement may contain multiple different entities, and that different entities may overlap, the use of sequence labeling may result in labeling errors. Therefore, in this embodiment, a label matrix can be used to label the entity positions in the training statement. Specifically, the label matrix can be an n×n matrix, where n is the number of characters in the training statement. The value of the element in the i-th row and j-th column of the label matrix indicates whether the text fragment composed of the i-th character to the j-th character in the training statement belongs to an entity.

[0174] Furthermore, the label matrix can also be represented as an l×n×n matrix, where l represents the number of entity label types. If all entity labels are considered to be of the same type, then l takes the value of 1, and the label matrix is ​​transformed into an n×n matrix. For ease of explanation, the following description uses the label matrix with l taking the value of 1 as an example.

[0175] Reference Figure 3 This example illustrates a label matrix.

[0176] Figure 3In the example, the training statement is: "Liu Mouhua's work is Forget-Love Potion." The element in row 1, column 3 has a value of 1, indicating that the first three characters "Liu Mouhua" are entities. The element in row 8, column 10 has a value of 1, indicating that the eighth and tenth characters "Forget-Love Potion" are entities. All other elements are 0, indicating that other characters in the training statement are not entities.

[0177] Furthermore, considering that in some entity recognition tasks, it is necessary not only to identify entities but also to determine their types, multiple different label matrices can be used. Each label matrix corresponds to a type of entity, and entities of that type are labeled in the corresponding label matrix. This form of label matrix can also be called a Multi-Head matrix.

[0178] The value of each element in a multi-head matrix can be represented as:

[0179] f(h i h j , α) = score α (i, j)

[0180] Where h is the sentence vector output by the language model, and score α (i, j) represents the index start =i and index end =j is the score of the continuous segment between them on the α-th entity type. Assume that a training statement t of length n, after being encoded by a language model, yields a vector sequence [h1, h2, ..., h...]. n ], via q i，α =W q，α h i +b q，α and k i，α =W k，α h i +b k，α Transformation yields the query vector sequence [q] 1，α q 2，α , ..., q n，α ] and key vector [k 1，α k 2，α , ..., k n，α In a multi-head matrix, the continuous segment t consisting of the i-th to j-th elements. i，j The score for an entity of type α is:

[0181]

[0182] Finally, in this embodiment, a Loss function is designed based on the above-mentioned scoring function for backpropagation to fine-tune the language model, that is, to update its weights.

[0183] Furthermore, considering that multi-head labeling may generate a large number of 0 labels representing no entity, loss functions such as Focal Loss, Label Smoothing Loss, and negative sampling can be used to alleviate the problems of sample label imbalance and overfitting.

[0184] In some embodiments of this application, based on the trained entity recognition model obtained from the foregoing embodiments, an entity recognition method is further provided. Specifically, the text to be processed in the target domain can be obtained, and the text to be processed can be input into the entity recognition model trained in the foregoing embodiments to obtain the entities contained in the text to be processed output by the model.

[0185] According to the solution in this application, users do not need to collect a large amount of corpus in advance for model training. The entity recognition process can achieve a fast cold start based on the entity recognition model trained above, and can provide real-time entity recognition inference services for the text input by the user.

[0186] Of course, when further optimization of the model is needed, a small number of labeled samples can be provided to achieve fine-grained optimization and adjustment of the model.

[0187] Reference Figure 4 It provides an example of an overall flowchart for entity recognition.

[0188] The flowchart in the dashed box on the right represents the offline pre-training process of the entity recognition model. It primarily uses a domain knowledge graph and a random walk sampling strategy to obtain multiple walk paths. Training statements are then generated according to these walk paths, and semantic expansion is performed on these training statements to obtain expanded training statements. These expanded training statements have two uses: first, they serve as knowledge-enhancing corpora (i.e., masked statements after masking entity, concept, and relation names) to train the language model; second, they are used as training samples with entity location annotations as training data in the language model fine-tuning stage to fine-tune the network parameters of the language model, resulting in the final entity recognition model.

[0189] Figure 4 The left side of the middle section represents the online entity recognition stage, where domain text is obtained. This domain text can be the input question-and-answer text or other text, which can be directly input into the trained entity recognition model to obtain the entity recognition result, thus achieving a cold start in the entity recognition stage.

[0190] The entity recognition model training apparatus provided in the embodiments of this application is described below. The entity recognition model training apparatus described below can be referred to in correspondence with the entity recognition model training method described above.

[0191] See Figure 5 , Figure 5 This is a schematic diagram of the structure of an entity recognition model training device disclosed in an embodiment of this application.

[0192] like Figure 5 As shown, the device may include:

[0193] Knowledge graph acquisition unit 11 is used to acquire knowledge graphs for the target domain;

[0194] Sampling unit 12 is used to perform walk sampling from the knowledge graph to obtain at least one walk path. Each walk path contains a sequence of quintuples. Each quintuple in the sequence includes a head entity and its concept, a tail entity and its concept, and the relation name between the head entity and the tail entity.

[0195] The training statement generation unit 13 is used to generate corresponding training statements for each traversal path based on the sequence of five tuples contained therein.

[0196] The model pre-training unit 14 is used to mask the entities, relation names and / or concepts in the training statement to obtain a masked statement, and to train the language model with the goal of predicting the masked characters in the masked statement.

[0197] The model fine-tuning unit 15 is used to annotate the entity positions in the training sentences and use the training sentences with annotated entity positions as training data to fine-tune the trained language model to obtain an adjusted entity recognition model, which is used to process entity recognition tasks in the target domain.

[0198] Optionally, the process by which the sampling unit performs walk sampling from the knowledge graph to obtain at least one walk path may include:

[0199] Select at least one node from the target knowledge graph as the starting node;

[0200] For each starting node, according to the sampling strategy, the nodes in the target knowledge graph are transferred along the directed edges with different probabilities. Each time a node is transferred, a quintuple is generated based on the two nodes before and after the transfer and the directed edges. The probability of the current node being transferred to any neighboring node during each transfer is determined based on the distance between the nodes previously traversed by the current node and the neighboring nodes, and / or the semantic relationship between the current node and the neighboring nodes.

[0201] The generated quintuples are added to the traversal path corresponding to the starting node until the length of the traversal path reaches the set length, thus obtaining the traversal path corresponding to each starting node.

[0202] Optionally, the process by which the sampling unit determines the probability of the current node moving to any neighboring node based on the distance between the nodes previously traversed by the current node and the neighboring nodes, and / or the semantic relationship between the current node and the neighboring nodes, may include:

[0203] Based on the distance between the nodes previously visited by the current node and the neighboring nodes, and / or the semantic relationship between the current node and the neighboring nodes, determine the transfer score for the current node to move to any neighboring node;

[0204] For any target neighbor node:

[0205] The probability of the current node moving to the target neighbor node is determined based on the transfer score of the current node moving to the target neighbor node and the sum of the transfer scores of the current node moving to each neighbor node.

[0206] Optionally, the process by which the sampling unit determines the transfer score for the current node to move to any neighboring node based on the distance between the nodes previously visited by the current node and the neighboring nodes, and / or the semantic relationship between the current node and the neighboring nodes, may include:

[0207] Based on the distance between the previous node of the current node and the neighboring node, determine the first transfer score for the current node to transfer to the neighboring node;

[0208] Based on the semantic relationship between the current node and the neighboring nodes, determine the second transfer score for the current node to transfer to the neighboring nodes;

[0209] Based on the first transfer score and / or the second transfer score, determine the total transfer score for the current node to transfer to the neighboring node.

[0210] Optionally, the process by which the sampling unit determines the first transfer score for the current node to transfer to the neighboring node based on the distance between the previous node and the neighboring node may include:

[0211] Determine the shortest number of jumps between the previous node of the current node and the neighboring nodes. The shortest number of jumps is in the range of {0,1,2}. Different shortest number of jumps correspond to different transfer score settings. The transfer score setting corresponding to the shortest number of jumps 0 is controlled by the first hyperparameter p, and the transfer score setting corresponding to the shortest number of jumps 2 is controlled by the second hyperparameter q.

[0212] Based on the shortest number of jumps between the current node's previous node and the neighboring node, a corresponding transfer score setting value is selected as the first transfer score for the current node to transfer to the neighboring node.

[0213] Optionally, the process by which the sampling unit determines the second transfer score for the current node to transfer to the neighboring node based on the semantic relationship between the current node and the neighboring nodes may include:

[0214] For any target neighbor node of the current node:

[0215] Determine the target relationship name between the current node and the target neighbor node;

[0216] Count the total number of neighboring nodes of the current node that have the target relation name with the current node;

[0217] Based on the total number, the number of all neighboring nodes of the current node, and the third hyperparameter λ, a second transfer score is determined for the current node to move to the target neighboring node, wherein the larger the total number, the smaller the second transfer score.

[0218] Optionally, the process by which the sampling unit determines the total transfer score of the current node to the neighboring node based on the first transfer score and / or the second transfer score may include:

[0219] The product of the first transfer score and the second transfer score is taken as the total transfer score for the current node to transfer to the neighboring node.

[0220] Optionally, the process by which the training statement generation unit generates a corresponding training statement for each traversal path based on the 5-tuple sequence it contains may include:

[0221] For each traversal path, based on the quintuple sequence it contains, a corresponding training statement is generated according to a predefined generation strategy, wherein the generation strategy includes:

[0222] Path concatenation: According to the sorting order of each quintuple in the traversal path, concatenate the head entity, relation name, and tail entity of each quintuple into a training statement containing subject, verb, and object; and / or, according to the sorting order of each quintuple in the traversal path, concatenate the head entity and its concept, relation name, tail entity and its concept of each quintuple into a training statement containing subject, verb, and object.

[0223] Optionally, the generation strategy may also include one or more of the following:

[0224] Entity ablation: Some entities in the statement generated according to the path concatenation strategy are omitted and converted into implicit semantic representations to obtain the training statement;

[0225] Object fronting: The positions of the subject and object in the sentences generated according to the path splicing strategy are swapped, and pronouns are added to ensure the semantic consistency of the training sentences;

[0226] Concept replacement: In the statement generated according to the path concatenation strategy, some entities are replaced with corresponding concepts or higher-level concepts to obtain the training statement;

[0227] Sentence transformation: Some entity and relation names in the statement generated according to the path concatenation strategy are omitted, and the statement is transformed into a question, rhetorical question and / or interrogative sentence for the omitted part, so as to obtain the transformed training statement.

[0228] Optionally, the apparatus of this application may further include: a semantic expansion unit, configured to semantically expand the training statement before masking entities, relation names and / or concepts in the training statement and labeling entity positions in the training statement, to obtain an expanded training statement.

[0229] Optionally, the process by which the semantic expansion unit performs semantic expansion on the training statement to obtain the expanded training statement may include:

[0230] The entities in the training statement are randomly reduced or repeated at a set ratio to obtain the first expanded training statement, which is used as the processing object for entity position annotation.

[0231] Optionally, the process by which the semantic expansion unit performs semantic expansion on the training statement to obtain the expanded training statement may include:

[0232] A similar sentence generation model is used to generate similar sentences to the training sentences, which are then used as the second expanded training sentences.

[0233] The similar sentence generation model is trained using similar sentence pairs as training data.

[0234] Optionally, the process by which the model fine-tuning unit annotates the entity locations in the training statement may include:

[0235] The entity positions in the training statement are labeled in the form of a label matrix, where the label matrix is ​​an n×n matrix, n is the number of characters contained in the training statement, and the value of the element in the i-th row and j-th column of the label matrix indicates whether the text segment composed of the i-th character to the j-th character in the training statement belongs to an entity.

[0236] The entity recognition model training device provided in this application embodiment can be applied to entity recognition model training devices, such as terminals: mobile phones, computers, etc. Optionally, Figure 6 The hardware structure block diagram of the entity recognition model training device is shown below. Figure 6 The hardware structure of an entity recognition model training device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0237] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0238] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0239] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0240] The memory stores a program, which the processor can call. The program is used for:

[0241] Obtain the knowledge graph of the target domain;

[0242] Walk sampling is performed from the knowledge graph to obtain at least one walk path. Each walk path contains a sequence of quintuples. Each quintuple in the sequence includes a head entity and its concept, a tail entity and its concept, and the relation name between the head entity and the tail entity.

[0243] For each traversal path, a corresponding training statement is generated based on the sequence of quintuples contained therein;

[0244] The entities, relation names, and / or concepts in the training statements are masked to obtain masked statements. The language model is trained with the goal of predicting the masked characters in the masked statements.

[0245] The entity locations in the training sentences are labeled, and the training sentences with labeled entity locations are used as training data to fine-tune the trained language model to obtain an adjusted entity recognition model, which is used to process entity recognition tasks in the target domain.

[0246] Optionally, the refined and extended functions of the program can be found in the description above.

[0247] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:

[0248] Obtain the knowledge graph of the target domain;

[0249] Walk sampling is performed from the knowledge graph to obtain at least one walk path. Each walk path contains a sequence of quintuples. Each quintuple in the sequence includes a head entity and its concept, a tail entity and its concept, and the relation name between the head entity and the tail entity.

[0250] For each traversal path, a corresponding training statement is generated based on the sequence of quintuples contained therein;

[0251] The entities, relation names, and / or concepts in the training statements are masked to obtain masked statements. The language model is trained with the goal of predicting the masked characters in the masked statements.

[0252] The entity locations in the training sentences are labeled, and the training sentences with labeled entity locations are used as training data to fine-tune the trained language model to obtain an adjusted entity recognition model, which is used to process entity recognition tasks in the target domain.

[0253] Optionally, the refined and extended functions of the program can be found in the description above.

[0254] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0255] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0256] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training an entity recognition model, characterized in that, include: Obtain the knowledge graph of the target domain; Select at least one node from the knowledge graph as the starting node; For each starting node, according to the sampling strategy, nodes in the knowledge graph are transitioned along directed edges with different probabilities. Each time a node undergoes a transition, a quintuple is generated based on the two nodes before and after the transition, as well as the directed edge. The probability of transitioning from the current node to any neighboring node during each transition is determined based on the distances between previously visited nodes and their neighboring nodes, and the semantic relationships between the current node and its neighboring nodes. Specifically, determining the probability of transitioning from the current node to any neighboring node based on the distances between previously visited nodes and their neighboring nodes, and the semantic relationships between the current node and its neighboring nodes, includes: determining a first transition score based on the distance between the current node's previous node and its neighboring nodes; for any node... A target neighbor node: Determine the semantic relationship between the current node and the target neighbor node to obtain the target relationship name. Count the total number of neighbor nodes of the current node that have the target relationship name. Based on the total number, the total number of all neighbor nodes of the current node, and a third hyperparameter, determine the second transfer score for the current node to move to the target neighbor node. The larger the total number, the smaller the second transfer score, ensuring that the node moves along a semantically clear relationship. The larger the third hyperparameter, the more the traversal path tends to move along a semantically clear relationship. Based on the first transfer score and the second transfer score, determine the total transfer score for the current node to move to the neighbor node. Based on the total transfer score, determine the probability of the current node moving to any neighbor node. The generated quintuples are added to the walk path corresponding to the starting node until the length of the walk path reaches the set length, thus obtaining the walk path corresponding to each starting node. Each walk path contains a sequence of quintuples. Each quintuple in the sequence includes the head entity and its concept, the tail entity and its concept, and the relation name between the head entity and the tail entity. For each traversal path, a corresponding training statement is generated based on the sequence of quintuples contained therein; The entities, relation names, and / or concepts in the training statements are masked to obtain masked statements. The language model is trained with the goal of predicting the masked characters in the masked statements. The entity locations in the training sentences are labeled, and the training sentences labeled with entity locations are used as training data to fine-tune the network parameters of the trained language model to obtain an adjusted entity recognition model, which is used to process entity recognition tasks in the target domain.

2. The method according to claim 1, characterized in that, The process of determining the probability of transitioning from the current node to any neighboring node based on the total transition score includes: For any target neighbor node: The probability of the current node moving to the target neighbor node is determined based on the total transfer score of the current node moving to the target neighbor node and the sum of the total transfer scores of the current node moving to each neighbor node.

3. The method according to claim 1, characterized in that, The step of determining the first transfer score for the current node to transfer to the neighboring node based on the distance between the current node's previous node and the neighboring node includes: Determine the shortest number of jumps between the previous node of the current node and the neighboring nodes. The shortest number of jumps is in the range of {0,1,2}. Different shortest number of jumps correspond to different transfer score settings. The transfer score setting corresponding to the shortest number of jumps 0 is controlled by the first hyperparameter, and the transfer score setting corresponding to the shortest number of jumps 2 is controlled by the second hyperparameter. Based on the shortest number of jumps between the current node's previous node and the neighboring node, a corresponding transfer score setting value is selected as the first transfer score for the current node to transfer to the neighboring node.

4. The method according to claim 1, characterized in that, The step of determining the total transfer score for the current node to transfer to the neighboring node based on the first transfer score and the second transfer score includes: The product of the first transfer score and the second transfer score is taken as the total transfer score for the current node to transfer to the neighboring node.

5. The method according to claim 1, characterized in that, For each traversal path, a corresponding training statement is generated based on the 5-tuple sequence it contains, including: For each traversal path, based on the quintuple sequence it contains, a corresponding training statement is generated according to a predefined generation strategy, wherein the generation strategy includes: Path concatenation: According to the sorting order of each quintuple in the traversal path, concatenate the head entity, relation name, and tail entity of each quintuple into a training statement containing subject, verb, and object; and / or, according to the sorting order of each quintuple in the traversal path, concatenate the head entity and its concept, relation name, tail entity and its concept of each quintuple into a training statement containing subject, verb, and object.

6. The method according to claim 5, characterized in that, The generation strategy also includes one or more of the following: Entity ablation: Some entities in the statement generated according to the path concatenation strategy are omitted and converted into implicit semantic representations to obtain the training statement; Object fronting: The positions of the subject and object in the sentences generated according to the path splicing strategy are swapped, and pronouns are added to ensure the semantic consistency of the training sentences; Concept replacement: In the statement generated according to the path concatenation strategy, some entities are replaced with corresponding concepts or higher-level concepts to obtain the training statement; Sentence transformation: Some entity and relation names in the statement generated according to the path concatenation strategy are omitted, and the statement is transformed into a question, rhetorical question and / or interrogative sentence for the omitted part, so as to obtain the transformed training statement.

7. The method according to any one of claims 1-6, characterized in that, Before masking the entities, relation names, and / or concepts in the training statements, and before labeling the entity positions in the training statements, the method further includes: The training statement is semantically expanded to obtain the expanded training statement; The process of masking entities, relation names, and / or concepts in the training statements, and labeling the positions of entities in the training statements, includes: Mask the entities, relation names, and / or concepts in the expanded training statement, and label the entity positions in the expanded training statement.

8. The method according to claim 7, characterized in that, The semantic expansion of the training statement to obtain the expanded training statement includes: The entities in the training statement are randomly reduced or repeated at a set ratio to obtain the first expanded training statement, which is used as the processing object for entity position annotation.

9. The method according to claim 7, characterized in that, The semantic expansion of the training statement to obtain the expanded training statement includes: A similar sentence generation model is used to generate similar sentences to the training sentences, which are then used as the second expanded training sentences. The similar sentence generation model is trained using similar sentence pairs as training data.

10. The method according to any one of claims 1-6, characterized in that, The annotation of entity positions in the training statements includes: The entity positions in the training statement are labeled in the form of a label matrix, where the label matrix is ​​an n×n matrix, n is the number of characters contained in the training statement, and the value of the element in the i-th row and j-th column of the label matrix indicates whether the text segment composed of the i-th character to the j-th character in the training statement belongs to an entity.

11. An entity recognition method, characterized in that, include: Obtain the text to be processed within the target domain; The text to be processed is input into the entity recognition model obtained by any one of the methods of claims 1-10, and the entities contained in the text to be processed are obtained from the model output.

12. A training device for an entity recognition model, characterized in that, include: The knowledge graph acquisition unit is used to acquire the knowledge graph of the target domain. A sampling unit is used to select at least one node in the knowledge graph as a starting node; For each starting node, according to the sampling strategy, nodes in the knowledge graph are transitioned along directed edges with different probabilities. Each time a node undergoes a transition, a quintuple is generated based on the two nodes before and after the transition, as well as the directed edge. The probability of transitioning from the current node to any neighboring node during each transition is determined based on the distance between the nodes previously traversed by the current node and the neighboring nodes, and the semantic relationship between the current node and the neighboring nodes. Specifically, determining the probability of transitioning from the current node to any neighboring node based on the distance between the nodes previously traversed by the current node and the neighboring nodes, and the semantic relationship between the current node and the neighboring nodes, includes: determining a first transition score based on the distance between the current node's previous node and the neighboring node; for any target neighboring node of the current node: determining the semantic relationship between the current node and the target neighboring node to obtain the target relationship name, and counting the number of neighboring nodes of the current node that are related to the current node. The total number of neighboring nodes with the target relation name is used to determine a second transfer score for the current node to move to the target neighboring node. Based on the total number of neighboring nodes, the total number of all neighboring nodes of the current node, and a third hyperparameter, the second transfer score is determined. A larger total number of neighboring nodes results in a smaller second transfer score, ensuring that the node moves along semantically explicit relations. A larger third hyperparameter indicates that the traversal path is more likely to move along semantically explicit relations. The total transfer score for the current node to move to the neighboring node is determined based on the first and second transfer scores. The probability of the current node moving to any neighboring node is determined based on the total transfer score. The generated quintuples are added to the traversal path corresponding to the starting node until the length of the traversal path reaches a set length, resulting in a traversal path corresponding to each starting node. Each traversal path contains a sequence of quintuples, where each quintuple includes a head entity and its concept, a tail entity and its concept, and the relation name between the head and tail entities. The training statement generation unit is used to generate corresponding training statements for each traversal path based on the sequence of 5-tuples contained therein. The model pre-training unit is used to mask the entities, relation names and / or concepts in the training statement to obtain the masked statement, and to train the language model with the goal of predicting the masked characters in the masked statement. The model fine-tuning unit is used to annotate the entity positions in the training sentences and use the annotated training sentences as training data to fine-tune the network parameters of the trained language model to obtain an adjusted entity recognition model for handling entity recognition tasks in the target domain.

13. An entity recognition model training device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the entity recognition model training method as described in any one of claims 1 to 10.

14. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the entity recognition model training method as described in any one of claims 1 to 10.

Images