Text entity recognition method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a text entity recognition method and device, electronic equipment and a storage medium, and belongs to the technical field of text processing and artificial intelligence. The method comprises the following steps: acquiring an entity annotation text; performing text coding on the entity annotation text based on the language sub-model to obtain a coded text vector, and performing context understanding on the coded text vector to obtain context representation of a target word group; on the basis of the conditional random field sub-model and the target phrase context representation, performing sample entity category prediction on the coded text vector to obtain a prediction category of a sample entity; based on the prediction category and the sample entity category, adjusting model parameters to obtain a target text entity recognition model; and performing entity recognition on the target text based on the target text entity recognition model to obtain a target entity category. According to the embodiment of the invention, the accuracy of text entity recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of text processing and artificial intelligence technology, and in particular to a text entity recognition method and device, an electronic device and a storage medium. Background Art

[0002] Text entity recognition refers to the identification and classification of entity phrases in text. For example, in the medical field, by identifying symptom entities in medical text, doctors can accelerate their understanding of the patient's condition.

[0003] At present, the common method of text entity recognition is usually to use a separate conditional random field model to perform entity recognition on text. However, the recognition accuracy of this model is low, and it does not handle complex entity boundaries and nested entities well, which can easily lead to text entity recognition errors. Therefore, how to improve the accuracy of text entity recognition has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a text entity recognition method and device, an electronic device and a storage medium, aiming to improve the accuracy of text entity recognition.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a text entity recognition method, the method comprising:

[0006] Acquire entity annotation text annotated with a sample entity category, wherein the entity annotation text includes a sample entity, and the sample entity category is used to indicate a category of the sample entity;

[0007] Acquire an initial text entity recognition model, wherein the initial text entity recognition model includes a language sub-model and a conditional random field sub-model;

[0008] Performing text encoding on the entity annotation text based on the language sub-model to obtain an encoded text vector;

[0009] Performing context understanding on the encoded text vector based on the language sub-model to obtain a context representation of the target phrase;

[0010] Based on the conditional random field sub-model and the target phrase context representation, predicting the sample entity category of the encoded text vector to obtain the predicted category of the sample entity;

[0011] Based on the predicted category and the sample entity category, adjusting the initial text entity recognition model to obtain a target text entity recognition model;

[0012] Entity recognition is performed on the target text based on the target text entity recognition model to obtain a target entity category, wherein the target text contains a target entity, the target entity category represents the category of the target entity, and the target entity category is at least one of the sample entity categories.

[0013] In some embodiments, the language sub-model includes a first semantic understanding model and a second semantic understanding model, and performing context understanding on the encoded text vector based on the language sub-model to obtain a context representation of the target phrase includes:

[0014] Based on the first semantic understanding model, performing phrase context semantic learning on the encoded text vector to obtain an initial phrase context representation;

[0015] Based on the second semantic understanding model and the initial phrase context representation, deep context learning is performed on the encoded text vector to obtain the target phrase context representation.

[0016] In some embodiments, performing phrase context semantic learning on the encoded text vector based on the first semantic understanding model to obtain an initial phrase context representation includes:

[0017] Based on the first semantic understanding model, segment the encoded text vector into sentences to obtain an encoded sentence vector;

[0018] Adding a tag to the encoded sentence vector to obtain a sentence identifier;

[0019] Performing phrase segmentation on the encoded sentence vector to obtain an encoded word vector;

[0020] Based on the first semantic understanding model and the sentence identifier, the encoded word vector is subjected to neighboring phrase context understanding to obtain the initial phrase context representation.

[0021] In some embodiments, the performing neighboring phrase context understanding on the encoded word vector based on the first semantic understanding model and the sentence identifier to obtain the initial phrase context representation includes:

[0022] Based on the sentence identifier, perform a neighboring phrase search on the encoded word vector to obtain a target neighboring phrase vector;

[0023] Based on the target neighboring phrase vector, the encoded word vector is semantically understood to obtain the initial phrase context representation.

[0024] In some embodiments, the performing deep context learning on the encoded text vector based on the second semantic understanding model and the initial phrase context representation to obtain the target phrase context representation includes:

[0025] Based on the second semantic understanding model, performing attention weight allocation on the encoded text vector to obtain a phrase attention weight;

[0026] Based on the phrase attention weight, performing context understanding on the encoded text vector to obtain an attention context representation;

[0027] Performing nonlinear changes on the initial phrase context representation to obtain deep features of the encoded text vector;

[0028] Perform a residual connection on the attention context representation and the deep feature to obtain the target phrase context representation.

[0029] In some embodiments, the step of predicting a sample entity category for the encoded text vector based on the conditional random field sub-model and the target phrase context representation to obtain a predicted category of the sample entity includes:

[0030] Based on the conditional random field sub-model and the target phrase context representation, linearly transform the encoded text vector to obtain an entity state score;

[0031] Based on the conditional random field sub-model, the label transfer probability of the encoded text vector is calculated to obtain the entity label transfer probability;

[0032] Merging the entity state score and the entity label transition probability to obtain a sample entity category probability;

[0033] The predicted category is determined based on the sample entity category probability.

[0034] In some embodiments, the adjusting the initial text entity recognition model based on the predicted category and the sample entity category to obtain a target text entity recognition model includes:

[0035] Based on the predicted category and the sample entity category, calculating sample entity category loss data of the initial text entity recognition model;

[0036] Based on the sample entity category loss data, the parameters of the initial text entity recognition model are adjusted to obtain the target text entity recognition model.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a text entity recognition device, the device comprising:

[0038] An annotated text acquisition module, used to acquire entity annotated text annotated with a sample entity category, wherein the entity annotated text includes a sample entity, and the sample entity category is used to indicate the category of the sample entity;

[0039] An initial model building module, used to obtain an initial text entity recognition model, wherein the initial text entity recognition model includes a language sub-model and a conditional random field sub-model;

[0040] A label text encoding module, used for encoding the entity label text based on the language sub-model to obtain an encoded text vector;

[0041] A context learning module, used for performing context understanding on the encoded text vector based on the language sub-model to obtain a context representation of a target phrase;

[0042] An entity category prediction module, used to predict the sample entity category of the encoded text vector based on the conditional random field sub-model and the target phrase context representation, to obtain the predicted category of the sample entity;

[0043] An initial model optimization module, used to adjust the initial text entity recognition model based on the predicted category and the sample entity category to obtain a target text entity recognition model;

[0044] The text entity recognition module is used to perform entity recognition on the target text based on the target text entity recognition model to obtain a target entity category, wherein the target text contains a target entity, the target entity category represents the category of the target entity, and the target entity category is at least one of the sample entity categories.

[0045] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0046] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0047] The text entity recognition method and device, electronic device and storage medium proposed in the present application use the language model in the initial text entity recognition model to perform text encoding on the entity annotation text to obtain the encoded text vector, and continue to use the language sub-model to perform context understanding on the encoded text vector to obtain the context representation of the target phrase, which can improve the semantic understanding of the sample entity in the entity annotation text by the initial text entity recognition model, thereby improving the accuracy of text entity recognition. Secondly, the conditional random field sub-model and the target phrase context representation in the initial text entity recognition model are used to predict the sample entity category of the encoded text vector to obtain the predicted category of the sample entity, which can reduce the dependence on manually annotated text data. In addition, a language sub-model is added on the basis of the conditional random field sub-model, so that the initial text entity recognition model can handle the problems of nested entities and complex entity boundaries, thereby improving the accuracy of text entity recognition. Finally, according to the predicted category and the sample entity category in the entity annotation text, the initial text entity recognition model is adjusted to obtain the target text entity recognition model, which can improve the accuracy of the model in text entity recognition, thereby improving the accuracy of text entity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flowchart of a text entity recognition method provided by an embodiment of the present application;

[0049] Figure 2 yes Figure 1 Flow chart of step S104 in FIG.

[0050] Figure 3 yes Figure 2 Flow chart of step S201 in FIG.

[0051] Figure 4 yes Figure 3 Flow chart of step S304 in FIG.

[0052] Figure 5 yes Figure 2 Flow chart of step S202 in FIG.

[0053] Figure 6 yes Figure 1 Flow chart of step S105 in FIG.

[0054] Figure 7 yes Figure 1 Flow chart of step S106 in FIG.

[0055] Figure 8 is a structural diagram of a text entity recognition device provided in an embodiment of the present application;

[0056] Fig. 9It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0060] First, some nouns involved in this application are analyzed:

[0061] Bidirectional Encoder Model: The Bidirectional Encoder Model is a pre-trained language model based on the Transformer architecture that can effectively capture contextual information in the text. The Bidirectional Encoder Model achieves bidirectional encoding by simultaneously considering the semantics of the left and right sides of a word in the context, and is suitable for various natural language processing tasks such as text classification, question answering, and named entity recognition. Its pre-training process includes two main tasks: masked language model and next sentence prediction, which gives it powerful text understanding capabilities and is widely used in various language understanding applications.

[0062] Conditional Random Fields: Conditional random fields are a statistical modeling method for labeling and segmenting sequence data. They are particularly suitable for tasks in natural language processing and computer vision, such as named entity recognition and image segmentation. Conditional random fields improve the accuracy of predictions by modeling conditional probability distributions and considering the dependencies between adjacent labels. Unlike traditional independent hypothesis methods, conditional random fields can effectively capture the interactions between labels in the entire sequence, ensuring that the generated label sequence is globally consistent and has strong flexibility and expressiveness.

[0063] Attention mechanism: The attention mechanism is a computational method that mimics human visual attention, which aims to enable the model to dynamically focus on the most relevant information when processing input data, thereby improving performance. Its benefit is that it can effectively capture long-distance dependencies and important features, significantly improving the accuracy and efficiency of tasks such as natural language processing, image recognition, and translation. By adaptively adjusting the focus of attention, the attention mechanism enhances the model's expressiveness and enables it to perform better in complex tasks.

[0064] Multi-head attention mechanism: Multi-head attention mechanism is a deep learning technique for simultaneously focusing on different parts of the input when processing sequence data. It captures different contextual information by dividing the input features into multiple subspaces (i.e., multiple "heads"), each of which independently calculates attention weights. Finally, the outputs of these heads are concatenated or weighted to form a rich feature representation. Multi-head attention mechanism is widely used in fields such as natural language processing and computer vision, enhancing the model's ability to understand complex dependencies.

[0065] Text entity recognition refers to the identification and classification of entity phrases in text. For example, in the medical field, by identifying symptom entities in medical text, doctors can accelerate their understanding of the patient's condition.

[0066] At present, the common method of text entity recognition is usually to use a separate conditional random field model to perform entity recognition on text. However, the recognition accuracy of this model is low, and it does not handle complex entity boundaries and nested entities well, which can easily lead to text entity recognition errors. Therefore, how to improve the accuracy of text entity recognition has become a technical problem that needs to be solved urgently.

[0067] Based on this, the embodiments of the present application provide a text entity recognition method and device, an electronic device and a storage medium, aiming to improve the accuracy of text entity recognition.

[0068] The text entity recognition method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the text entity recognition method in the embodiments of the present application is described.

[0069] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0070] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0071] The text entity recognition method provided in the embodiment of the present application relates to the field of text processing and artificial intelligence technology. The text entity recognition method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the text entity recognition method, etc., but is not limited to the above forms.

[0072] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0073] Figure 1 is an optional flowchart of the text entity recognition method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S107.

[0074] Step S101, obtaining entity annotation text annotated with a sample entity category, wherein the entity annotation text includes a sample entity, and the sample entity category is used to indicate the category of the sample entity;

[0075] Step S102, obtaining an initial text entity recognition model, wherein the initial text entity recognition model includes a language sub-model and a conditional random field sub-model;

[0076] Step S103, performing text encoding on the entity annotation text based on the language sub-model to obtain an encoded text vector;

[0077] Step S104, performing context understanding on the encoded text vector based on the language sub-model to obtain a context representation of the target phrase;

[0078] Step S105, based on the conditional random field sub-model and the target phrase context representation, perform sample entity category prediction on the encoded text vector to obtain the predicted category of the sample entity;

[0079] Step S106, based on the predicted category and the sample entity category, the initial text entity recognition model is adjusted to obtain a target text entity recognition model;

[0080] Step S107, performing entity recognition on the target text based on the target text entity recognition model to obtain a target entity category, wherein the target text contains a target entity, the target entity category represents the category of the target entity, and the target entity category is at least one of the sample entity categories.

[0081] In the steps S101 to S107 shown in the embodiment of the present application, entity annotated text is obtained by acquiring text annotated with sample entity categories, wherein the entity annotated text contains sample entities, and the sample entity categories are used to represent the categories of the sample entities, and then an initial text entity recognition model including a language sub-model and a conditional random field sub-model is obtained. Furthermore, the entity annotated text is encoded using the above-mentioned language sub-model to obtain an encoded text vector, and then the language sub-model is used to perform context understanding on the encoded text vector to obtain a target phrase context representation. Secondly, the conditional random field sub-model and the target phrase context representation are used to predict the sample entity category of the encoded text vector to obtain the predicted category of the sample entity, and the initial text entity recognition model is adjusted according to the predicted category and the sample entity category to obtain a target text entity recognition model. Finally, the target text entity recognition model is used to perform entity recognition on the target text containing the target entity to obtain a target entity category representing the category of the target entity, wherein the target entity category is at least one of the above-mentioned sample entity categories. Therefore, the present application uses the language model in the initial text entity recognition model to perform text encoding on the entity annotation text to obtain the encoded text vector, and continues to use the language sub-model to perform context understanding on the encoded text vector to obtain the context representation of the target phrase, which can improve the semantic understanding of the sample entities in the entity annotation text by the initial text entity recognition model, thereby improving the accuracy of text entity recognition. Secondly, the conditional random field sub-model and the target phrase context representation in the initial text entity recognition model are used to predict the sample entity category of the encoded text vector to obtain the predicted category of the sample entity, which can reduce the dependence on manually annotated text data. In addition, a language sub-model is added on the basis of the conditional random field sub-model, so that the initial text entity recognition model can handle the problems of nested entities and complex entity boundaries, thereby improving the accuracy of text entity recognition. Finally, according to the predicted category and the sample entity category in the entity annotation text, the initial text entity recognition model is adjusted to obtain the target text entity recognition model, which can improve the accuracy of the model in text entity recognition, thereby improving the accuracy of text entity recognition.

[0082] In step S101 of some embodiments, the sample entity category can be used to represent the phrase category of the entity phrase, such as disease symptoms, drug names, etc. The sample entity refers to a phrase or word with a specific meaning in the entity annotation text, such as amoxicillin, fever, etc.

[0083] The present application can obtain text phrases or text words by performing word segmentation processing on a small amount of initial text, and then perform named entity recognition on the text phrases or text words to determine the entity category of the sample entity in the initial text, that is, the sample entity category, and annotate the initial text based on the sample entity category to obtain entity annotated text.

[0084] It should be noted that the present application only needs to annotate a small amount of initial text to obtain entity annotated text, that is, a small amount of sample data is used for model training, thereby reducing the cost and resource consumption of text entity annotation.

[0085] In step S102 of some embodiments, the initial text entity recognition model can be a large model composed of two sub-models, wherein the two sub-models are a language sub-model and a conditional random field sub-model, and the conditional random field sub-model can be a CRF (Conditional Random Fields) model. In detail, the language sub-model can be composed of two semantic understanding models, and the two semantic understanding models are a first semantic understanding model and a second semantic understanding model. The first semantic understanding model can be a Bert (Bidirectional Encoder Representations from Transformers) model, and the second semantic understanding model can be a transformer conversion model, i.e., a Transformer model.

[0086] This application constructs a Bert model, a Transformer model and a CRF model respectively through pre-set parameters and model architecture, and then connects the output of the Bert model with the input of the Transformer model, and connects the output of the Transformer model with the input of the CRF model, so as to obtain an initial text entity recognition model.

[0087] It is important to know that by connecting the output of the Bert model with the input of the Transformer model, and connecting the output of the Transformer model with the input of the CRF model, the CRF model's semantic understanding of the input entity annotation text can be improved, thereby improving the accuracy of the CRF model in identifying entity categories in entity annotation text.

[0088] In step S103 of some embodiments, the first semantic understanding model of the language sub-model may be used to encode the entity annotation text input into the initial text entity recognition model, thereby obtaining an encoded text vector.

[0089] In step S104 of some embodiments, after obtaining the encoded text vector of the entity annotated text, the first semantic understanding model in the above-mentioned language sub-model is used to perform a first contextual understanding of the encoded text vector to obtain an initial phrase context representation, and then the second semantic understanding model in the above-mentioned language sub-model is used to perform a second contextual understanding of the encoded text vector based on the obtained initial phrase context representation to obtain a target phrase context representation.

[0090] For details, see Figure 2 In some embodiments, step S104 may include but is not limited to steps S201 to S202:

[0091] Step S201, based on the first semantic understanding model, performing phrase context semantic learning on the encoded text vector to obtain an initial phrase context representation;

[0092] Step S202: Based on the second semantic understanding model and the initial phrase context representation, deep context learning is performed on the encoded text vector to obtain the target phrase context representation.

[0093] In step S201 of some embodiments, the encoded text vector is sentence segmented using a first semantic understanding model to obtain an encoded sentence vector for each sentence in the entity annotated text, and then the initial phrase context representation of each target phrase is determined based on the help of neighboring phrases of the target phrase in the encoded sentence vector for semantic understanding of the target phrase, wherein the target phrase can be any phrase or word in the encoded sentence vector.

[0094] For details, see Figure 3 In some embodiments, step S201 may include but is not limited to steps S301 to S304:

[0095] Step S301, based on the first semantic understanding model, sentence segmentation is performed on the encoded text vector to obtain an encoded sentence vector;

[0096] Step S302, adding a tag to the encoded sentence vector to obtain a sentence identifier;

[0097] Step S303, segmenting the encoded sentence vector into phrases to obtain encoded word vectors;

[0098] Step S304, based on the first semantic understanding model and the sentence identifier, the encoded word vector is subjected to neighboring phrase context understanding to obtain an initial phrase context representation.

[0099] In step S301 of some embodiments, the encoded text vector is segmented into sentences according to the division of each sentence in the entity annotated text, so as to obtain the encoded sentence vector.

[0100] In step S302 of some embodiments, a sentence start identification tag is added at the beginning of each encoded sentence vector and a sentence end identification tag is added at the end of each encoded sentence vector, thereby marking the encoded sentence vector and obtaining a sentence identifier.

[0101] In step S303 of some embodiments, by segmenting the phrases in the text sentence represented by each encoded sentence vector, a vector representation of each phrase, ie, an encoded word vector, can be obtained.

[0102] In step S304 of some embodiments, according to the above sentence identification, the target neighboring phrase vector of the encoded word vector is searched, and then the encoded word vector is semantically understood in combination with the target neighboring phrase vector to obtain the initial phrase context representation.

[0103] For details, see Figure 4 In some embodiments, step S304 may include but is not limited to steps S401 to S402:

[0104] Step S401, based on the sentence identifier, perform a neighboring phrase search on the encoded word vector to obtain a target neighboring phrase vector;

[0105] Step S402, based on the target neighboring phrase vector, semantic understanding is performed on the encoded word vector to obtain an initial phrase context representation.

[0106] In step S401 of some embodiments, the sentence identifier can be used to limit the scope of the encoded word vector to search for neighboring phrases. For example, in the sentence "Xiao Ming has a fever recently", the neighboring phrases of "Xiao Ming" are only "recently", while the neighboring phrases of "point" can be "have" and "fever".

[0107] According to the above sentence identification, the present application searches for the neighboring phrases of the phrase represented by each encoded word vector to obtain the target neighboring phrase, and then obtains the encoded vector representation of the target neighboring phrase to obtain the target neighboring phrase vector.

[0108] In step S402 of some embodiments, combining the target neighboring phrase vector of the encoded word vector can help the first semantic understanding model deepen its semantic understanding of the encoded word vector, thereby obtaining the initial phrase context representation of the encoded word vector. Specifically, the first semantic understanding model can perform semantic understanding of the encoded word vector based on the word meanings of the phrase represented by the encoded word vector and the target neighboring phrase vector.

[0109] In step S401 to step S402 of this embodiment, according to the sentence identifier, the encoded word vector is searched for adjacent phrases to obtain the target adjacent phrase vector, and then the encoded word vector is semantically understood according to the target adjacent phrase vector to obtain the initial phrase context representation.

[0110] In steps S301 to S304 shown in the present embodiment, the first semantic understanding model is used to perform sentence segmentation on the encoded text vector to obtain an encoded sentence vector, and then a mark is added to the encoded sentence vector to obtain a sentence identifier, which helps the first semantic understanding model to distinguish the beginning and end of a sentence. Furthermore, the first semantic understanding model is used to perform phrase segmentation on the encoded sentence vector to obtain an encoded word vector, and then the encoded word vector is subjected to neighboring phrase context understanding based on the first semantic understanding model and the sentence identifier to obtain an initial phrase context representation, which can preliminarily improve the initial text entity recognition model's semantic understanding of sample entities in the entity annotated text, thereby making the initial text entity recognition model more accurate in identifying the entity category of the sample entity.

[0111] In step S202 of some embodiments, the attention mechanism in the second semantic understanding model is used to perform feature transformation on the encoded text to obtain an attention context representation of the encoded text vector, and then a nonlinear operation is performed on the initial phrase context representation output by the first semantic understanding model to obtain the deep features of the encoded text vector. Finally, the aforementioned attention context representation is residually connected with the deep features to obtain the target phrase context representation.

[0112] For details, see Figure 5 In some embodiments, step S202 may include but is not limited to steps S501 to S504:

[0113] Step S501, based on the second semantic understanding model, assigning attention weights to the encoded text vectors to obtain phrase attention weights;

[0114] Step S502, based on the phrase attention weight, context understanding is performed on the encoded text vector to obtain an attention context representation;

[0115] Step S503, performing nonlinear changes on the initial phrase context representation to obtain deep features of the encoded text vector;

[0116] Step S504, performing residual connection on the attention context representation and the deep features to obtain the context representation of the target phrase.

[0117] In steps S501 and S502 of some embodiments, the multi-head attention mechanism in the second semantic understanding model can be used to assign attention weights to the encoded text vectors, so that the phrase attention weights of each encoded word vector can be obtained. Then, based on the phrase attention weights of the encoded word vectors, a deeper contextual understanding of each encoded word vector can be performed, and the attention context representation of the encoded word vector can be obtained. In detail, the second semantic understanding model can perform semantic understanding of any target phrase based on all the phrases in a sentence, thereby deepening the contextual understanding of the target phrase. For example, in the sentence "Xiao Ming has a little fever recently", the phrase "fever" can be deeply contextualized based on the phrases "Xiao Ming", "recently", "have", and "a little", thereby obtaining an attention context representation.

[0118] In step S503 of some embodiments, the nonlinear activation function in the second semantic understanding model can be used to perform nonlinear changes on the initial phrase context representation output by the first semantic understanding model, so as to obtain deep features of each encoded word vector.

[0119] In step S504 of some embodiments, residual connection is performed between the attention context representation of the above-mentioned encoded word vector and the deep features of the above-mentioned encoded word vector, which can deepen the contextual understanding of the encoded word vector and avoid forgetting detailed information, so as to obtain the contextual representation of the target phrase.

[0120] In steps S501 to S504 shown in the present embodiment, attention weights are allocated to the encoded text vector according to the second semantic understanding model to obtain phrase attention weights, and then contextual understanding is performed on the encoded text vector according to the phrase attention weights to obtain attention context representation, which can further improve the semantic understanding of the sample entity by the initial text entity recognition model. Furthermore, the second semantic understanding model is used to perform nonlinear changes on the initial phrase context representation to obtain deep features of the encoded text vector, and then the attention context representation and the deep features are residually connected to obtain the target phrase context representation, which can avoid missing detailed information of the encoded word vector in the semantic understanding process, thereby further improving the semantic understanding of the sample entity by the initial text entity recognition model and improving the accuracy of text entity recognition.

[0121] In steps S201 to S203 shown in the present embodiment, by utilizing the first semantic understanding model, phrase context semantic learning is performed on the encoded text vector to obtain an initial phrase context representation, and then utilizing the second semantic understanding model and the initial phrase context representation to perform deep context learning on the encoded text vector to obtain a target phrase context representation. This can deepen the semantic understanding of each phrase or word in the entity annotated text, thereby making the model more accurate in identifying the category of entity phrases in the entity annotated text and improving the accuracy of text entity recognition.

[0122] In step 105 of some embodiments, the entity state score and entity label transition probability of each sample entity can be calculated based on the target phrase context representation using the conditional random field sub-model, and then the entity state score and entity label transition probability are merged to obtain the sample entity category probability of the sample entity. Finally, the predicted category of the sample entity can be determined based on the sample entity category probability.

[0123] For details, see Figure 6 In some embodiments, step S105 may include but is not limited to steps S601 to S604:

[0124] Step S601, based on the conditional random field sub-model and the target phrase context representation, a linear transformation is performed on the encoded text vector to obtain an entity state score;

[0125] Step S602, based on the conditional random field sub-model, calculate the label transfer probability of the encoded text vector to obtain the entity label transfer probability;

[0126] Step S603, merging the entity state score and the entity label transition probability to obtain the sample entity category probability;

[0127] Step S604: Determine the predicted category based on the sample entity category probability.

[0128] In step S601 of some embodiments, the encoded word vector representing the sample entity can be linearly calculated based on the linear activation function in the conditional random field sub-model and the above-mentioned target phrase context representation, so as to obtain the entity state score of the encoded word vector representing the sample entity.

[0129] In step S602 of some embodiments, the trained label transition probability prediction model can be placed in a conditional random field sub-model so that the conditional random field sub-model can calculate the entity label transition probability of the encoded word vector of the sample entity.

[0130] In step S603 of some embodiments, the Viterbi algorithm may be used to merge the entity state score and the entity label transition probability to obtain the category probability of the sample entity, that is, the sample entity category probability.

[0131] In step S604 of some embodiments, the confidence of the sample entity in each entity category can be determined based on the sample entity category probability, and then the entity category with the highest confidence is used as the predicted category of the sample entity.

[0132] In steps S601 to S604 shown in the present embodiment, the encoded text vector is linearly transformed according to the conditional random field sub-model and the target phrase context representation to obtain an entity state score, and then the label transfer probability of the encoded text vector is calculated to obtain an entity label transfer probability, and the entity state score and the entity label transfer probability are merged to obtain the confidence of the sample entity in each entity category, that is, the sample entity category, which can improve the accuracy of the model in recognizing text entities. Furthermore, according to the sample entity category probability, the predicted category of the sample entity can be clarified.

[0133] In step S106 of some embodiments, by calculating the difference between the predicted category output by the initial text entity recognition model and the sample entity category annotated in the above-mentioned entity annotation text, the gap between the predicted category and the sample entity category, that is, the sample entity category loss data, can be clearly identified. Then, according to the sample entity category loss data, the parameters of the initial text entity recognition model are adjusted to obtain the target text entity recognition model.

[0134] For details, see Figure 7 In some embodiments, step S106 may include but is not limited to steps S701 to S702:

[0135] Step S701, calculating sample entity category loss data of the initial text entity recognition model based on the predicted category and the sample entity category;

[0136] Step S702: Based on the sample entity category loss data, the parameters of the initial text entity recognition model are adjusted to obtain a target text entity recognition model.

[0137] In step S701 of some embodiments, the sample entity category loss function of the initial text entity recognition model can be used to calculate the gap between the predicted category and the sample entity category, that is, the sample entity category loss data, wherein the sample entity category loss function can be a loss function set when obtaining the initial text entity recognition model.

[0138] In step S702 of some embodiments, the initial text entity recognition model is back-propagated according to the above-mentioned sample entity category loss data, and while back-propagating, the parameters of the initial text entity recognition model are optimized and adjusted so that the initial text entity recognition model is converted into a target text entity recognition model.

[0139] In steps S701 to S702 shown in this embodiment, the sample entity category loss data of the initial text entity recognition model is calculated based on the predicted category and the sample entity category, and then the parameters of the initial text entity recognition model are adjusted based on the sample entity category loss data to obtain the target text entity recognition model, which can improve the model's recognition accuracy of entity categories, thereby improving the accuracy of text entity recognition.

[0140] In step S107 of some embodiments, the target text containing the target entity can be input into the trained target text entity recognition model. The target text entity recognition model processes the target text through the methods of steps S103 to S105 to obtain the target entity category of the target text.

[0141] In an application scenario, taking entity recognition of medical text as an example, the entity category of the medical text is obtained by annotating the entity phrases in the medical text, and then an initial medical text entity recognition model composed of the Bert model, the Transformer model and the CRF model is obtained. Among them, the Bert model and the Transformer model belong to the language sub-model, and the CRF model belongs to the conditional random field sub-model. The Bert model is used to encode the above medical text to obtain the encoded medical text vector of the feature information in the medical text. Furthermore, the Bert model and the Transformer model are used to encode the encoded medical text vector. The text vector is used for contextual understanding to obtain the target phrase context representation of each phrase in the medical text. Then, based on the target phrase context representation of each phrase in the medical text, the CRF model is used to predict the entity category of each phrase in the medical text to obtain the predicted entity category of each medical phrase. Secondly, according to the predicted entity category and the above-mentioned medical entity category, the initial medical text entity recognition model is adjusted to convert the initial medical text entity recognition model into a trained target medical text entity recognition model. Finally, the target medical text entity recognition model is used to perform text entity recognition on medical texts without labeled entity category information to obtain the target entity category of the medical entity.

[0142] The present application uses the language model in the initial text entity recognition model to perform text encoding on the entity annotation text to obtain the encoded text vector, and continues to use the language sub-model to perform context understanding on the encoded text vector to obtain the context representation of the target phrase, which can improve the semantic understanding of the sample entities in the entity annotation text by the initial text entity recognition model, thereby improving the accuracy of text entity recognition. Secondly, the conditional random field sub-model and the target phrase context representation in the initial text entity recognition model are used to predict the sample entity category of the encoded text vector to obtain the predicted category of the sample entity, which can reduce the dependence on manually annotated text data. In addition, a language sub-model is added on the basis of the conditional random field sub-model, so that the initial text entity recognition model can handle the problems of nested entities and complex entity boundaries, thereby improving the accuracy of text entity recognition. Finally, according to the predicted category and the sample entity category in the entity annotation text, the initial text entity recognition model is adjusted to obtain the target text entity recognition model, which can improve the accuracy of the model in text entity recognition, thereby improving the accuracy of text entity recognition.

[0143] See also Figure 8 The present application also provides a text entity recognition device, which can implement the above text entity recognition method, and the device includes:

[0144] The annotated text acquisition module 801 is used to acquire entity annotated text annotated with a sample entity category, wherein the entity annotated text includes a sample entity, and the sample entity category is used to indicate the category of the sample entity;

[0145] An initial model building module 802 is used to obtain an initial text entity recognition model, wherein the initial text entity recognition model includes a language sub-model and a conditional random field sub-model;

[0146] The annotation text encoding module 803 is used to perform text encoding on the entity annotation text based on the language sub-model to obtain an encoded text vector;

[0147] A context learning module 804 is used to perform context understanding on the encoded text vector based on the language sub-model to obtain a context representation of the target phrase;

[0148] An entity category prediction module 805 is used to predict the sample entity category of the encoded text vector based on the conditional random field sub-model and the target phrase context representation to obtain the predicted category of the sample entity;

[0149] An initial model optimization module 806 is used to adjust the initial text entity recognition model based on the predicted category and the sample entity category to obtain a target text entity recognition model;

[0150] The text entity recognition module 807 is used to perform entity recognition on the target text based on the target text entity recognition model to obtain a target entity category, wherein the target text contains a target entity, the target entity category represents the category of the target entity, and the target entity category is at least one of the sample entity categories.

[0151] The specific implementation of the text entity recognition device is basically the same as the specific implementation of the above-mentioned text entity recognition method, and will not be repeated here.

[0152] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned text entity recognition method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0153] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0154] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0155] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the text entity recognition method of the embodiment of this application;

[0156] Input / output interface 903, used to implement information input and output;

[0157] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0158] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0159] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0160] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned text entity recognition method is implemented.

[0161] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0162] The text entity recognition method, text entity recognition device, electronic device and storage medium provided in the embodiments of the present application obtain text annotated with sample entity categories to obtain entity annotated text, wherein the entity annotated text contains sample entities, and the sample entity categories are used to represent the categories of the sample entities, and then an initial text entity recognition model including a language sub-model and a conditional random field sub-model is obtained. Further, the entity annotated text is encoded using the above-mentioned language sub-model to obtain an encoded text vector, and then the language sub-model is used to perform context understanding on the encoded text vector to obtain a target phrase context representation. Secondly, the conditional random field sub-model and the target phrase context representation are used to predict the sample entity category of the encoded text vector to obtain the predicted category of the sample entity, and the initial text entity recognition model is adjusted according to the predicted category and the sample entity category to obtain a target text entity recognition model. Finally, the target text entity recognition model is used to perform entity recognition on the target text containing the target entity to obtain a target entity category representing the category of the target entity, wherein the target entity category is at least one of the above-mentioned sample entity categories.

[0163] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0164] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0165] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0167] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0168] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0170] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0172] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0173] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A text entity recognition method, characterized in that: The method comprises: Acquire entity annotation text annotated with a sample entity category, wherein the entity annotation text includes a sample entity, and the sample entity category is used to indicate a category of the sample entity; Acquire an initial text entity recognition model, wherein the initial text entity recognition model includes a language sub-model and a conditional random field sub-model; Performing text encoding on the entity annotation text based on the language sub-model to obtain an encoded text vector; Performing context understanding on the encoded text vector based on the language sub-model to obtain a context representation of the target phrase; Based on the conditional random field sub-model and the target phrase context representation, predicting the sample entity category of the encoded text vector to obtain the predicted category of the sample entity; Based on the predicted category and the sample entity category, adjusting the initial text entity recognition model to obtain a target text entity recognition model; Entity recognition is performed on the target text based on the target text entity recognition model to obtain a target entity category, wherein the target text contains a target entity, the target entity category represents the category of the target entity, and the target entity category is at least one of the sample entity categories.

2. The method according to claim 1, characterized in that The language sub-model includes a first semantic understanding model and a second semantic understanding model, and the context understanding of the encoded text vector based on the language sub-model to obtain the context representation of the target phrase includes: Based on the first semantic understanding model, performing phrase context semantic learning on the encoded text vector to obtain an initial phrase context representation; Based on the second semantic understanding model and the initial phrase context representation, deep context learning is performed on the encoded text vector to obtain the target phrase context representation.

3. The method according to claim 2, characterized in that The step of performing phrase context semantic learning on the encoded text vector based on the first semantic understanding model to obtain an initial phrase context representation includes: Based on the first semantic understanding model, segment the encoded text vector into sentences to obtain an encoded sentence vector; Adding a tag to the encoded sentence vector to obtain a sentence identifier; Performing phrase segmentation on the encoded sentence vector to obtain an encoded word vector; Based on the first semantic understanding model and the sentence identifier, the encoded word vector is subjected to neighboring phrase context understanding to obtain the initial phrase context representation.

4. The method according to claim 3, characterized in that: The step of performing neighboring phrase context understanding on the encoded word vector based on the first semantic understanding model and the sentence identifier to obtain the initial phrase context representation includes: Based on the sentence identifier, perform a neighboring phrase search on the encoded word vector to obtain a target neighboring phrase vector; Based on the target neighboring phrase vector, the encoded word vector is semantically understood to obtain the initial phrase context representation.

5. The method according to claim 2, characterized in that: The step of performing deep context learning on the encoded text vector based on the second semantic understanding model and the initial phrase context representation to obtain the target phrase context representation includes: Based on the second semantic understanding model, performing attention weight allocation on the encoded text vector to obtain a phrase attention weight; Based on the phrase attention weight, performing context understanding on the encoded text vector to obtain an attention context representation; Performing nonlinear changes on the initial phrase context representation to obtain deep features of the encoded text vector; Perform a residual connection on the attention context representation and the deep feature to obtain the target phrase context representation.

6. The method according to any one of claims 1 to 5, characterized in that: The step of performing sample entity category prediction on the encoded text vector based on the conditional random field sub-model and the target phrase context representation to obtain the predicted category of the sample entity includes: Based on the conditional random field sub-model and the target phrase context representation, linearly transform the encoded text vector to obtain an entity state score; Based on the conditional random field sub-model, the label transfer probability of the encoded text vector is calculated to obtain the entity label transfer probability; Merging the entity state score and the entity label transition probability to obtain a sample entity category probability; The predicted category is determined based on the sample entity category probability.

7. The method according to any one of claims 1 to 5, characterized in that: The method of adjusting the initial text entity recognition model based on the predicted category and the sample entity category to obtain a target text entity recognition model includes: Based on the predicted category and the sample entity category, calculating sample entity category loss data of the initial text entity recognition model; Based on the sample entity category loss data, the parameters of the initial text entity recognition model are adjusted to obtain the target text entity recognition model.

8. A text entity recognition device, characterized in that: The device comprises: An annotated text acquisition module, used to acquire entity annotated text annotated with a sample entity category, wherein the entity annotated text includes a sample entity, and the sample entity category is used to indicate the category of the sample entity; An initial model building module, used to obtain an initial text entity recognition model, wherein the initial text entity recognition model includes a language sub-model and a conditional random field sub-model; A label text encoding module, used for encoding the entity label text based on the language sub-model to obtain an encoded text vector; A context learning module, used for performing context understanding on the encoded text vector based on the language sub-model to obtain a context representation of a target phrase; An entity category prediction module, used to predict the sample entity category of the encoded text vector based on the conditional random field sub-model and the target phrase context representation, to obtain the predicted category of the sample entity; An initial model optimization module, used to adjust the initial text entity recognition model based on the predicted category and the sample entity category to obtain a target text entity recognition model; The text entity recognition module is used to perform entity recognition on the target text based on the target text entity recognition model to obtain a target entity category, wherein the target text contains a target entity, the target entity category represents the category of the target entity, and the target entity category is at least one of the sample entity categories.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the text entity recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text entity recognition method according to any one of claims 1 to 7 is implemented.

Images