Training method, device, equipment and medium for entity recognition model

Abstract

The present application provides a training method, apparatus, device, and medium for an entity recognition model, relating to the field of artificial intelligence. The method comprises: obtaining sample text and entity annotations of the sample text, wherein the entity annotations are used to represent target entities of the sample text; calling a feature extraction layer in the entity recognition model to perform feature extraction processing on the sample text to obtain an identification sequence of the sample text, wherein the identifiers in the identification sequence correspond one-to-one to the text of the text content of the sample text; calling a classifier in the entity recognition model to perform entity recognition processing on the identification sequence to obtain a predicted target entity, wherein the classifier is used to identify the head and tail of each predicted target entity in the sample text, and the predicted target entity exists at least two, and the predicted target entity is obtained based on a single-round output of the entity recognition model; and training the entity recognition model based on the error between the entity annotations and the predicted target entity. This method can improve the accuracy of the entity recognition model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method, apparatus, device, and medium for training an entity recognition model. Background Art

[0002] With the development of science and technology, a large number of articles are generated every day. In order to meet the needs of different objects, it is necessary to classify and match the articles and provide the required articles to the objects in a timely manner.

[0003] The relevant technology needs to first use the entity recognition model to extract entity words from the text, and then extract features from the text through the entity recognition model, such as text features (such as text length, part-of-speech statistics), game entity features (such as part of speech, word length), and interaction features between text and entities (such as whether the entity appears in the news title, the frequency of entity appearance, etc.), and then calculate the score of each entity based on the above features, and take the entity with the highest score as the key entity.

[0004] When a text includes multiple key entities, related technologies need to call the entity recognition model multiple times, resulting in low efficiency in entity recognition. Summary of the Invention

[0005] The present invention provides a method, apparatus, device, and medium for training an entity recognition model. The entity recognition model generated by the method can output multiple target entities at a time. The technical solution is as follows:

[0006] According to one aspect of the present application, a method for training an entity recognition model is provided, the method comprising:

[0007] Obtaining a sample text and an entity annotation of the sample text, wherein the entity annotation is used to represent a target entity of the sample text;

[0008] Calling a feature extraction layer in the entity recognition model to perform feature extraction processing on the sample text to obtain an identification sequence of the sample text, wherein the identifications in the identification sequence correspond one-to-one to the characters of the text content of the sample text;

[0009] Calling a classifier in the entity recognition model to perform entity recognition processing on the identification sequence to obtain a predicted target entity, wherein the classifier is used to identify the head and tail of each predicted target entity in the sample text, and there are at least two predicted target entities, and the predicted target entities are obtained based on a single round of output of the entity recognition model;

[0010] The entity recognition model is trained based on the error between the entity annotation and the predicted target entity.

[0011] According to one aspect of the present application, a training device for an entity recognition model is provided, the device comprising:

[0012] An acquisition module, configured to acquire a sample text and an entity annotation of the sample text, wherein the entity annotation is used to represent a target entity of the sample text;

[0013] a feature extraction module, configured to call the feature extraction layer in the entity recognition model to perform feature extraction processing on the sample text to obtain an identifier sequence of the sample text, wherein the identifiers in the identifier sequence correspond one-to-one to the characters in the text content of the sample text;

[0014] An entity recognition module is configured to call a classifier in the entity recognition model to perform entity recognition processing on the identification sequence to obtain a predicted target entity, wherein the classifier is configured to identify the head and tail of each predicted target entity in the sample text, wherein there are at least two predicted target entities, and the predicted target entities are obtained based on a single-round output of the entity recognition model;

[0015] A training module is used to train the entity recognition model based on the error between the entity annotation and the predicted target entity.

[0016] According to another aspect of the present application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the entity recognition model training method described above.

[0017] According to another aspect of the present application, a computer storage medium is provided, in which at least one program code is stored. The program code is loaded and executed by a processor to implement the training method of the entity recognition model as described above.

[0018] According to another aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the entity recognition model training method described above.

[0019] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0020] The sample text is processed through the feature extraction layer and classifier in the entity recognition model to obtain the predicted target entity of the sample text. The classifier identifies the head and tail of each predicted target entity, thereby identifying all possible predicted target entities in the sample text. Therefore, the entity recognition model provided in the embodiment of the application can output multiple entities in the text at once. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A computer system provided by an embodiment of the present application is shown;

[0023] Figure 2 A schematic diagram of an entity recognition model provided in an embodiment of the present application is shown;

[0024] Figure 3 A schematic diagram of a flow chart of a method for training an entity recognition model provided in an embodiment of the present application is shown;

[0025] Figure 4 A schematic diagram of an entity recognition model provided in an embodiment of the present application is shown;

[0026] Figure 5 A schematic diagram of a flow chart of a method for training an entity recognition model provided in an embodiment of the present application is shown;

[0027] Figure 6 A flow chart of a method for training an entity recognition model provided in an embodiment of the present application is shown;

[0028] Figure 7 A flow chart showing a method for using an entity recognition model provided in an embodiment of the present application is shown;

[0029] Figure 8 A schematic diagram of a training device for an entity recognition model provided in an embodiment of the present application is shown;

[0030] Figure 9 A schematic diagram of a server provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0032] First, the nouns involved in the embodiments of this application are introduced:

[0033] Artificial Intelligence (AI): The theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also encompasses the study of the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0034] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0035] Machine Learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0036] Entity: refers to a word with a specific meaning. In this embodiment of the application, the game entity refers to the game name mentioned in the news text.

[0037] Machine Reading Comprehension (MRC) is a technology that uses algorithms to enable computers to understand the semantics of text and answer related questions. Because both text and questions are written in human language, MRC falls under the purview of Natural Language Processing (NLP).

[0038] Named Entity Recognition (NER): also known as "proper name recognition", refers to the identification of entities with specific meanings in text, including but not limited to at least one of the following: names of people, places, organizations, and proper nouns.

[0039] Recall Rate (also known as Recall Rate): It is the ratio of the number of relevant texts retrieved to the number of all relevant texts in the document library, which measures the recall rate of the retrieval system.

[0040] Accuracy: refers to the number of samples correctly classified by the classifier divided by the total number of samples. Generally speaking, the higher the accuracy, the better the classifier.

[0041] BERT (Bidirectional Encoder Representations from Transformers): is a pre-training technology for natural language processing.

[0042] Word embedding is a general term for language models and representation learning techniques in natural language processing (NLP). Conceptually, it involves embedding a high-dimensional space (the number of dimensions equal to the number of words) into a much lower-dimensional continuous vector space, where each word or phrase is mapped to a vector in the real domain.

[0043] F1 Score: A metric used in statistics to measure the accuracy of a binary classification model. It combines both the model's precision and recall. The F1 score can be considered the harmonic mean of the model's precision and recall, with a maximum value of 1 and a minimum value of 0.

[0044] Figure 1 FIG. 1 is a block diagram of a computer system according to an exemplary embodiment of the present application. The computer system 100 includes a terminal 120 and a server 140 .

[0045] An application is installed on the terminal 120. The application can be a small program within an app (application), a dedicated application, or a web client. The terminal 120 is at least one of a smartphone, a tablet computer, an e-book reader, an MP3 player, an MP4 player, a laptop computer, and a desktop computer.

[0046] The terminal 120 is connected to the server 140 via a wireless network or a wired network.

[0047] The server 140 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 140 is used to provide background services for applications through an entity recognition model, which is used to identify entities in text. Optionally, the server 140 undertakes the main computing work and the terminal 120 undertakes the secondary computing work; or, the server 140 undertakes the secondary computing work and the terminal 120 undertakes the main computing work; or, the server 140 and the terminal 120 both adopt a distributed computing architecture for collaborative computing.

[0048] In an optional embodiment, the aforementioned application is a news broadcast application, which is used to provide news text. Server 140 is provided with an entity recognition model, which is used to identify entities in the news text. Server 140 uses the entity recognition model to identify entities in the news text and classifies the news text based on the identified entities. For example, if server 140 identifies the entity in the news text as "XX game" through the entity recognition model, the news text is classified as a game-related news text.

[0049] Figure 2 A schematic diagram of an entity recognition model provided in an embodiment of the present application is shown. The entity recognition model 200 includes an input 21, an encoding layer 22, a decoding layer 23, a classifier 24 and a training layer 25.

[0050] Input 21 includes a text title, text content, and a preset question. The text title and text content are extracted from the same text. The preset question can be set by the technician according to actual needs. For example, the preset question is "Which games are mentioned in this article?" Optionally, input 21 is input word by word into the entity recognition model 200. For example, Figure 2 As shown, the preset question includes m words, and q1, q2, ..., qm are used to represent the m words of the preset question, and the m words are input into the entity recognition model 200 word by word. Exemplarily, the text title includes n words, and t1, t2, ..., tn are used to represent the n words of the text title, and the n words are input into the entity recognition model 200 word by word. Exemplarily, the text content includes o words, and c1, c2, ..., co are used to represent the o words of the preset question, and the o words are input into the entity recognition model 200 word by word.

[0051] The encoding layer 22 is used to encode the input 21. Optionally, the output of the encoding layer 22 includes at least one of a paragraph feature vector 201, a position feature vector 202, and a frequency feature vector 203. The paragraph feature vector 201 is used to distinguish different sentences or paragraphs in the text. Exemplarily, if the text is located in a preset question, the paragraph feature vector of the text is set to 0; if the text is located in the text title, the paragraph feature vector of the text is set to 1; if the text is located in the text content, the paragraph feature vector of the text is set to 2. The position feature vector 202 is used to represent the position information of the text in the text. Exemplarily, the position feature vector of the ath (a is a positive integer)th text in the text content is set to Ea. The frequency feature vector 203 is used to represent the frequency of occurrence or frequency of occurrence of the text in the text. Optionally, the output of the encoding layer 22 also includes a text feature vector, which is a vector representation of the text itself.

[0052] Decoding layer 23 is used to extract features related to the entity. In an optional embodiment, decoding layer 23 adopts the structure of the BERT model. Optionally, decoding layer 23 outputs a sequence of identifiers, where the identifiers in the identifier sequence correspond one-to-one with the characters in the text content. The identifiers in the identifier sequence are used to represent the features of the characters in the text content.

[0053] The classifier 24 is used to identify the head and tail of each target entity in the text. Optionally, the classifier 24 includes a start classifier and an end classifier. The start classifier is used to identify the start identifier in the identifier sequence, and the start identifier is the identifier corresponding to the head of each target entity. The end classifier is used to identify the end identifier in the identifier sequence, and the end identifier is the identifier corresponding to the tail of each target entity. For example, assuming that the text content is "XX Province employee Y speaks", the identifier sequence output by the feature extraction layer is "ABCDEFGH", the start classifier identifies that identifiers A and D are the start identifiers, and the end classifier identifies that C and F are the end identifiers, then according to the correspondence between the text content and the identifier sequence, it is determined that the target entities in the text content include "XX Province", "Employee Y" and "XX Province Employee Y".

[0054] The training layer 25 is used to calculate the error between the predicted target entity predicted by the entity recognition model 200 and the entity annotation during the training process of the entity recognition model 200. Optionally, the training layer 25 includes a target entity recognition task 204, which is used to measure whether the predicted target entity output by the entity recognition model 200 is accurate. Optionally, the training layer 25 also includes an entity recognition task 205, which is used to measure whether the predicted target entity output by the entity recognition model 200 constitutes an entity. Optionally, the training layer 25 also includes a target entity task 206, which is used to predict whether the input text has a target entity.

[0055] Figure 3 The flowchart shows a method for training an entity recognition model provided by an embodiment of the present application. This method can be executed by the Figure 1 terminal 120 or server 140 shown below. The method includes the following steps:

[0056] Step 302: Obtain a sample text and the entity annotation of the sample text. The entity annotation is used to represent the target entity of the sample text.

[0057] Optionally, the sample text is any form of text. Exemplarily, the sample text is at least one of news, novels, essays, argumentative essays, expository essays, product introductions, legal provisions, and reports. The embodiments of the present application do not specifically limit the form of the sample text.

[0058] In an optional implementation, the sample text refers to a text with a word count less than a preset word count threshold. Here, the word count refers to the total number of words in the text title and the text content. The preset word count threshold can be set by those skilled in the art. For example, the preset word count threshold is set to 1200.

[0059] Optionally, the entity annotation includes at least two target entities of the sample text.

[0060] Optionally, the correlation between the target entity and the text theme is greater than the correlation threshold. Alternatively, the target entity is an entity used to represent the text theme.

[0061] Step 304: Invoke the feature extraction layer in the entity recognition model to perform feature extraction processing on the sample text, and obtain an identification sequence of the sample text. The identifications in the identification sequence correspond one by one to the words in the text content of the sample text.

[0062] Exemplarily, if the input text is "Please attend the meeting", the output identification sequence is "ABCDE", and the words in the text correspond one by one to the identifications in the representation sequence according to the arrangement order. For example, "attend" corresponds to "B", and "meeting" corresponds to "D".

[0063] Optionally, the feature extraction layer is a fully connected feedforward neural network.

[0064] Step 306: Invoke the classifier in the entity recognition model to perform entity recognition processing on the identification sequence, and obtain predicted target entities. The classifier is used to identify the heads and tails of each predicted target entity in the sample text. There are at least two predicted target entities, and the predicted target entities are obtained based on the single-round output of the entity recognition model.

[0065] Optionally, the classifier includes a start classifier and an end classifier. The start classifier is used to identify the head of each predicted target entity in the sample text. The end classifier is used to identify the tail of each predicted target entity in the sample text.

[0066] Step 308: Train the entity recognition model based on the error between the entity annotation and the predicted target entity.

[0067] Optionally, the entity recognition model is trained based on at least two errors between the entity annotation and the predicted target entity, wherein the at least two errors include a primary error and an auxiliary error, the primary error being used to indicate the accuracy of the predicted target entity, and the auxiliary error being used to correct the primary error.

[0068] Exemplarily, the entity recognition model is trained through an error back propagation algorithm based on the error between the entity annotation and the predicted target entity.

[0069] In summary, the embodiment of the present application processes the sample text through the feature extraction layer and classifier in the entity recognition model to obtain the predicted target entity of the sample text. The classifier recognizes the head and tail of each predicted target entity, thereby identifying all possible predicted target entities in the sample text. Therefore, the entity recognition model provided by the embodiment of the present application can output multiple entities in the text at once.

[0070] For example, Figure 4 Schematic diagram of an entity recognition model provided by an embodiment of the present application is shown. The entity recognition model 400 includes an input 41, an encoding layer 42, a decoding layer 43, a start classifier 44 and an end classifier 45.

[0071] Input 41 includes a text title, text content, and preset questions. The text title and text content are extracted from the same text. The preset questions can be set by technical personnel according to actual needs. For example, the preset question is "Which games are mentioned in this article?" Exemplarily, input 41 also includes a CLS identifier and a SEP identifier. The CLS identifier is located at the first position of the input and can be used for subsequent classification tasks. The SEP identifier is used to divide the content of input 41. For example, the SEP identifier is placed between the text content and the preset question to divide the text content and the preset question.

[0072] The encoding layer 42 is used to encode the input 21. Optionally, the output of the encoding layer 42 includes at least one of a paragraph feature vector 201, a position feature vector 202, and a frequency feature vector 203. The paragraph feature vector 201 is used to distinguish different sentences or paragraphs in the text. For example, if the text is located in a preset question, the paragraph feature vector of the text is set to 0; if the text is located in the text title, the paragraph feature vector of the text is set to 1; if the text is located in the text content, the paragraph feature vector of the text is set to 2. The position feature vector 202 is used to represent the position information of the text in the text. The frequency feature vector 203 is used to represent the frequency of occurrence or frequency of occurrence of the text in the text. Optionally, the output of the encoding layer 42 also includes a text feature vector, which is a vector representation of the text itself.

[0073] The decoding layer 43 is used to extract features related to the entity. In an optional embodiment, the decoding layer 43 adopts the structure of the BERT model. Optionally, the decoding layer 43 outputs an identifier sequence, where the identifiers in the identifier sequence correspond one-to-one with the text content. The identifiers in the identifier sequence are used to represent the characteristics of the text content. In addition, as shown in the figure, the output of the decoding layer 43 can also include a question identifier sequence, where the identifiers in the question identifier sequence are used to represent the characteristics of the text of the preset question.

[0074] The start classifier 44 is used to identify the start identifier in the identifier sequence, where the start identifier is the identifier corresponding to the head of each target entity.

[0075] The end classifier 45 is used to identify the end identifier in the identifier sequence, where the end identifier is the identifier corresponding to the end of each target entity.

[0076] In the following embodiments, the present application embodiment can output multiple target entities at one time. In a possible implementation, the above steps 304 and 306 may include the following sub-steps: Figure 5 A method for using an entity recognition model provided in an embodiment of the present application is shown.

[0077] Step 501: Call the feature extraction layer in the entity recognition model to perform feature extraction processing on the sample text to obtain an identification sequence of the sample text.

[0078] In the embodiment of the present application, the entity recognition model adopts the structure of the MRC model.

[0079] Optionally, the feature extraction layer includes an encoding layer and a decoding layer. The encoding layer in the entity recognition model is called to encode the sample text to obtain a vector group of the sample text; the decoding layer in the entity recognition model is called to decode the vector group to obtain an identification sequence of the sample text.

[0080] Step 502: calling a starting classifier to perform a first classification process on the identifier sequence to obtain a starting identifier in the identifier sequence, where the starting identifier is an identifier corresponding to the head of each predicted target entity.

[0081] The starting classifier is used to identify the head of each predicted target entity in the sample text.

[0082] Optionally, the starting classifier is a binary classifier. Optionally, the starting classifier is called to perform a first classification process on the identifier sequence to obtain a starting score for each identifier in the identifier sequence. The starting identifier in the identifier sequence is determined based on the starting score. For example, an identifier with a starting score greater than a first preset value is determined as the starting identifier, and an identifier with a starting score less than the first preset value is determined as a non-starting identifier, where the non-starting identifier refers to an identifier in the identifier sequence other than the starting identifier.

[0083] Step 503: calling the end classifier to perform a second classification process on the identifier sequence to obtain the end identifier in the identifier sequence, where the end identifier is the identifier corresponding to the tail of each predicted target entity.

[0084] The ending classifier is used to identify the tail of each predicted target entity in the sample text.

[0085] Optionally, the end classifier is a binary classifier. Optionally, the end classifier is called to perform a second classification process on the identifier sequence to obtain an end score for each identifier in the identifier sequence. The end identifier in the identifier sequence is determined based on the end score. For example, an identifier with an end score greater than a second preset value is determined as an end identifier, and an identifier with an end score less than the second preset value is determined as a non-end identifier, where a non-end identifier refers to an identifier in the identifier sequence other than the end identifier.

[0086] It should be noted that the embodiment of the present application does not specifically limit the order of step 502 and step 503. Step 502 can be executed first and then step 503; step 503 can be executed first and then step 502; or step 502 and step 503 can be executed simultaneously.

[0087] Step 504: Train the entity recognition model based on the error between the entity annotation and the predicted target entity.

[0088] In an optional implementation, the j ending identifiers located after the i-th starting identifier are determined, where i and j are positive integers; and the predicted target entity is obtained based on the text between the i-th starting identifier and the j-th ending identifier. The values ​​of i and j can be set by technical personnel according to actual needs. For example, the identifier sequence is "ABCDEFG", and the text content corresponding to the identifier sequence is "The director of the neighborhood committee speaks", wherein A and D are identified as starting identifiers, and C and E are identified as ending identifiers, then C is the first ending identifier located after A, and E is the second ending identifier located after A. Based on the text between A and C, the predicted target entity is "neighborhood committee", and based on the text between A and E, the predicted target entity is "director of the neighborhood committee". E is the first ending identifier located after D, and based on the text between E and D, the predicted target entity is "director".

[0089] In another optional implementation, b ending identifiers located before the ath ending identifier are determined, where a and b are positive integers; and a predicted target entity is obtained based on the text between the ath ending identifier and the bth ending identifier.

[0090] Illustratively, Table 1 shows a comparison table of the entity recognition model obtained by the training method of the entity recognition model provided by this application and the related art.

[0091] Table 1 Comparison table of the MRC model used in this application and related technologies

[0092] method Accuracy Recall F1 score Frequency Model 56.07 74.49 64.0 Basic BERT model 73.27 78.6 75.8 L2R model 85.18 36.1 50.1 This application adopts the MRC model structure 74.19 79.15 76.6

[0093] To sum up, the embodiment of the present application uses the start classifier and the end classifier to identify all possible target entities in the text content at one time, while the entity recognition model in the related art can only output one target entity at a time. Therefore, the embodiment of the present application is not affected by the number of target entities and can output as many target entities as possible, thereby improving the efficiency of the entity recognition model.

[0094] In the following examples, the location of the entity in the news (i.e., whether it is in the title or text content) and the frequency of the entity's appearance are all highly important and will affect the determination of the target entity. Therefore, when obtaining the identification sequence, multiple features are extracted to improve the accuracy of target entity recognition. Figure 6 A method for using an entity recognition model provided in an embodiment of the present application is shown.

[0095] Step 601: Obtain preset questions.

[0096] Preset questions can be set by technical personnel according to actual needs. For example, the preset question is "Which games are mentioned in this article?" or "Is this article about game A?"

[0097] Step 602: Combine the preset question, text title and text content to obtain a text combination.

[0098] Exemplarily, the words in the preset question, text title and text content are spliced ​​in order to obtain a text combination.

[0099] Step 603: Call the encoding layer in the entity recognition model to encode the text combination to obtain a vector group of the sample text.

[0100] Optionally, the vector group includes a text feature vector and a frequency feature vector, the text feature vector is used to represent the text in the sample text, and the frequency feature vector is used to represent the frequency of occurrence of the text in the sample text; the encoding layer includes a frequency encoding layer and a text encoding layer. Then, the frequency encoding layer in the entity recognition model is called to perform frequency encoding processing on the text combination to obtain a frequency feature vector; and the text encoding layer in the entity recognition model is called to perform text encoding processing on the text combination to obtain a text feature vector. For example, if entity A appears 4 times in the sample text, the value of the frequency feature vector corresponding to entity A is set to 4.

[0101] Optionally, the vector group also includes a paragraph feature vector; the encoding layer also includes a paragraph encoder. Then, a first numerical value is assigned to the preset question in the text combination; a second numerical value is assigned to the text title in the text combination; and a third numerical value is assigned to the text content in the text combination; wherein the first numerical value, the second numerical value, and the third numerical value are different values; the paragraph encoder in the entity recognition model is called to encode the text combination to obtain a paragraph feature vector. Exemplarily, 0 is assigned to the preset question in the text combination; 1 is assigned to the text title in the text combination; and 2 is assigned to the text content in the text combination; the paragraph encoder in the entity recognition model is called to encode the text combination to obtain a paragraph feature vector.

[0102] Optionally, the vector group also includes a positional feature vector, which is used to indicate the position of a character in the sample text. Then, the character encoding layer in the entity recognition model is called to perform character encoding processing on the text combination to obtain a positional feature vector. For example, if character B is the 7th character in the sample text, the value of the positional feature vector corresponding to character B is set to 7.

[0103] Step 604: Call the decoding layer in the entity recognition model to decode the vector group to obtain the vector group of the sample text.

[0104] Optionally, the decoding layer is a fully connected feed-forward neural network.

[0105] For example, Table 2 shows a comparison table of the entity recognition model obtained by the entity recognition model training method provided by this application and related technologies. According to the content of Table 2, it can be seen that adding entity recognition tasks and tasks with target entities can increase the accuracy, recall rate and F1 score of the entity recognition model.

[0106] Table 2 Comparison of features added in this application and related technologies

[0107]

[0108]

[0109] In summary, the embodiments of the present application increase the number of text features, so that the vector group can describe the text from multiple angles, so that the entity recognition model can more accurately determine the target entity in the text, thereby improving the performance of the entity recognition model.

[0110] In the following embodiments, the entity recognition model may identify a target entity that is not an entity. For example, the target entity identified by the target recognition model is a "meridian", but "meridian" has no real meaning, so "meridian" does not constitute an entity, and the real target entity should be a "neuron". On the other hand, some texts do not have target entities. In this case, if the entity recognition model outputs the target entity, the target entity obtained in this way is wrong. Therefore, when training the entity recognition model, the embodiment of the present application will add at least two additional tasks, namely the entity recognition task and the target entity task.

[0111] In an alternative embodiment, step 308 may be implemented by training the entity recognition model based on at least two errors between the entity annotation and the predicted target entity. The at least two errors include a primary error and an auxiliary error, where the primary error indicates the accuracy of the predicted target entity and the auxiliary error corrects the primary error.

[0112] For the above entity recognition task, there are the following sub-steps:

[0113] 1. Obtain a first auxiliary error based on the predicted target entity and the second annotation.

[0114] The second annotation is used to represent the entity of the sample text.

[0115] The first auxiliary error is used to indicate the accuracy of predicting the entity boundary of the target entity. The first auxiliary error belongs to the auxiliary error mentioned above.

[0116] 2. Obtain the main error based on the predicted target entity and the first annotation.

[0117] The first annotation is used to represent the target entity of the sample text.

[0118] 3. Based on the first auxiliary error and the main error, the entity recognition model is trained.

[0119] Optionally, the entity recognition model is trained based on the first auxiliary error and the main error through an error back propagation algorithm.

[0120] For the above entity recognition task, there are the following sub-steps:

[0121] 1. Obtain a second auxiliary error based on the predicted target entity and the third annotation.

[0122] The third tag is used to indicate whether the target entity of the sample text exists. For example, if the third tag is 0, it means that the target entity of the sample text does not exist; if the third tag is 1, it means that the target entity of the sample text exists.

[0123] The second auxiliary error is used to represent the predicted target entity.

[0124] 2. Obtain the main error based on the predicted target entity and the first annotation.

[0125] The first annotation is used to represent the target entity of the sample text.

[0126] 3. Based on the second auxiliary error and the main error, the entity recognition model is trained.

[0127] Optionally, the entity recognition model is trained based on the second auxiliary error and the main error through an error back propagation algorithm.

[0128] In an optional implementation of the present application, the entity recognition model is trained based on the first auxiliary error, the second auxiliary error and the main error.

[0129] For example, Table 3 shows a comparison table of the entity recognition model obtained by the entity recognition model training method provided by this application and related technologies. According to the content of Table 3, it can be seen that adding entity recognition tasks and tasks with target entities can increase the accuracy, recall rate and F1 score of the entity recognition model.

[0130] Table 3 Comparison of features added in this application and related technologies

[0131]

[0132]

[0133] In summary, when training an entity recognition model, the embodiment of the present application provides auxiliary tasks to assist in the training of the entity recognition model, which can ensure accurate entity boundary recognition of the target entity and improve the recall rate of the model.

[0134] In the following embodiment, the trained entity recognition model can be used to identify game entities in news texts. The figure shows a flowchart of a method for using an entity recognition model provided in an embodiment of the present application.

[0135] Step 701: Obtain news text.

[0136] Optionally, the news text is downloaded from a server, or the news text is stored in a local storage of a computer device, or the news text is downloaded from a network.

[0137] Step 702: Call the entity recognition model to perform entity recognition processing on the news text to obtain at least two game entities in the news text.

[0138] The training method of the entity recognition model may refer to the above steps 302 to 308 and will not be repeated here.

[0139] It should be noted that in addition to identifying game entities, other entities can also be identified here, such as commodity entities, geographical entities, etc. The embodiment of this application only uses game entities for illustration. Those skilled in the art can adjust the entities output by the entity recognition model according to actual needs.

[0140] Step 703: Classify the news text based on at least two game entities.

[0141] For example, news text 1 includes game entity A and game entity C, and news text 2 includes game entity B and game entity C. Then news text 1 and news text 2 can be classified as type C text, which is text describing game entity C.

[0142] In summary, the embodiment of the present application can identify game entities in news texts and classify news texts according to game entities. After obtaining the classification of news texts, merchants can push news texts to objects according to the needs and preferences of the objects.

[0143] Figure 8 A training device for an entity recognition model provided by an exemplary embodiment of the present application is shown.

[0144] An acquisition module 801 is used to acquire a sample text and an entity annotation of the sample text, where the entity annotation is used to represent a target entity of the sample text;

[0145] A feature extraction module 802 is configured to call a feature extraction layer in an entity recognition model to perform feature extraction on the sample text to obtain a sequence of identifiers for the sample text, wherein the identifiers in the sequence of identifiers correspond one-to-one to the text content of the sample text;

[0146] Entity recognition module 803 is used to call the classifier in the entity recognition model to perform entity recognition processing on the identification sequence to obtain a predicted target entity. The classifier is used to identify the head and tail of each predicted target entity in the sample text. There are at least two predicted target entities. The predicted target entity is obtained based on the output of a single round of the entity recognition model.

[0147] The training module 804 is used to train the entity recognition model based on the error between the entity label and the predicted target entity.

[0148] In an optional embodiment, the classifier includes a start classifier and an end classifier; the entity recognition module 803 is also used to call the start classifier to perform a first classification process on the identifier sequence to obtain a start identifier in the identifier sequence, which is an identifier corresponding to the head of each predicted target entity; call the end classifier to perform a second classification process on the identifier sequence to obtain an end identifier in the identifier sequence, which is an identifier corresponding to the tail of each predicted target entity; and determine the predicted target entity based on the start identifier and the end identifier.

[0149] In an optional embodiment, the entity recognition module 803 is further used to determine the j ending identifiers located after the i-th starting identifier, where i and j are positive integers; and obtain the predicted target entity based on the text between the i-th starting identifier and the j-th ending identifier.

[0150] In an optional embodiment, the sample text includes a text title and text content; the feature extraction layer includes an encoding layer and a decoding layer; the feature extraction module 802 is also used to obtain a preset question; combine the preset question, text title and text content to obtain a text combination; call the encoding layer in the entity recognition model to encode the text combination to obtain a vector group of the sample text; call the decoding layer in the entity recognition model to decode the vector group to obtain an identification sequence of the sample text.

[0151] In an optional embodiment, the vector group includes a text feature vector and a frequency feature vector, the text feature vector is used to represent the text in the sample text, and the frequency feature vector is used to represent the frequency of occurrence of the text in the sample text; the encoding layer includes a frequency encoding layer and a text encoding layer; the feature extraction module 802 is also used to call the frequency encoding layer in the entity recognition model to perform frequency encoding processing on the text combination to obtain a frequency feature vector; call the text encoding layer in the entity recognition model to perform text encoding processing on the text combination to obtain a text feature vector.

[0152] In an optional embodiment, the vector group also includes a paragraph feature vector; the encoding layer also includes a paragraph encoder; the feature extraction module 802 is further used to assign a first numerical value to a preset question in the text combination; assign a second numerical value to a text title in the text combination; and assign a third numerical value to a text content in the text combination; wherein the first numerical value, the second numerical value, and the third numerical value are different values; and the paragraph encoder in the entity recognition model is called to encode the text combination to obtain a paragraph feature vector.

[0153] In an optional embodiment, the vector group also includes a position feature vector, which is used to represent the position of the text in the sample text; the feature extraction module 802 is also used to call the text encoding layer in the entity recognition model to perform text encoding processing on the text combination to obtain the position feature vector.

[0154] In an optional embodiment, the training module 804 is also used to train the entity recognition model based on at least two errors between the entity annotation and the predicted target entity, where the at least two errors include a main error and an auxiliary error, the main error is used to indicate the accuracy of the predicted target entity, and the auxiliary error is used to correct the main error.

[0155] In an optional embodiment, the auxiliary error includes a first auxiliary error, which is used to represent the accuracy of the entity boundary of the predicted target entity; the entity annotation includes a first annotation and a second annotation, the first annotation is used to represent the target entity of the sample text, and the second annotation is used to represent the entity of the sample text; the training module 804 is also used to obtain the first auxiliary error based on the predicted target entity and the second annotation; obtain the main error based on the predicted target entity and the first annotation; and train the entity recognition model based on the first auxiliary error and the main error.

[0156] In an optional embodiment, the auxiliary error includes a second auxiliary error, which is used to indicate the predicted target entity; the entity annotation also includes a third annotation, which is used to indicate whether the target entity of the sample text exists; the training module 804 is also used to obtain the second auxiliary error based on the predicted target entity and the third annotation; obtain the main error based on the predicted target entity and the first annotation; and train the entity recognition model based on the second auxiliary error and the main error.

[0157] In an optional embodiment, the feature extraction module 802 is further used to obtain news text; call the entity recognition model to perform entity recognition processing on the news text to obtain at least two game entities in the news text; and classify the news text based on the at least two game entities.

[0158] In summary, the embodiment of the present application processes the sample text through the feature extraction layer and classifier in the entity recognition model to obtain the predicted target entity of the sample text. The classifier recognizes the head and tail of each predicted target entity, thereby identifying all possible predicted target entities in the sample text. Therefore, the entity recognition model provided by the embodiment of the present application can output multiple entities in the text at once.

[0159] Figure 9 1 is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device 900 includes a central processing unit (CPU) 901, a system memory 904 including a random access memory (RAM) 902 and a read-only memory (ROM) 903, and a system bus 905 connecting the system memory 904 and the CPU 901. The computer device 900 also includes a basic input / output system (I / O system) 906 for facilitating information transmission between various components within the computer device, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.

[0160] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse and a keyboard for user input. The display 908 and the input device 909 are both connected to the central processing unit 901 via an input / output controller 910 connected to the system bus 905. The basic input / output system 906 may also include an input / output controller 910 for receiving and processing input from a variety of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 910 also provides output to a display screen, a printer, or other types of output devices.

[0161] The mass storage device 907 is connected to the central processing unit 901 via a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable medium provide non-volatile storage for the computer device 900. In other words, the mass storage device 907 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0162] Without loss of generality, the computer device readable medium may include computer device storage media and communication media. Computer device storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer device readable instructions, data structures, program modules or other data. Computer device storage media include RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), CD-ROM, Digital Video Disc (DVD) or other optical storage, tape cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer device storage media are not limited to the above-mentioned ones. The above-mentioned system memory 904 and mass storage device 907 can be collectively referred to as memory.

[0163] According to various embodiments of the present disclosure, the computer device 900 may also be connected to a remote computer device on a network such as the Internet for operation. That is, the computer device 900 may be connected to the network 911 via the network interface unit 912 connected to the system bus 905, or the network interface unit 912 may be used to connect to other types of networks or remote computer device systems (not shown).

[0164] The memory also includes one or more programs, which are stored in the memory. The central processing unit 901 implements all or part of the steps of the above-mentioned entity recognition model training method by executing the one or more programs.

[0165] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, code set or instruction set is stored. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the training method of the entity recognition model provided by the above-mentioned various method embodiments.

[0166] The present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the training method of the entity recognition model provided by the above method embodiment.

[0167] Optionally, the present application also provides a computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the entity recognition model training method described in the above aspects.

[0168] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0169] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0170] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A training method for an entity recognition model, characterized in that: The method comprises: Acquire a sample text and an entity annotation of the sample text, wherein the entity annotation is used to represent a target entity of the sample text; the sample text includes a text title and text content; Get preset questions; Combining the preset question, the text title, and the text content to obtain a text combination; Calling the encoding layer in the entity recognition model to encode the text combination to obtain the vector group of the sample text; wherein the vector group includes a paragraph feature vector; the encoding layer includes a paragraph encoder; assigning a first numerical value to the preset question in the text combination; assigning a second numerical value to the text title in the text combination; and assigning a third numerical value to the text content in the text combination; wherein the first numerical value, the second numerical value, and the third numerical value are different values; calling the paragraph encoder in the entity recognition model to encode the text combination to obtain the paragraph feature vector; Calling a decoding layer in the entity recognition model to decode the vector group to obtain an identification sequence of the sample text, wherein the identifications in the identification sequence correspond one-to-one to the characters of the text content of the sample text; Calling a classifier in the entity recognition model to perform entity recognition processing on the identification sequence to obtain a predicted target entity, wherein the classifier is used to identify the head and tail of each predicted target entity in the sample text, and there are at least two predicted target entities, and the predicted target entities are obtained based on a single round of output of the entity recognition model; The entity recognition model is trained based on the error between the entity annotation and the predicted target entity.

2. The method according to claim 1, characterized in that The classifier includes a starting classifier and an ending classifier; The calling of the classifier in the entity recognition model to perform entity recognition processing on the identification sequence to obtain a predicted target entity includes: Calling the starting classifier to perform a first classification process on the identifier sequence to obtain a starting identifier in the identifier sequence, where the starting identifier is an identifier corresponding to a head of each of the predicted target entities; Calling the end classifier to perform a second classification process on the identifier sequence to obtain an end identifier in the identifier sequence, where the end identifier is an identifier corresponding to the tail of each predicted target entity; The predicted target entity is determined based on the start identifier and the end identifier.

3. The method according to claim 2, characterized in that The determining the predicted target entity based on the start identifier and the end identifier includes: Determine the j ending markers located after the i-th starting marker, where i and j are positive integers; The predicted target entity is obtained according to the text between the i-th start identifier and the j-th end identifier.

4. The method according to claim 1, wherein The vector group further includes a character feature vector and a frequency feature vector, wherein the character feature vector is used to represent the characters in the sample text, and the frequency feature vector is used to represent the frequency of occurrence of the characters in the sample text; the coding layer further includes a frequency coding layer and a character coding layer; The calling of the encoding layer in the entity recognition model to encode the text combination to obtain the vector group of the sample text includes: Calling the frequency coding layer in the entity recognition model to perform frequency coding processing on the text combination to obtain the frequency feature vector; The text encoding layer in the entity recognition model is called to perform text encoding processing on the text combination to obtain the text feature vector.

5. The method according to claim 1, wherein The vector group further includes a position feature vector, and the position feature vector is used to represent the position of the characters in the sample text; the encoding layer further includes a character encoding layer; The calling of the encoding layer in the entity recognition model to encode the text combination to obtain the vector group of the sample text includes: The text encoding layer in the entity recognition model is called to perform text encoding processing on the text combination to obtain the position feature vector.

6. The method according to any one of claims 1 to 3, characterized in that The training of the entity recognition model based on the error between the entity annotation and the predicted target entity includes: The entity recognition model is trained based on at least two errors between the entity annotation and the predicted target entity, the at least two errors including a main error and an auxiliary error, the main error being used to represent the accuracy of the predicted target entity, and the auxiliary error being used to correct the main error.

7. The method according to claim 6, characterized in that The auxiliary error includes a first auxiliary error, and the first auxiliary error is used to represent the accuracy of the entity boundary of the predicted target entity; the entity annotation includes a first annotation and a second annotation, the first annotation is used to represent the target entity of the sample text, and the second annotation is used to represent the entity of the sample text; The training of the entity recognition model based on at least two errors between the entity annotation and the predicted target entity includes: Obtaining the first auxiliary error based on the predicted target entity and the second annotation; Obtaining the main error based on the predicted target entity and the first annotation; The entity recognition model is trained based on the first auxiliary error and the main error.

8. The method according to claim 6, characterized in that The auxiliary error includes a second auxiliary error, and the second auxiliary error is used to represent the predicted target entity; the entity annotation also includes a third annotation, and the third annotation is used to indicate whether the target entity of the sample text exists; The training of the entity recognition model based on at least two errors between the entity annotation and the predicted target entity includes: Obtaining a second auxiliary error based on the predicted target entity and the third annotation; Obtaining the main error based on the predicted target entity and the first annotation; The entity recognition model is trained based on the second auxiliary error and the main error.

9. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Get news text; Calling the entity recognition model to perform entity recognition processing on the news text to obtain at least two game entities in the news text; The news text is classified based on the at least two game entities.

10. A training device for an entity recognition model, characterized in that: The device comprises: An acquisition module, configured to acquire a sample text and an entity annotation of the sample text, wherein the entity annotation is used to represent a target entity of the sample text; the sample text includes a text title and text content; A feature extraction module is configured to obtain a preset question; combine the preset question, the text title, and the text content to obtain a text combination; call the encoding layer in the entity recognition model to encode the text combination to obtain a vector group of the sample text; wherein the vector group includes a paragraph feature vector; the encoding layer includes a paragraph encoder; assign a first numerical value to the preset question in the text combination; assign a second numerical value to the text title in the text combination; and assign a third numerical value to the text content in the text combination; wherein the first numerical value, the second numerical value, and the third numerical value are different values; call the paragraph encoder in the entity recognition model to encode the text combination to obtain the paragraph feature vector; call the decoding layer in the entity recognition model to decode the vector group to obtain an identification sequence of the sample text, wherein the identifications in the identification sequence correspond one-to-one to the characters in the text content of the sample text; An entity recognition module is configured to call a classifier in the entity recognition model to perform entity recognition processing on the identification sequence to obtain a predicted target entity, wherein the classifier is configured to identify the head and tail of each predicted target entity in the sample text, wherein there are at least two predicted target entities, and the predicted target entities are obtained based on a single-round output of the entity recognition model; A training module is used to train the entity recognition model based on the error between the entity annotation and the predicted target entity.

11. A computer device, characterized in that: The computer device includes: a processor and a memory, wherein at least one program is stored in the memory, and the at least one program is loaded and executed by the processor to implement the training method of the entity recognition model according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the training method of the entity recognition model according to any one of claims 1 to 9.

13. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the training method of the entity recognition model according to any one of claims 1 to 9 is implemented.

Images