Entity recognition method and device, training method and device of entity recognition model

Abstract

The application discloses an entity recognition method and device, a training method and device of an entity recognition model. The method comprises the following steps: obtaining the positions of at least one entity start ID and the positions of at least one entity end ID in a text to be recognized through a parallel first branch network and a second branch network of an entity recognition model according to the text to be recognized and at least one preset entity prompt corresponding to the text to be recognized; determining the entity end ID adjacent to each entity start ID from the at least one entity end ID; determining the text between the position of the entity start ID and the position of the entity end ID adjacent to the entity start ID as an entity; and outputting the positions of the at least one entity start ID and the positions of the at least one entity end ID through the first branch network and the second branch network respectively, so that the positions of the entity start ID and the positions of the entity end ID do not compete with each other in the entity recognition process, and the accuracy of entity recognition is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to an entity recognition method and apparatus, and an entity recognition model training method and apparatus. Background Technology

[0002] In financial institutions such as securities firms and banks, it is often necessary to extract information from important macro-level financial news, meso-level industry news, and micro-level corporate news. Named Entity Recognition (NER) is a crucial task in information extraction, serving as a foundational tool in subsequent tasks such as event extraction, association analysis, and knowledge graphs. NER is a subtask of information extraction that aims to locate and classify named entities in text into predefined categories, such as people, organizations, locations, time expressions, quantities, currency values, and percentages. While neural network models are typically used to identify the start and end positions of each entity, these positions are related to the entity's location and are not easily distinguishable by neural network models. Directly classifying the start and end positions of each entity using neural network models can lead to competition between these positions, reducing the accuracy of entity recognition. Summary of the Invention

[0003] In view of this, the embodiments of this application are committed to providing an entity recognition method and apparatus, and an entity recognition model training method and apparatus, which can improve the accuracy of entity recognition.

[0004] According to a first aspect of the embodiments of this application, an entity recognition method is provided, comprising: obtaining the position of at least one entity start ID and the position of at least one entity end ID in the text to be recognized based on the text to be recognized and at least one preset entity prompt corresponding to the text to be recognized, through a first branch network and a second branch network in parallel of an entity recognition model, wherein the at least one preset entity prompt is used to describe a preset entity in the text to be recognized, the first branch network is used to output the position of at least one entity start ID, each entity start ID position being the start ID position of the preset entity, the second branch network is used to output the position of at least one entity end ID, each entity end ID position being the end ID position of the preset entity; determining the entity end ID adjacent to each entity start ID from the at least one entity end ID; and determining the text between the position of the entity start ID and the position of the entity end ID adjacent to the entity start ID as an entity.

[0005] By incorporating parallel first and second branch networks into the entity recognition model—the first branch network outputting the location of at least one entity start ID and the second branch network outputting the location of at least one entity end ID—the first classification task for the entity start ID location and the second classification task for the entity end ID location are separated. In other words, the first and second classification tasks are processed as two independent binary classification tasks to obtain at least one entity start ID location and at least one entity end ID location respectively. This avoids the competition between the start ID location and the end ID location of the preset entity when using a conventional classification network to directly perform three classifications on the start ID location, the end ID location, and the ID locations of other characters in the text to be recognized besides the preset entity. This improves the accuracy of entity recognition.

[0006] According to a second aspect of the embodiments of this application, a training method for an entity recognition model is provided, comprising: determining training data based on a text sample and at least one preset entity prompt corresponding to the text sample, wherein the preset entity prompt is used to describe a preset entity in the text sample, and the training data is labeled with multiple tags, the multiple tags including a first tag for the position of the start ID of the preset entity, a second tag for the position of other character IDs in the text sample other than the start ID of the preset entity, a third tag for the position of the end ID of the preset entity, and a fourth tag for the position of other character IDs in the text sample other than the end ID of the preset entity; inputting the training data into the backbone network of the entity recognition model for feature extraction to obtain an output vector; and inputting the output vector into a first branch network of the entity recognition model. The system performs classification to obtain a first classification result for each character ID in the text sample. Based on the first classification result, the first label, and the second label, it obtains the first loss function value of the first branch network. The first classification result is used to characterize whether the position of each character ID in the text sample is the starting ID position of the preset entity. The output vector is then input into the second branch network of the entity recognition model for classification to obtain a second classification result for each character ID in the text sample. Based on the second classification result, the third label, and the fourth label, it obtains the second loss function value of the second branch network. The second classification result is used to characterize whether the position of each character ID in the text sample is the ending ID position of the preset entity. The parameters in the entity recognition model are updated based on the first and second loss function values.

[0007] In the entity recognition model, a first branch network and a second branch network are set up in parallel. During the training process, the first branch network only learns whether the position of each character ID in the text sample is the starting ID position of the preset entity to complete the first classification task. The second branch network only learns whether the position of each character ID in the text sample is the ending ID position of the preset entity to complete the second classification task. By separating the first and second classification tasks, that is, dividing them into two independent binary classification tasks by the first and second branch networks, the competition between the starting ID position and the ending ID position of the preset entity can be avoided when using a conventional classification network to directly perform three classifications on the starting ID position, the ending ID position, and the positions of other character IDs in the text sample besides the preset entity. This reduces the convergence difficulty of the entity recognition model and improves the accuracy of entity recognition.

[0008] According to a third aspect of the embodiments of this application, an entity recognition device is provided, comprising: a first acquisition module configured to obtain, based on a text to be recognized and at least one preset entity prompt corresponding to the text to be recognized, at least one entity start ID position and at least one entity end ID position in the text to be recognized through a parallel first branch network and a second branch network of an entity recognition model, wherein the at least one preset entity prompt is used to describe a preset entity in the text to be recognized, the first branch network is used to output at least one entity start ID position, each entity start ID position being the start ID position of the preset entity, and the second branch network is used to output at least one entity end ID position, each entity end ID position being the end ID position of the preset entity; a first determination module configured to determine an entity end ID adjacent to each entity start ID from the at least one entity end ID; and a second determination module configured to determine that the text between the entity start ID position and the entity end ID position adjacent to the entity start ID is an entity.

[0009] According to a fourth aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory storing computer program instructions, which, when executed by the processor, cause the processor to perform the methods mentioned in the first or second aspect.

[0010] According to a fifth aspect of the embodiments of this application, a computer-readable storage medium is provided, the storage medium storing computer program instructions, which, when executed by a processor, cause the processor to perform the methods mentioned in the first or second aspect above.

[0011] The entity recognition method provided in the embodiments of this application sets up a first branch network and a second branch network in parallel in the entity recognition model. The first branch network and the second branch network respectively output the position of at least one entity start ID and the position of at least one entity end ID. This avoids the competition between the position of the start ID and the position of the end ID of the preset entity when using a conventional classification network to directly classify the position of the start ID of the preset entity, the position of the end ID of the preset entity, and the positions of the IDs of other characters in the text to be recognized, thereby improving the accuracy of entity recognition. Attached Figure Description

[0012] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0013] Figure 1a This is a schematic diagram of the system architecture for an application scenario of the entity recognition method provided in one embodiment of this application.

[0014] Figure 1b This is a block diagram of an entity recognition system provided in one embodiment of this application.

[0015] Figure 2 This is a flowchart illustrating an entity recognition method provided in one embodiment of this application.

[0016] Figure 3 This is a data processing flowchart of an entity recognition method provided in one embodiment of this application.

[0017] Figure 4 This is a schematic diagram showing the location of the entity start ID and the location of the entity end ID according to an embodiment of this application.

[0018] Figure 5 This is a flowchart illustrating an entity recognition method provided in one embodiment of this application.

[0019] Figure 6 This is a flowchart illustrating an entity recognition method provided in one embodiment of this application.

[0020] Figure 7 This is a flowchart illustrating a training method for an entity recognition model provided in one embodiment of this application.

[0021] Figure 8 This is a schematic diagram of a masking process provided in one embodiment of this application.

[0022] Figure 9 This is a data processing flowchart of a training method for an entity recognition model provided in one embodiment of this application.

[0023] Figure 10 This is a block diagram of an entity recognition device provided in one embodiment of this application.

[0024] Figure 11 This is a block diagram of a training apparatus for an entity recognition model provided in one embodiment of this application.

[0025] Figure 12 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] This application can be applied to scenarios such as financial institutions or medical institutions. For example, financial institutions such as securities firms and banks typically need to extract information from important macroeconomic and financial news, meso-level industry news, and micro-level corporate news. Macroeconomic news refers to news involving changes in macroeconomic indicators (e.g., CPI, GDP information). Meso-level news mainly targets news from specific industries (e.g., metallurgy, internet, finance). Micro-level news refers to news occurring at the company level. Generally speaking, micro-level news is the most numerous and diverse, making it the focus of information extraction tasks. For information extraction tasks, the Network Execution Environment (NER) task is crucial, serving as a foundational tool in subsequent event extraction, association analysis, and knowledge graph tasks.

[0028] NER, also known as entity recognition, entity chunking, and entity extraction, is a subtask of information extraction that aims to locate and classify entities in text into predefined categories, such as people, organizations, locations, time expressions, quantities, currency values, percentages, etc. NER is a hot research area in Natural Language Processing (NLP), with the goal of identifying entities in text and categorizing them into corresponding entity types.

[0029] With the development of deep learning, entity recognition tasks can now generally be implemented using neural network models. A typical neural network model is BiLSTM-CRF, where BiLSTM (Bi-directional Long Short-Term Memory) and CRF (Full Connected / Dense Conditional Random Field). The BiLSTM-CRF model applied to NER mainly consists of embedding layers (primarily word vectors, character vectors, and some additional features), bidirectional LSTM layers, and CRF layers. In terms of features, BiLSTM-CRF inherits the advantages of deep learning methods, requiring no feature engineering and achieving good recognition results using only word vectors and character vectors. Furthermore, with high-quality dictionary features, the recognition performance can be further improved.

[0030] BiLSTM-CRF is a classic model architecture and serves as a baseline model. However, with the development of pre-trained models, BiLSTM-CRF no longer holds an advantage in entity recognition accuracy. At this point, BERT (Bidirectional Encoder Representations from Transformer)-based model architectures have begun to play a greater role. BERT-based models can also be referred to as pre-trained models.

[0031] Pre-trained models have brought revolutionary progress to NLP, and in scenarios requiring high entity recognition accuracy, most applications employ entity recognition methods based on pre-trained models. These methods typically use a sequential approach combining BERT and BiLSTM-CRF. While this can improve entity recognition performance to some extent, the sequential connection method can only perform three classifications: the start ID of the pre-trained entity, the end ID of the pre-trained entity, and the IDs of other characters in the text. Since the start and end IDs of the pre-trained entity are related to the same entity and are difficult to distinguish, they compete with each other during the three-class classification process, thus reducing the accuracy of entity recognition.

[0032] To address the aforementioned issues, this application provides an entity recognition method. The method establishes a first branch network and a second branch network in parallel within the entity recognition model. These two branches output at least one entity start ID location and at least one entity end ID location, respectively. This avoids the competition between the start ID location and the end ID location of the preset entity when using a conventional classification network to directly classify the start ID location, the end ID location, and the ID locations of other characters in the text to be recognized (excluding the preset entity). This improves the accuracy of entity recognition.

[0033] Since the embodiments of this application involve applications of neural networks, for ease of understanding, the relevant terms and concepts such as neural networks that may be involved in the embodiments of this application will be briefly introduced below.

[0034] A neural network is a computational model composed of numerous interconnected nodes (or neurons). Each node corresponds to a policy function, and the connection between any two nodes represents a weighted value for the signal passing through that connection. A neural network typically consists of multiple layers, cascaded together. The output of the i-th layer is connected to the input of the (i+1)-th layer, the output of the (i+1)-th layer is connected to the input of the (i+2)-th layer, and so on. Training samples are input into the cascaded neural network layers, and each layer outputs a result, which becomes the input of the next layer. This process continues, with multiple layers calculating the output. The predicted result of each layer is compared to the actual target value. Based on the difference between the predicted and actual values, the weight matrix and policy function of each layer are adjusted. The neural network continuously undergoes this adjustment process using training samples, refining its weights and other parameters until the predicted result matches the actual target value. This process is called the training process of the neural network. After training, a neural network model is obtained.

[0035] During neural network training, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the neural network predicts the target value or a value very close to it. Therefore, a loss function or objective function can be used to compare the difference between the predicted and target values; these are important equations for measuring this difference. Taking the loss function as an example, a higher loss value indicates a greater difference, and training the neural network becomes a process of minimizing this loss.

[0036] The following is combined with Figure 1a This application provides a detailed description of the system architecture for application scenarios of the entity recognition method mentioned in the embodiments. For example... Figure 1a As shown, the application scenarios provided in this application embodiment involve server 140 and multiple terminal devices 110, 120, and 130.

[0037] Terminal devices 110, 120, and 130 can be mobile terminal devices such as mobile phones, game consoles, tablets, and in-vehicle computers; alternatively, they can be personal computers (PCs), such as laptops and desktop computers. Those skilled in the art will understand that the types of the aforementioned terminal devices 110, 120, and 130 can be the same or different, and their number can be more or less. For example, there can be one of each of the aforementioned terminals, or dozens or hundreds of terminals, or even more. This application does not limit the number or type of terminals in its embodiments.

[0038] Terminal devices 110, 120, and 130 are connected to server 140 via a communication network. Optionally, the communication network can be a wired network or a wireless network. Optionally, server 140 can be a single server, a combination of several servers, or a cloud computing service center.

[0039] In one embodiment, terminal devices 110, 120, and 130 can obtain news data from various news websites through a crawler system installed on them. Of course, terminal devices 110, 120, and 130 can also obtain news data in other ways, and this application embodiment does not specifically limit this.

[0040] In one embodiment, server 140 receives news data obtained by terminal devices 110, 120, and 130, and trains parallel first branch networks and second branch networks using the news data. The first branch network is used to output the location of at least one entity start ID in the news data, and the second branch network is used to output the location of at least one entity end ID in the news data, so as to obtain an entity recognition model.

[0041] In one embodiment, terminal devices 110, 120, and 130 acquire news data from information websites that users want to follow, i.e., the text to be identified in this application, and send the text to be identified and at least one corresponding preset entity prompt to server 140. After receiving the text to be identified and at least one corresponding preset entity prompt, server 140 processes the text to be identified and at least one corresponding preset entity prompt using the entity recognition model deployed on it. The processing procedure is as follows: First, server 140 outputs the position of the start ID of at least one entity in the text to be identified through the first branch network based on the text to be identified and at least one corresponding preset entity prompt. Then, server 140 outputs the position of the end ID of at least one entity in the text to be identified through the second branch network based on the text to be identified and at least one corresponding preset entity prompt. Server 140 determines the entity end ID adjacent to each entity start ID from the at least one entity end ID, and determines the text between the position of the entity start ID and the position of the adjacent entity end ID as the entity to be identified. Server 140 can also send the entity to terminal devices 110, 120, and 130 so that users can view the entity.

[0042] This application provides an entity recognition method. In the entity recognition model, a first branch network and a second branch network are set up in parallel. The first and second branch networks output the location of at least one entity start ID and at least one entity end ID, respectively. This avoids the competition between the start ID and end ID of a preset entity when using a conventional classification network to directly classify the start ID, end ID, and IDs of other characters in the text to be recognized (excluding the preset entity) in three ways, thereby improving the accuracy of entity recognition.

[0043] The following is combined with Figure 1b A block diagram of the entity recognition system mentioned in the embodiments of this application is provided in detail. For example... Figure 1bAs shown, the text acquisition module 150, such as a web crawler system, is used to acquire the text of the public account or website that the user wants to follow; the text is then input into the text preprocessing module 160, for example, to perform length truncation, keyword filtering, etc.; the preprocessed text is then input into the entity recognition module 170 to obtain the entity to be recognized; the entity is then input into the postprocessing module 180 to apply the entity to downstream tasks to complete business requirements, such as information extraction or opinion analysis of the entity.

[0044] The following is combined with Figures 2 to 6 This application provides a detailed description of the entity recognition method mentioned in its embodiments.

[0045] Figure 2 This is a flowchart illustrating an entity recognition method provided in one embodiment of this application. Figure 2 The method described by Figure 1a The server 140 or other type of electronic device with data processing capabilities mentioned herein shall perform the execution. For example... Figure 2 As shown, the method includes the following steps.

[0046] Step S210: Based on the text to be identified and at least one preset entity prompt corresponding to the text to be identified, the starting ID and ending ID of at least one entity in the text to be identified are obtained through the parallel first branch network and second branch network of the entity recognition model.

[0047] In one example, the text to be identified may be a paragraph or a sentence from financial news, or it may be a paragraph or a sentence from a medical journal. This application does not specifically limit the application scenario of the text to be identified, nor does it specifically limit the presentation form of the text to be identified.

[0048] In addition, the text to be identified can be the text formed by directly extracting the original paragraphs or sentences from different business scenarios, thereby reducing the time cost, labor cost and equipment cost of data processing; the text to be identified can also be the text formed by truncating the original paragraphs or sentences from different business scenarios, that is, retaining the text containing important information and discarding the text containing unimportant information.

[0049] In one example, a web crawler system can be used to obtain the original text to be recognized; the original text to be recognized is then truncated to a preset length to obtain the text to be recognized.

[0050] A web crawler system mainly consists of several key aspects: webpage crawling, webpage analysis and link discovery, task deduplication and scheduling, data preprocessing and storage, anti-crawler strategies, and progress display. Through a web crawler system, one can obtain the raw text to be identified from important informational websites, such as news data, for entity recognition.

[0051] When the original text to be recognized is news data, since the length of news data varies, and entity recognition models (e.g., the BERT model) have a maxlen (i.e., maximum text length) parameter, the news data is truncated to a preset length to meet the requirements of the entity recognition model. Taking maxlen=512 as an example, it is generally believed that the first 500 characters of a news article contain most of the information the article wants to convey. Therefore, in order to reduce the time required for inference by the entity recognition model and improve the efficiency of entity recognition, the original text to be recognized is directly truncated to a preset length of 512 characters to discard the portion exceeding the preset length of 512 characters, retaining the portion within the preset length of 512 characters as the text to be recognized. However, it should be noted that the embodiments of this application do not specifically limit the value of the preset length. For different types of original text to be recognized, those skilled in the art can adopt different preset lengths.

[0052] In addition, to make the data processing speed of the entity recognition model faster during the training process, the original text samples can be directly truncated to a preset length to discard the part that exceeds the preset length and keep the part within the preset length as the text sample, that is, the training data.

[0053] In one example, there are many types of entities, and different entity recognition tasks will correspond to different types. For example, if the entity type is a company, a business entity, or a social organization, then the corresponding entity recognition task will be named "Organization". All entity recognition tasks are constructed into a list, and a description of each entity recognition task is constructed. In this application, the description of the entity recognition task is called an entity hint, which is used to describe the entity.

[0054] Therefore, entities and entity hints can be constructed using a one-to-one correspondence list. The following example, a news scenario, illustrates this one-to-one correspondence list format.

[0055]

[0056] It should be understood that the construction of entity prompts is related to the business scenario. When the entity to be identified is different, the wording of the entity prompts will also be adjusted accordingly. In other words, the application in other business scenarios is similar to that in the news scenario, except that the name of the entity recognition task and the entity prompts in other business scenarios are different from those in this scenario. For example, the name of the entity recognition task in the medical scenario may be disease, medicine, doctor, or hospital, etc., and each of them has its own corresponding entity prompts.

[0057] In one example, there are multiple entity prompts in a preset business scenario, but the text to be recognized may not necessarily contain entities corresponding to each entity prompt. Therefore, entity prompts that are not relevant to the text to be recognized are excluded, and only the entity recognition model is used to reason about the entity prompts (i.e., preset entity prompts) related to the text to be recognized and the text to be recognized, thereby reducing the inference time of the entity recognition model and improving the efficiency of entity recognition.

[0058] The preset entity prompts are screened from multiple entity prompts in the following way: through a sliding window, keywords corresponding to multiple entity prompts in the preset business scenario are extracted from the text to be recognized; preset phrases adjacent to the keywords are determined in the text to be recognized; when the keywords and the preset phrases meet the preset regular rules, the entity prompt corresponding to the keyword is determined as the preset entity prompt.

[0059] Taking the above news scenario as an example of the preset business scenario, the screening process of the preset entity prompts is described. Suppose multiple entity prompts in the news scenario include "entity prompt for finding the time when something happened".

[0060] Using the sliding window method, all possible keywords corresponding to the "entity prompt for finding the time when something happened" are extracted from the text to be recognized. For example, keywords such as year, month, or day. Preset phrases adjacent to each keyword are determined in the text to be recognized. The preset phrase is a phrase adjacent to the left and / or right of the keyword. For example, if the keyword is year, month, or day, the preset phrase can be "Chinese and English numbers", and the preset phrase can also be "This year has this moment" adjacent to year. The combination of the keyword and the preset phrase adjacent to the keyword may or may not meet the preset regular rules. The preset regular rule is that there are Chinese and English numbers appearing near the keyword, that is, the combination of the keyword and the preset phrase adjacent to the keyword can form a sentence related to time. When the combination of the keyword and the preset phrase adjacent to the keyword meets the preset regular rules, it means that there is an entity corresponding to the "entity prompt for finding the time when something happened" in the text to be recognized, and the "entity prompt for finding the time when something happened" is determined as the preset entity prompt.

[0061] For example, the keywords are year, month, and day, and the preset phrases adjacent to them are 2022, 11, and 23. The sentence formed by their combination is "November 23, 2022", which means it meets the preset regular rules. Another example, the keyword is year, and the preset phrase adjacent to it is "This year has this moment". The sentence formed by their combination is "Every year has this moment", which means it does not meet the preset regular rules.

[0062] Similarly, by using preset regular expression rules, the system judges all entity prompts to determine which entity prompts are related to the text to be recognized and which are not.

[0063] This application does not specifically limit the length of the sliding window; those skilled in the art can make different choices based on actual needs. Similarly, this application does not specifically limit the form of the preset regular expression rule; the preset regular expression rule will differ for different entity prompts, and those skilled in the art can make different choices based on actual needs.

[0064] In addition to using regular expressions to filter preset entity prompts from multiple entity prompts, the following methods can also be used. Taking the news scenario above as an example, the filtering process for preset entity prompts will be explained. Suppose that multiple entity prompts in the news scenario include "find entity prompts for organizations such as companies, commercial institutions, or social organizations".

[0065] Specifically, a list of organizations to be followed is created. These organizations are preprocessed by removing common characters such as "joint-stock company" and "limited liability company" to obtain preprocessed organizations. A sliding window is used to extract all possible keywords from this list. For example, with a window length of 2, if the organization to be followed is "Mashang Consumer Co., Ltd.", and the preprocessed organization is "Mashang Consumer", then all possible keywords are: "Mashang", "Shangxiao", and "Consumer". All possible keywords are initially screened, removing keywords that lack obvious organizational characteristics and appear frequently. For example, the frequently appearing keyword is "Consumer". The initially screened keywords are then obtained, such as "Mashang" or "Shangxiao". The text to be identified is compared with the initially screened keywords. If the text to be identified contains the initially screened keywords, it means that the text contains an entity corresponding to the "entity prompt for finding companies, commercial organizations, or social organizations", and the "entity prompt for finding companies, commercial organizations, or social organizations" is determined to be the preset entity prompt.

[0066] The text to be identified can correspond to one or more preset entity prompts, and this application embodiment does not specifically limit this. For example, when the text to be identified contains both time and organizational-related entities, the number of preset entity prompts is two: one to find the time when the event occurred, and the other to find the organization, such as a company, business, or social organization.

[0067] In one example, a search engine is used to identify multiple entity suggestions for a preset business scenario. Specifically, for each preset business scenario, there will be multiple original entity suggestions. However, these original entity suggestions may not be standardized and may not be recognized by the entity recognition model. For example, an original entity suggestion might be "find the time when the incident occurred." Obviously, neither "the incident" nor "the time" can be recognized by the entity recognition model. Therefore, in order to standardize the original entity suggestions, a search engine can be used to first perform simple preprocessing on the original entity suggestions to obtain preprocessed entity suggestions, such as case conversion, length truncation, simplified / traditional character conversion, etc. This embodiment does not limit the specific content of the preprocessing. Then, the preprocessed entity suggestions are subjected to word segmentation to obtain segmented entity suggestions, such as Chinese word segmentation, domain-specific word segmentation, stop word filtering, etc. This embodiment does not limit the specific content of the word segmentation. Next, the segmented entity suggestions are rewritten and corrected to obtain rewritten and corrected entity suggestions, such as query correction, query alignment, query expansion, etc. This embodiment does not limit the specific content of the rewriting and correction. Finally, the rewritten and corrected entity suggestions are analyzed for the importance of terms and the precise intent of the user to obtain entity suggestions for the preset business scenario. For example, the importance of terms can be analyzed by weighting the words in the rewritten and corrected entity suggestions, and the precise intent of the user can be identified by machine learning and deep learning. This embodiment does not limit the specific content of the importance of terms and the precise intent of the user. Of course, failure analysis and manual intervention may be involved in the final stage to adapt to the needs of each business scenario.

[0068] When the original entity suggestion is "find the time when the incident occurred", after going through the search engine, the entity suggestion for the preset business scenario is "find the time when the incident occurred".

[0069] The reason for determining the preset entity prompts corresponding to the text to be recognized is as follows. While using BERT and BiLSTM-CRF sequentially for entity recognition can improve the entity recognition performance of the pre-trained model to some extent, the following two problems still exist:

[0070] 1. Entity recognition tasks only perform B, I, O classification tasks at the word level, without making reasonable use of the semantic information of the entire sentence or even paragraph.

[0071] 2. The semantic information defined by the entity recognition task is not used in the pre-trained model. For example, an entity recognition task is to find the company entity in an article. Without pre-set entity hints, the pre-trained model only performs the classification of sequence labels and does not know what specific task it is performing.

[0072] Therefore, this application embodiment introduces a preset entity hint, which is used to describe a preset entity in the text to be identified, so that the entity recognition model can effectively utilize the preset entity hint to more accurately determine the location of the entity start ID and the entity end ID in the text to be identified, thereby improving the accuracy of entity recognition.

[0073] In one example, the text to be recognized and at least one preset entity prompt corresponding to the text to be recognized are directly input into the parallel first branch network and second branch network of the entity recognition model to obtain the position of the start ID of at least one entity and the position of the end ID of at least one entity in the text to be recognized, respectively.

[0074] In another example, the text to be recognized and at least one preset entity prompt corresponding to the text to be recognized are concatenated to obtain at least one concatenation result; the at least one concatenation result is input into the parallel first branch network and second branch network of the entity recognition model to obtain the position of at least one entity start ID and the position of at least one entity end ID in the text to be recognized, respectively.

[0075] During concatenation, the predefined entity hints are concatenated with the text to be recognized using the sentence-initial vector [CLS] and sentence-final vector [SEP] to obtain the concatenation result. For example, if the predefined entity hint is to find the time of an event, and the text to be recognized is "XXX Company will be listed on September 28th", then the concatenation result is "[CLS]Find the time of an event [SEP] XXX Company will be listed on September 28th [SEP]". This result is then input into the entity recognition model, which can use the [CLS] start character and the [SEP] end character to identify which is the predefined entity hint and which is the text to be recognized.

[0076] Of course, the splicing result of "[CLS]Find the time of the event[SEP] XXX Company will be listed on September 28th[SEP]" is just an example. Other splicing results can also be adopted. This application does not specifically limit this. For example, a pre-defined entity hint can be appended to the text to be recognized to obtain a concatenated result, such as "[CLS] XXX Company will be listed on September 28th [SEP] Find the time of the event [SEP]". Alternatively, when the text to be recognized is a paragraph, the pre-defined entity hint can be inserted before or after any sentence in that paragraph to obtain a concatenated result. Specifically, the text to be recognized is "At the end of the first half of 2022, five biological reagent companies rushed to the secondary market in just 7 days, accepting the test of investors. Companies such as Feipeng Bio and Nearshore Protein, which were waiting outside the secondary market, had already made investors salivate." The concatenated result could be "[CLS] Find the time of the event [SEP] At the end of the first half of 2022, five biological reagent companies rushed to the secondary market in just 7 days, accepting the test of investors [SEP], [CLS] Find companies, commercial institutions, social organizations, and other organizations [SEP] Companies such as Feipeng Bio and Nearshore Protein, which were waiting outside the secondary market, had already made investors salivate [SEP]". In other words, the two sentences joined together begin with the [CLS] start character and end with the [SEP] end character.

[0077] Regardless of how the text to be recognized and its corresponding preset entity hints are input into the entity recognition model, it's equivalent to adding a description of the entity recognition task to the input data. The neural network model, aided by the preset entity hints, can more effectively find entities. In other words, the entity recognition model performs a task similar to reading comprehension, finding the entity's termination ID and the corresponding entity's termination ID to identify the entity. Reading comprehension (Machine Reading Comprehension) can be understood as follows: given context C and question Q, the machine reading comprehension task requires the neural network model to learn a function F such that A = F(C, Q) provides the correct answer A to question Q.

[0078] The entity recognition model can be a pre-trained model. This application does not specifically limit the type of entity recognition model; other types of models can also be used to recognize entities. Those skilled in the art can make different choices based on actual needs. For example, the pre-trained model can be BERT (Bidirectional Encoder Representations from Transformer), which utilizes the transformer's self-attention mechanism to consider the contextual meaning of entities in the text when recognizing them. Therefore, it has a good effect on feature extraction of entities in context. In other words, the self-attention module in the BERT model can effectively utilize the attention mechanism to integrate information from entity cues that is helpful for entity recognition into the BERT model, improving the prediction performance of entity recognition.

[0079] This entity recognition model has a branching structure, which includes parallel first and second branch networks, such as... Figure 3 As shown, this example demonstrates how an entity recognition model with a branching structure outputs the location of at least one entity start ID and at least one entity end ID in the text to be recognized. Specifically, the text to be recognized and at least one preset entity hint corresponding to the text are input into a first branch network to obtain the location of at least one entity start ID in the text to be recognized; the text to be recognized and at least one preset entity hint corresponding to the text are input into a second branch network to obtain the location of at least one entity end ID in the text to be recognized.

[0080] Step S220: Determine the entity termination ID adjacent to each entity start ID from at least one entity termination ID.

[0081] In one example, when at least one entity termination ID includes only one entity termination ID and at least one entity start ID also includes only one entity start ID, it is determined that the entity termination ID is adjacent to the entity start ID.

[0082] In another example, when at least one entity termination ID includes multiple entity termination IDs, the entity termination ID adjacent to each entity start ID is determined from the multiple entity termination IDs.

[0083] The entity termination ID, which is adjacent to the entity start ID, is located after the entity start ID, such as... Figure 4As shown, the ID adjacent to the entity start ID2 can be either entity end ID1 or entity end ID2. However, it can be seen from the text to be identified that entity start ID1 and entity end ID1 are in one group, and entity start ID2 and entity end ID2 are in another group. Therefore, the entity end ID that is adjacent to the entity start ID and located after the entity start ID is determined from multiple entity end IDs.

[0084] Step S230: Determine the text between the location of the entity start ID and the location of the entity end ID adjacent to the entity start ID as the entity.

[0085] An entity corresponds to an entity start ID and an entity end ID. In other words, the text between the entity start ID and the entity end ID refers to the entity.

[0086] For example, when the entity recognition task is named "Time," the entity can be a specific date such as September 28th; when the entity recognition task is named "Organization," the entity can be a specific organization such as "XXX Company." Figure 4 As shown, the text between the entity start ID1 and the adjacent and subsequent entity end ID1 is "XXX Company", so XXX Company is the entity representing the organization to be identified. The text between the entity start ID2 and the adjacent and subsequent entity end ID2 is "September 28th", so September 28th is the entity representing the time to be identified.

[0087] The entity recognition method provided in the embodiments of this application sets up a first branch network and a second branch network in parallel in the entity recognition model. The first branch network and the second branch network respectively output the position of at least one entity start ID and the position of at least one entity end ID. This avoids the competition between the position of the start ID and the position of the end ID of the preset entity when using a conventional classification network to directly classify the position of the start ID of the preset entity, the position of the end ID of the preset entity, and the positions of the IDs of other characters in the text to be recognized, thereby improving the accuracy of entity recognition.

[0088] In another embodiment of this application, such as Figure 5 As shown, Figure 2 The step S210 shown includes the following:

[0089] Step S510: The text to be recognized and at least one preset entity prompt are concatenated to obtain at least one concatenation result.

[0090] In one example, the text to be recognized is concatenated with each preset entity prompt to obtain at least one concatenation result.

[0091] However, to reduce the time spent on concatenation and the inference time of the entity recognition model, thereby improving the efficiency of entity recognition, the text to be recognized and at least one preset entity hint can be concatenated on demand. That is, only preset entity hints that meet the requirements are concatenated with the text to be recognized, reducing the time spent on concatenation. Furthermore, because concatenation is done on demand, the preset concatenation results input into the entity recognition model all meet the requirements; preset entity hints that do not meet the requirements are not input into the entity recognition model for inference. Therefore, the inference time of the entity recognition model is also reduced.

[0092] Specifically, in cases where at least one splicing result includes multiple splicing results, such as Figure 6 As shown, the on-demand splicing process includes the following:

[0093] Step S511: Concatenate the text to be recognized and the (i-1)th preset entity prompt corresponding to the text to be recognized to obtain the (i-1)th concatenation result.

[0094] According to such Figure 2 The method described in step S210 is used to concatenate the text to be recognized and the (i-1)th preset entity prompt corresponding to the text to be recognized.

[0095] Step S512: Determine whether the (i-1)th preset entity prompt and the i-th preset entity prompt corresponding to the text to be recognized are related.

[0096] For example, when the (i-1)th preset entity hint is to find organizations such as companies, business entities, and social organizations, the i-th preset entity hint is to find names such as executives, founders, and spokespeople. Obviously, an organization contains names such as executives, founders, and spokespeople. Therefore, the (i-1)th preset entity hint and the i-th preset entity hint are related.

[0097] For example, if the (i-1)th preset entity hint is to find the GDP growth amount, and the i-th preset entity hint is to find the names of executives, founders, spokespeople, etc., then the GDP growth amount does not contain the names of executives, founders, spokespeople, etc. Therefore, the (i-1)th preset entity hint and the i-th preset entity hint are not related.

[0098] Step S513: If the (i-1)th preset entity prompt and the ith preset entity prompt are related, the text to be recognized and the ith preset entity prompt corresponding to the text to be recognized are concatenated to obtain the ith concatenation result.

[0099] It is only meaningful to concatenate the text to be recognized and the corresponding i-th preset entity prompt when there is a relationship between the (i-1)th preset entity prompt and the i-th preset entity prompt. Otherwise, it will only increase the time spent on concatenation and entity recognition model inference.

[0100] For example, when the (i-1)th preset entity hint is to find GDP growth, and the ith preset entity hint is to find names such as executives, founders, and spokespeople, as mentioned above, the (i-1)th preset entity hint and the ith preset entity hint are not related. GDP is the baseline for entity recognition in this preset scenario. Under this baseline, even if there are entities related to names in the text to be recognized, they should not be recognized; otherwise, it will only increase the time spent on concatenation and entity recognition model inference. Therefore, to improve the efficiency of entity recognition, when the (i-1)th preset entity hint and the ith preset entity hint are not related, the text to be recognized and the ith preset entity hint will not be concatenated.

[0101] Step S514: Iterate through steps S512 and S513 up to the nth time to obtain multiple splicing results, where i is an integer greater than or equal to 1 and less than or equal to n, and n is an integer greater than or equal to 2.

[0102] Because there are relationships between multiple concatenated results, the entity recognition model's process of recognizing multiple pre-defined entities in the text to be recognized is a serial entity recognition process. A serial entity recognition process means that the recognition of one pre-defined entity is sequential to the recognition of the next. For example, in a news scenario, the pre-defined entity prompts for finding organizations such as companies, businesses, and social organizations are related to the pre-defined entity prompts for finding names such as executives, founders, and spokespeople. In this case, the entity recognition model will only recognize the pre-defined entities for executives, founders, and spokespeople after successfully identifying the location of the organization's pre-defined entity.

[0103] Step S520: Input each splicing result into the backbone network of the entity recognition model for feature extraction to obtain the output vector.

[0104] The entity recognition model includes a backbone network for feature extraction and parallel first and second branch networks following the backbone network. The embodiments of this application do not limit the specific type of the backbone network; it can be ResNet, ResNext, or DenseNet, or it can be an SWM classifier, a linear regression classifier, etc.

[0105] Step S530: Input the output vector into the first branch network for classification to obtain the location of at least one entity's starting ID.

[0106] The first branch network may include, but is not limited to, a fully connected layer. The first branch network may also include other neural network layers, such as convolutional layers or pooling layers, etc., which are not specifically limited in this application embodiment. Furthermore, this application embodiment does not limit the number of each type of neural network layer.

[0107] Step S540: Input the output vector into the second branch network for classification to obtain the location of at least one entity termination ID.

[0108] The second branch network may include, but is not limited to, a fully connected layer, and the first branch network may also include other neural network layers, such as convolutional layers or pooling layers, etc., which are not specifically limited in this application embodiment. In addition, the number of each type of neural network layer is not limited in this application embodiment.

[0109] By designing an entity recognition model with a first branch network and a second branch network, the multi-class task of identifying the entity's start ID position, the entity's end ID position, and other character ID positions can be divided into two binary classification tasks. Specifically, the output vector is input into the first branch network for binary classification to obtain at least one entity start ID position, and then the output vector is input into the second branch network for binary classification to obtain at least one entity end ID position.

[0110] In another embodiment of this application, in order to improve the efficiency of entity recognition, the method further includes: performing semantic recognition on the text to be recognized to obtain the semantic recognition result of the text to be recognized; and determining whether the semantic recognition result is related to at least one preset entity prompt.

[0111] The text to be recognized is input into the semantic recognition model to obtain the semantic recognition result of the text. For example, if the text to be recognized is a report related to the establishment of a company, then the semantic recognition result can be "company established".

[0112] When the preset entity prompt is to find specific locations such as cities or places of occurrence, the semantic recognition result "company establishment" is related to the location-related preset entity prompt, that is, in which location the company was established.

[0113] Execution is only performed if the semantic recognition result is related to at least one preset entity prompt. Figure 6 The step S511 shown is to reduce the time spent on stitching and at the same time reduce the inference time of the entity recognition model, thereby improving the efficiency of entity recognition.

[0114] The following is combined with Figures 7 to 9 This application provides a detailed description of the training method for the entity recognition model mentioned in the embodiments. Figure 7 The method described by Figure 1a The server 140 or other type of electronic device with data processing capabilities mentioned herein shall perform the execution. For example... Figure 7 As shown, the training method for the entity recognition model provided in this application includes the following steps.

[0115] It should be noted that some of the content mentioned in the following embodiments related to the training method of entity recognition model is the same as that mentioned in the embodiments related to entity recognition method. The differences between the two will be described in detail below, and the similarities will not be repeated. For details, please refer to the embodiments related to entity recognition method.

[0116] Step S710: Determine training data based on the text sample and at least one preset entity prompt corresponding to the text sample. The preset entity prompt is used to describe the preset entity in the text sample. The training data is labeled with multiple labels, including a first label at the position of the start ID of the preset entity, a second label at the position of the character IDs of the text sample other than the start ID of the preset entity, a third label at the position of the end ID of the preset entity, and a fourth label at the position of the character IDs of the text sample other than the end ID of the preset entity.

[0117] In one example, a text sample and at least one corresponding preset entity prompt are concatenated, and the concatenated text sample and at least one corresponding preset entity prompt are directly used as training data.

[0118] For example, when a text sample contains multiple preset entities, the number of preset entity prompts is also multiple. Then, the concatenated text sample and one corresponding preset entity prompt can be used as training data, the concatenated text sample and another corresponding preset entity prompt can also be used as training data, and the concatenated text sample and two other corresponding preset entity prompts can also be used as training data. This application embodiment does not specifically limit this.

[0119] Step S720: Input the training data into the backbone network of the entity recognition model for feature extraction to obtain the output vector.

[0120] Step S730: Input the output vector into the first branch network of the entity recognition model for classification to obtain the first classification result of each character ID in the text sample, and obtain the first loss function value of the first branch network based on the first classification result, the first label and the second label.

[0121] The first classification result is used to characterize whether the position of each character ID in the text sample is the position of the starting ID of the preset entity. That is, the position of the character ID in the text sample can be the position of the starting ID of the preset entity, or it can be the position of other character IDs in the text sample besides the starting ID of the preset entity.

[0122] The first classification result can be presented as a probability value of 0-1, that is, the probability value that the position of each character ID in the text sample is the position of the starting ID of the preset entity. The first classification result can also be presented as 0 or 1, that is, 1 indicates that the position of the character ID in the text sample is the position of the starting ID of the preset entity in the text sample, and 0 indicates that the position of the character ID in the text sample is the position of any other character ID in the text sample besides the starting ID of the preset entity. This application embodiment does not specifically limit the presentation format of the first classification result.

[0123] For the first branch network, multiple labels include a first label and a second label. Based on the differences between the first classification result and the first label and the second label, the first loss function value of the first branch network can be obtained through the first loss function.

[0124] Step S740: Input the output vector into the second branch network of the entity recognition model for classification to obtain the second classification result of each character ID in the text sample, and obtain the second loss function value of the second branch network based on the second classification result, the third label and the fourth label.

[0125] The second classification result is used to characterize whether the position of each character ID in the text sample is the position of the termination ID of the preset entity. That is, the position of the character ID in the text sample can be the position of the termination ID of the preset entity, or it can be the position of other character IDs in the text sample besides the termination ID of the preset entity.

[0126] The second classification result can be presented as a probability value of 0-1, that is, the probability value that the position of each character ID in the text sample is the position of the termination ID of the preset entity. The second classification result can also be presented as 0 or 1, where 1 indicates that the position of the character ID in the text sample is the position of the termination ID of the preset entity, and 0 indicates that the position of the character ID in the text sample is the position of any other character ID besides the termination ID of the preset entity. This application does not specifically limit the presentation format of the second classification result.

[0127] For the second branch network, multiple labels, including the third and fourth labels, are used. Based on the differences between the second classification result and the third and fourth labels, the second loss function value of the second branch network can be obtained through the second loss function.

[0128] Step S750: Update the parameters in the entity recognition model based on the first loss function value and the second loss function value.

[0129] In one example, the first loss function value can be backpropagated to update the parameters of the backbone network and the first branch network, such as weights and biases. This application does not limit this.

[0130] In one example, the second loss function value can be backpropagated to update the parameters of the backbone network and the second branch network, such as weights and biases. This application does not limit this.

[0131] In one example, the first loss function value and the second loss function value can be summed or weighted summed to obtain the total loss function value. Then, the total loss function value is backpropagated to update the parameters of the backbone network, the first branch network, and the second branch network, such as weights and biases. This application does not limit this.

[0132] It should be noted that the first loss function and the second loss function can be loss functions of the same type or different types, and the embodiments of this application do not specifically limit this.

[0133] In the entity recognition model, a first branch network and a second branch network are set up in parallel. During the training process, the first branch network only learns whether the position of each character ID in the text sample is the starting ID position of the preset entity to complete the first classification task. The second branch network only learns whether the position of each character ID in the text sample is the ending ID position of the preset entity to complete the second classification task. By separating the first and second classification tasks, that is, dividing them into two independent binary classification tasks by the first and second branch networks, the competition between the starting ID position and the ending ID position of the preset entity can be avoided when using a conventional classification network to directly perform three classifications on the starting ID position, the ending ID position, and the positions of other character IDs in the text sample besides the preset entity. This reduces the convergence difficulty of the entity recognition model and improves the accuracy of entity recognition.

[0134] When training an entity recognition model using training data, the entity start ID and entity end ID in text samples have a significant disadvantage in terms of data volume. Entity start ID and entity end ID can be considered positive samples, or hard-to-learn samples, as they are more difficult to learn. The number of positive samples in text samples is relatively small, while the number of other character IDs besides entity start ID and entity end ID is relatively large. These other character IDs can be considered negative samples, or easy-to-learn samples, as they are easier to learn. The task of recognizing the entity start ID position in the first branch network is a binary classification task, and the task of recognizing the entity end ID position in the second branch network is also a binary classification task. This leads to a typical imbalance problem in the number of positive and negative samples (the number of easy / hard-to-learn samples). Therefore, to avoid the data imbalance problem, the first loss function and / or the second loss function in this application can adopt the Focal Loss function.

[0135] For the Focal loss function, the weight of positive samples is increased to balance the situation where the number of positive samples is insufficient. Furthermore, since samples with high confidence also incur loss in the standard cross-entropy loss function, and in the application scenario of this application, negative samples with high confidence often constitute the majority of text samples, the Focal loss function is used to remove or reduce the loss of negative samples with high confidence, thereby improving the training efficiency of the entity recognition model. In other words, the entity recognition model is made to pay less attention to negative samples with high confidence (i.e., easy-to-learn samples), and to incur a higher loss for positive samples with low confidence (difficult-to-learn samples).

[0136] The Focal Loss function is an improvement upon the standard Binary Cross Entropy (BCE) function. The expression for the Focal Loss function is shown below:

[0137]

[0138] Where, p t These are the probability values ​​corresponding to the positions of the entity's start ID and the entity's end ID. This corresponds to the part of the standard cross-entropy loss. As a weighting factor, is the modulation factor.

[0139] Modulation factor Generally, a value between 0 and 5 is taken. When, the Focal Loss function equals the standard cross-entropy function, when At that time, due to Therefore, the loss of the Focal Loss function is less than or equal to the loss of the standard cross-entropy function. This assumes that the confidence level of easily learned samples (negative samples) is... The confidence score is 0.9 for difficult-to-learn samples (positive samples). It is 0.6, take It was found that the loss for difficult-to-learn samples with a confidence level of 0.6 was 16 times higher than that for easy-to-learn samples with a confidence level of 0.9. Therefore, by setting... This can greatly enhance the loss of difficult-to-learn samples, while greatly suppressing the loss of easy-to-learn samples, thereby controlling the imbalance in the number of easy / difficult-to-learn samples.

[0140] Weighting factors Generally, the weighting factor is taken as a value between 0 and 1. For positive samples, the weighting factor is... For negative samples, the weighting factor is Therefore, by setting It can suppress the imbalance in the number of positive / negative samples.

[0141] Therefore, by employing the Focal Loss function, the accuracy of entity recognition models can be improved.

[0142] In another embodiment of this application, since the preset entity does not appear in the preset entity prompts, but the training data input to the entity recognition model is a complete sequence of concatenated text samples and at least one corresponding preset entity prompt, the complete sequence includes each character in the text sample, each character in the preset entity prompt, and special characters such as [CLS] and [SEP]. Therefore, in order to improve the efficiency of the entity recognition model training, that is, to avoid spending time training each character in the preset entity prompts and special characters such as [CLS] and [SEP], the loss function value is not calculated for each character in the preset entity prompts and special characters such as [CLS] and [SEP] in the complete sequence when calculating the first loss function value and the second loss function value.

[0143] Specifically, the training data is masked to remove characters other than those in the text samples, while retaining the characters in the text samples. For example, such as... Figure 8 As shown, each character in the preset entity prompts in the training data, as well as special characters such as [CLS] and [SEP], are masked out using 0 in the Mask, while the characters in the text samples in the training data are preserved using 1 in the Mask.

[0144] When calculating the loss function value, the entity recognition model only considers, for example... Figure 8The loss function value is calculated for the part corresponding to "1" (i.e., the characters in the text sample), while the loss function value is not calculated for the part corresponding to "0".

[0145] Figure 9 This is a data processing flowchart of a training method for an entity recognition model provided in one embodiment of this application. For example... Figure 9 As shown, the training method includes the following:

[0146] Training data (e.g., [CLS] find the time of the event [SEP] XXX Company will be listed on September 28th [SEP]) or masked training data (e.g., 00000000001111111111111110) are input into the backbone network of the entity recognition model to obtain the output vector. The output vector is then input into the fully connected layer (i.e., the first branch network) for recognizing the starting ID of the entity and the fully connected layer (i.e., the second branch network) for recognizing the ending ID of the entity to obtain the first loss function value and the second loss function value. Finally, the first loss function value and the second loss function value are weighted and summed according to the preset weight coefficients to obtain the total loss function value.

[0147] The above text combined Figures 2 to 9 The method embodiments of this application are described in detail below, in conjunction with... Figure 10 and Figure 11 The present application provides a detailed description of the apparatus embodiments. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.

[0148] Figure 10 This is a schematic structural diagram of the entity recognition device 1000 provided in an embodiment of this application. Figure 10 As shown, Figure 10 The device 1000 may include: a first acquisition module 1010, a first determination module 1020, and a second determination module 1030. These modules will be described in detail below.

[0149] The first acquisition module 1010 is configured to obtain the position of at least one entity start ID and the position of at least one entity end ID in the text to be identified by means of a first branch network and a second branch network of an entity recognition model, based on the text to be identified and at least one preset entity prompt corresponding to the text to be identified. The at least one preset entity prompt is used to describe the preset entity in the text to be identified. The first branch network is used to output the position of at least one entity start ID, and the position of each entity start ID is the position of the start ID of the preset entity. The second branch network is used to output the position of at least one entity end ID, and the position of each entity end ID is the position of the end ID of the preset entity.

[0150] The first determining module 1020 is configured to determine the entity termination ID adjacent to the start ID of each entity from at least one entity termination ID.

[0151] The second determining module 1030 is configured to determine the text between the location of the entity start ID and the location of the entity end ID adjacent to the entity start ID as an entity.

[0152] By incorporating parallel first and second branch networks into the entity recognition model—the first branch network outputting the location of at least one entity start ID and the second branch network outputting the location of at least one entity end ID—the first classification task for the entity start ID location and the second classification task for the entity end ID location are separated. In other words, the first and second classification tasks are processed as two independent binary classification tasks to obtain at least one entity start ID location and at least one entity end ID location respectively. This avoids the competition between the start ID location and the end ID location of the preset entity when using a conventional classification network to directly perform three classifications on the start ID location, the end ID location, and the ID locations of other characters in the text to be recognized besides the preset entity. This improves the accuracy of entity recognition.

[0153] In one embodiment of this application, the first acquisition module 1010 is further configured to: concatenate the text to be recognized and at least one preset entity prompt to obtain at least one concatenation result; input each concatenation result into the backbone network of the entity recognition model for feature extraction to obtain an output vector; input the output vector into the first branch network for classification to obtain the location of at least one entity start ID; and input the output vector into the second branch network for classification to obtain the location of at least one entity end ID.

[0154] In one embodiment of this application, when at least one splicing result includes multiple splicing results, the first acquisition module 1010, when splicing the text to be identified and at least one preset entity prompt to obtain at least one splicing result, is further configured to: a) splice the text to be identified and the (i-1)th preset entity prompt corresponding to the text to be identified to obtain the (i-1)th splicing result; b) determine whether the (i-1)th preset entity prompt and the i-th preset entity prompt corresponding to the text to be identified have a correlation relationship; c) if the (i-1)th preset entity prompt and the i-th preset entity prompt have a correlation relationship, splice the text to be identified and the i-th preset entity prompt corresponding to the text to be identified to obtain the i-th splicing result; iteratively execute the above steps b) and c) up to the nth time to obtain multiple splicing results, where i is an integer greater than or equal to 1 and less than or equal to n, and n is an integer greater than or equal to 2.

[0155] In one embodiment of this application, the device 1000 further includes: a second acquisition module 1040 configured to perform semantic recognition on the text to be recognized and obtain the semantic recognition result of the text to be recognized; and a judgment module 1050 configured to judge whether the semantic recognition result is associated with at least one preset entity prompt.

[0156] In one embodiment of this application, when the first acquisition module 1010 concatenates the text to be recognized and at least one preset entity prompt to obtain at least one concatenation result, it is further configured to: concatenate the text to be recognized and at least one preset entity prompt to obtain at least one concatenation result when the semantic recognition result is related to at least one preset entity prompt.

[0157] In one embodiment of this application, the device 1000 further includes: an extraction module 1060, configured to extract multiple keywords corresponding to multiple entity prompts under a preset business scenario from the text to be identified through a sliding window, wherein the preset business scenario is the business scenario to which the text to be identified belongs, and the multiple entity prompts are used to describe entities under the preset business scenario; and a third determination module 1070, configured to determine a preset word group adjacent to each keyword in the text to be identified; and determine the entity prompt corresponding to the keyword as a preset entity prompt if the keyword and the preset word group satisfy a preset regular expression rule.

[0158] Figure 11 This is a schematic structural diagram of the training device 1100 for the entity recognition model provided in this application embodiment. Figure 11 As shown, Figure 11 The training device 1100 may include a fourth determining module 1110 and a training module 1120. These modules are described in detail below.

[0159] The fourth determining module 1110 is configured to determine training data based on a text sample and at least one preset entity prompt corresponding to the text sample. The preset entity prompt is used to describe the preset entity in the text sample. The training data is labeled with multiple labels, including a first label at the position of the start ID of the preset entity, a second label at the position of the character ID of the text sample other than the start ID of the preset entity, a third label at the position of the end ID of the preset entity, and a fourth label at the position of the character ID of the text sample other than the end ID of the preset entity.

[0160] The training module 1120 is configured to input training data into the backbone network of the entity recognition model for feature extraction to obtain an output vector; input the output vector into the first branch network of the entity recognition model for classification to obtain a first classification result for each character ID in the text sample, and obtain a first loss function value for the first branch network based on the first classification result, the first label, and the second label, wherein the first classification result is used to characterize whether the position of each character ID in the text sample is the starting ID position of the preset entity; input the output vector into the second branch network of the entity recognition model for classification to obtain a second classification result for each character ID in the text sample, and obtain a second loss function value for the second branch network based on the second classification result, the third label, and the fourth label, wherein the second classification result is used to characterize whether the position of each character ID in the text sample is the ending ID position of the preset entity; and update the parameters in the entity recognition model based on the first loss function value and the second loss function value.

[0161] In one embodiment of this application, the entity recognition model uses the Focal Loss function as the loss function.

[0162] Figure 12 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Figure 12 The illustrated electronic device 1200 may include a memory 1210 and a processor 1220. The memory 1210 may be used to store executable code. The processor 1220 may be used to execute the executable code stored in the memory 1210 to implement the steps in the various methods described above. In some embodiments, the electronic device 1200 may further include a network interface 1230 through which the processor 1200 exchanges data with external devices.

[0163] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any other combination. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0164] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An entity recognition method, characterized in that, include: Based on the text to be identified and at least one preset entity prompt corresponding to the text to be identified, the starting ID position and the ending ID position of at least one entity in the text to be identified are obtained through the parallel first branch network and second branch network of the entity recognition model. The at least one preset entity prompt is used to describe the preset entity in the text to be identified. The first branch network is used to output the starting ID position of the at least one entity, and the starting ID position of each entity is the starting ID position of the preset entity. The second branch network is used to output the ending ID position of the at least one entity, and the ending ID position of each entity is the ending ID position of the preset entity. Determine the entity termination ID adjacent to each entity start ID from the at least one entity termination ID; The text between the starting ID of the entity and the ending ID of the entity adjacent to the starting ID is defined as the entity.

2. The method according to claim 1, characterized in that, The step of obtaining the location of at least one entity start ID and at least one entity end ID in the text to be identified, based on the text to be identified and at least one preset entity prompt corresponding to the text to be identified, through the parallel first branch network and second branch network of the entity recognition model, includes: The text to be recognized and the at least one preset entity prompt are concatenated to obtain at least one concatenation result; Each splicing result is input into the backbone network of the entity recognition model for feature extraction to obtain an output vector; The output vector is input into the first branch network for classification to obtain the location of the starting ID of the at least one entity. The output vector is input into the second branch network for classification to obtain the location of the termination ID of the at least one entity.

3. The method according to claim 2, characterized in that, When the at least one splicing result includes multiple splicing results, the step of splicing the text to be recognized and the at least one preset entity prompt to obtain at least one splicing result includes: a) Concatenate the text to be recognized and the (i-1)th preset entity prompt corresponding to the text to be recognized to obtain the (i-1)th concatenation result; b) Determine whether the (i-1)th preset entity prompt and the i-th preset entity prompt corresponding to the text to be identified are related; c) When the (i-1)th preset entity prompt is associated with the i-th preset entity prompt, the text to be identified and the i-th preset entity prompt corresponding to the text to be identified are concatenated to obtain the i-th concatenation result; Iteratively execute steps b) and c) above up to the nth time to obtain the multiple splicing results, where i is an integer greater than or equal to 1 and less than or equal to n, and n is an integer greater than or equal to 2.

4. The method according to claim 2, characterized in that, Also includes: Semantic recognition is performed on the text to be recognized to obtain the semantic recognition result of the text to be recognized; Determine whether the semantic recognition result is related to the at least one preset entity prompt; The step of concatenating the text to be recognized and the at least one preset entity prompt to obtain at least one concatenation result includes: If the semantic recognition result is related to the at least one preset entity prompt, the text to be recognized and the at least one preset entity prompt are concatenated to obtain the at least one concatenation result.

5. The method according to any one of claims 1 to 4, characterized in that, Also includes: By using a sliding window, multiple keywords corresponding to multiple entity prompts under a preset business scenario are extracted from the text to be identified. The preset business scenario is the business scenario to which the text to be identified belongs, and the multiple entity prompts are used to describe the entities under the preset business scenario. In the text to be identified, determine the preset word groups adjacent to each keyword; If the keyword and the preset phrase satisfy the preset regular expression rule, the entity prompt corresponding to the keyword is determined to be the preset entity prompt.

6. A training method for an entity recognition model, characterized in that, include: Training data is determined based on text samples and at least one preset entity prompt corresponding to the text samples, wherein the preset entity prompt is used to describe preset entities in the text samples, and the training data is labeled with multiple tags, including a first tag at the position of the start ID of the preset entity, a second tag at the position of the character IDs of the text samples other than the start ID of the preset entity, a third tag at the position of the end ID of the preset entity, and a fourth tag at the position of the character IDs of the text samples other than the end ID of the preset entity; The training data is input into the backbone network of the entity recognition model for feature extraction to obtain the output vector. The output vector is input into the first branch network of the entity recognition model for classification to obtain the first classification result of each character ID in the text sample. Based on the first classification result, the first label and the second label, the first loss function value of the first branch network is obtained. The first classification result is used to characterize whether the position of each character ID in the text sample is the starting ID position of the preset entity. The output vector is input into the second branch network of the entity recognition model for classification to obtain the second classification result of each character ID in the text sample. Based on the second classification result, the third label and the fourth label, the second loss function value of the second branch network is obtained. The second classification result is used to characterize whether the position of each character ID in the text sample is the position of the termination ID of the preset entity. The parameters in the entity recognition model are updated based on the first loss function value and the second loss function value.

7. The training method according to claim 6, characterized in that, The entity recognition model uses the FocalLoss function as its loss function.

8. An entity recognition device, characterized in that, include: The first acquisition module is configured to obtain the start ID position and the end ID position of at least one entity in the text to be identified and at least one preset entity prompt corresponding to the text to be identified through a first branch network and a second branch network of an entity recognition model, based on the text to be identified and at least one preset entity prompt corresponding to the text to be identified. The at least one preset entity prompt is used to describe a preset entity in the text to be identified. The first branch network is used to output the start ID position of the at least one entity, and the start ID position of each entity is the start ID position of the preset entity. The second branch network is used to output the end ID position of the at least one entity, and the end ID position of each entity is the end ID position of the preset entity. The first determining module is configured to determine the entity termination ID adjacent to the start ID of each entity from the at least one entity termination ID; The second determining module is configured to determine the text between the location of the entity's start ID and the location of the entity's end ID adjacent to the entity's start ID as the entity.

9. An electronic device, comprising: processor; as well as A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 7.

Images