Named entity recognition method, device, equipment and storage medium

Abstract

Embodiments of the present disclosure relate to a method, apparatus, device, and storage medium for identifying named entities. The method obtains a first text to be processed, searches for multiple second texts related to the first text based on the first text, determines at least one second text from the multiple second texts as the context of the first text, and identifies named entities in the first text based on the context. Because the second text is related text obtained by searching for the first text, named entity recognition is performed on the first text using the second text as the context of the first text, significantly enhancing the semantic information of the named entity recognition and thereby improving the accuracy of the named entity recognition.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of named entity recognition, and in particular to a method, apparatus, device, and storage medium for recognizing named entities. Background Art

[0002] Named entities are entities with specific meanings in text, such as organization names, personal names, and product names. Currently, named entity recognition (NER) can identify named entities in text, but this technology has low accuracy when the input text is short or lacks context. Summary of the Invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a method, apparatus, device and storage medium for identifying named entities.

[0004] A first aspect of an embodiment of the present disclosure provides a method for identifying a named entity, including:

[0005] Get the first text to be processed;

[0006] Based on the first text, searching for a plurality of second texts related to the first text content;

[0007] determining at least one second text from the plurality of second texts as a context of the first text;

[0008] Named entities are identified in the first text based on the context.

[0009] A second aspect of an embodiment of the present disclosure provides a device for identifying a named entity, including:

[0010] An acquisition module, used for acquiring a first text to be processed;

[0011] A search module, configured to search for a plurality of second texts related to the first text based on the first text;

[0012] a determination module, configured to determine at least one second text from the plurality of second texts as a context of the first text;

[0013] A recognition module is configured to recognize named entities in the first text based on the context.

[0014] A third aspect of an embodiment of the present disclosure provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method of the first aspect can be implemented.

[0015] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method of the first aspect described above can be implemented.

[0016] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art:

[0017] In an embodiment of the present disclosure, a first text to be processed is obtained, and based on the first text, multiple second texts related to the content of the first text are searched for. At least one second text is determined from the multiple second texts as the context of the first text, and named entities in the first text are identified based on the context. Because the second text is related text obtained by searching for the first text, named entity recognition is performed on the first text by using the second text as the context of the first text. This significantly enhances the semantic information of named entity recognition, thereby improving the accuracy of named entity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 is a schematic diagram of a named entity recognition scenario provided by an embodiment of the present disclosure;

[0021] Figure 2 is a flowchart of a method for identifying named entities provided by an embodiment of the present disclosure;

[0022] Figure 3 is a flowchart of another method for identifying named entities provided by an embodiment of the present disclosure;

[0023] Figure 4 is a schematic diagram of named entity recognition provided by an embodiment of the present disclosure;

[0024] Figure 5 is a schematic diagram of a named entity recognition model provided by an embodiment of the present disclosure;

[0025] Figure 6 Schematic diagram of a structure of a named entity recognition device provided by an embodiment of the present disclosure;

[0026] Figure 7 It is a structural diagram of a computer device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0029] In related technologies, named entity recognition technology can be used to identify named entities included in text. For example, in an information search task scenario, a user inputs a sentence, and the website needs to quickly retrieve the content topic that the user wants to search for. Named entity recognition is the most relevant module, which can extract the category relationship represented by multiple keywords in the sentence, and then obtain user needs. For sentence input, related technologies usually segment the sentence first, and then use a pre-trained model to extract word vectors after segmentation. After extraction, some subsequent processing can be performed to represent it as a word feature. Subsequently, the task model is used to train the given word features and specific tasks, and at the same time, the trained language model is fine-tuned. Both are trained simultaneously until convergence. However, due to the limited content of the input sentence, this method often has a relatively low accuracy problem. In another solution, the related technology can also use the same training method as the above solution when there is document-level context, and input the input sentence and document into the model at the same time to extract the word features of the sentence. In this case, the corpus is sufficient and the word feature effect is stronger, but this method must be performed in the document. In many real scenarios, document information does not exist. For example, there is no available context in the user's search query, tweets, short comments, etc. In this case, the accuracy of entity recognition is low.

[0030] In view of the shortcomings of related technologies in terms of the accuracy of named entity recognition, the embodiments of the present disclosure provide a new named entity recognition solution. For example, Figure 1 is a schematic diagram of a named entity recognition scenario provided by an embodiment of the present disclosure, such as Figure 1As shown, the computer device can obtain a first text to be processed, search for multiple second texts related to the content of the first text based on the first text, determine at least one second text from the multiple second texts as the context of the first text, and then identify named entities in the first text based on the context.

[0031] When there is no available context, domain knowledge can usually be relied upon to resolve ambiguity when annotating ambiguous named entities. This type of knowledge can usually be found through search engines. In addition, when an entity is uncertain, relevant knowledge can also be found through search engines. Therefore, the process of named entity recognition can benefit from the above search process. In this solution, since the second text is a related text obtained by searching for the first text, the semantic information of the named entity recognition is greatly enhanced by using the second text as the context of the first text to perform named entity recognition on the first text, thereby improving the accuracy of named entity recognition. Compared with the original input perspective based only on the first text, the word feature representation calculated by jointly modeling the input perspective of the first text and its retrieved external context can significantly improve the performance and accuracy of named entity recognition.

[0032] The named entity model of the embodiment of the present invention is a type of terminology recognition model, which can be applied to a variety of terminology recognition scenarios. For example, the terminology recognition model can be used to identify product names from user data in the e-commerce field (such as product review information, communication information with merchants, communication information with e-commerce customer service, etc.). The terminology recognition model can also be used to identify personal names from data in the fields of music, movie ticketing, etc. (such as lyrics, user reviews, movie introductions, etc.). The terminology recognition model can also be used to identify drug names or disease names from data in the medical field (such as diagnosis books, test reports, etc.).

[0033] The following examples illustrate the specific implementation methods of term recognition in the above three fields:

[0034] (1) E-commerce field:

[0035] A1. Obtain user data information.

[0036] The user data information includes product review information, communication information with merchants, communication information with e-commerce customer service, etc.

[0037] A2. Identify the product names in the user data information based on a term recognition model.

[0038] The term identification model is established by adopting the above-mentioned method for establishing the term identification model, and the terms involved in establishing the term identification model by adopting the above-mentioned method include product names.

[0039] A3. Mark the product name in the user data information.

[0040] A4. Analyze user data information labeled with product names to determine user preferences, and recommend products to the user based on the user preferences.

[0041] (2) Music, movie ticket sales and other fields

[0042] B1. Obtain data information including names generated in a preset platform.

[0043] The preset platform includes but is not limited to a music platform, a movie ticketing platform, etc. The data information is data information including a person's name, such as lyrics, music introduction, user comments, movie introduction, etc.

[0044] B2. Identify the names of people in the data information based on a term recognition model.

[0045] The term recognition model is established by adopting the above-mentioned method for establishing the term recognition model, and the terms involved in establishing the term recognition model by adopting the above-mentioned method include names of people.

[0046] B3. Mark the name of the person in the data information.

[0047] B4. Analyze the data information marked with the names according to the preset name analysis rules to obtain statistical information about the names.

[0048] The annotated names include, but are not limited to, names of characters in lyrics or films, and names of actors (including singers and actors). Specifically, user comments annotated with names can be analyzed to determine user preferences for the names, and recommendations can be made to users regarding other movies and songs performed by the actors. Users' interest in each name can also be calculated and ranked.

[0049] (3) Medical field

[0050] C1. Obtain medical data information.

[0051] Among them, medical data information includes diagnosis certificates, test reports, etc.

[0052] C2. Identify the medical names in the medical data information based on a term recognition model.

[0053] The term recognition model is established using the above-mentioned method for establishing the term recognition model, and the terms involved in establishing the term recognition model using the above-mentioned method include medical names, which include drug names and disease names.

[0054] C3. Highlight the medical name in the medical data information so that medical staff can quickly read valid information from the medical data information.

[0055] In order to better understand the inventive concept of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure are described below in conjunction with exemplary embodiments.

[0056] Figure 2 This is a flowchart of a method for identifying named entities provided by an embodiment of the present disclosure, which can be executed by a computer device. Figure 2 As shown, the method provided in this embodiment includes the following steps:

[0057] Step 101: Obtain a first text to be processed.

[0058] The first text may be any text that requires named entity recognition. In the embodiment of the present disclosure, the length of the first text is not limited. For example, the first text may be a sentence including multiple words or a short word.

[0059] In this embodiment, the computer device can obtain the first text to be processed. This embodiment does not limit the source of the first text. For example, the first text input by the user in real time can be obtained, and the first text sent by other computer devices can also be obtained.

[0060] Step 102: Based on the first text, search and obtain a plurality of second texts related to the first text content.

[0061] The second text may be a related text obtained by searching based on the first text. The number of the second text is not limited and may be multiple.

[0062] In this embodiment, after the computer device obtains the first text to be processed, it can input the first text or part of the content of the first text into the search engine as query content, obtain multiple texts related to the first text content returned by the search engine, and determine all or part of the multiple texts related to the first text content as second text.

[0063] In a feasible implementation, the computer device may use at least part of the content of the first text as a search keyword to search for text related to the first text content; and determine a preset number of texts from the search results as the second text.

[0064] At least part of the content in the first text may be at least one word among the multiple words obtained by word segmentation of the first text. The preset number may be set according to actual circumstances and is not limited thereto. The computer device may segment the first text to obtain multiple words, and determine at least one of the multiple words as a search keyword and input it into a search engine. The search engine may search based on the search keyword to obtain search results. The computer device may then obtain the search results returned by the search engine and determine a preset number of texts in the search results as the second text.

[0065] Optionally, when determining a preset number of texts from the search results as the second texts, the computer device may determine texts with a ranking higher than a first preset threshold from the search results as the second texts.

[0066] Since the multiple texts included in the search results can be sorted according to a preset sorting strategy, the preset sorting strategy can be a strategy pre-set in the search engine, and is not limited to specific ones. For example, the preset sorting strategy can be sorting according to text length. The first preset threshold can be a sorting threshold in the sorting results. For example, the first preset threshold can be 10, which means that the computer device can select the text ranked higher than 10 from the search results as the second text.

[0067] Step 103: Determine at least one second text from the plurality of second texts as the context of the first text.

[0068] The context may be information related to the text, and the context of the first text may be information related to the first text. For example, when the first text is a sentence, the context of the first text may be the content of the document in which the sentence is located.

[0069] In this embodiment, since the semantic relevance of the plurality of second texts may not meet the requirement, the computer device may select at least one second text from the plurality of second texts as the context of the first text based on the semantic relevance.

[0070] Step 104: Identify named entities in the first text based on the context.

[0071] Among them, named entities can be entities with specific meanings in the text, such as organization names, personal names, product names, place names, proper nouns, etc.

[0072] In this embodiment, after the computer device determines at least one second text from multiple second texts as the context of the first text, it can splice the first text and the context to obtain a spliced ​​text; and perform entity recognition on the spliced ​​text to obtain the named entities included in the first text.

[0073] Specifically, when the computer device splices the first text and the context, it can connect the context of the first text to the end of the first text to obtain a spliced ​​text. For example, when the first text is a sentence, the context of the first text can be connected to the end of the sentence to form a spliced ​​text; then a named entity recognition model can be used to identify named entities in the spliced ​​text, that is, based on the context of the first text, the conditional probability of multiple named entity sequences included in the first text is calculated, and the set of named entities with the highest probability is determined as the final output, thereby obtaining the named entities included in the first text.

[0074] The above-mentioned named entity recognition model can be set according to actual conditions. In the embodiment of the present disclosure, a pre-trained language model based on the Transformer model can be used as the named entity recognition model. The named entity recognition model can include a conditional random field (CRF) layer and a multimodal pre-trained embedding layer (Transformer-based embedding). The above is only an example and not a limitation. Other pre-trained language models may also be applicable, such as the XLM-R (XLM-RoBERTa) model, the BERT model, etc.

[0075] In this solution, named entities can be recognized based on the text to be processed and the semantically related text obtained by the search, which enhances the word feature expression ability of the text to be processed and thus improves the accuracy of named entity recognition.

[0076] In an embodiment of the present disclosure, a first text to be processed is obtained, and based on the first text, multiple second texts related to the content of the first text are searched for. At least one second text is determined from the multiple second texts as the context of the first text, and named entities in the first text are identified based on the context. Because the second text is related text obtained by searching for the first text, named entity recognition is performed on the first text by using the second text as the context of the first text. This significantly enhances the semantic information of named entity recognition, thereby improving the accuracy of named entity recognition.

[0077] For example, Figure 3 is a flowchart of another method for identifying named entities provided by an embodiment of the present disclosure, such as Figure 3 As shown, in a feasible implementation manner, the computer device may determine at least one second text as the context of the first text from a plurality of second texts by the following method:

[0078] Step 301: Determine the semantic relevance between each second text and the first text.

[0079] Semantic relevance can be understood as the semantic similarity or similarity between two texts.

[0080] Specifically, the computer device may use a semantic relevance algorithm to respectively calculate the semantic relevance between each second text and the first text.

[0081] The above-mentioned semantic relevance algorithm can be determined according to actual conditions. For example, the embodiment of the present disclosure can use the bidirectional encoder representation evaluation indicator (Bidirectional Encoder Representations from Transformers Score, BERTScore) to evaluate the semantic relevance. The BERT model can be a pre-trained language model, and BERTScore is a language generation indicator, that is, an evaluation indicator based on the context embedding of the pre-trained BERT model, which can be used to calculate the sum of cosine similarities between the word features (also known as word representations) of two texts to characterize the semantic relevance of the two texts. The word features can be generated from the pre-trained context embedding of the BERT model; when the BERTScore value is large, the second text and the first text are more likely to have a strong semantic relationship.

[0082] For example, taking the determination of the semantic relevance between a second text and the first text as an example, given the word features of the first text and the word features of the second text, the BERTScore can use precision (P) and recall (R) to represent the semantic relevance, specifically:

[0083]

[0084] Where R is the word in the input first text, which is the word with the closest semantics found from the second text with reference to the word in the second text, P is the word with the closest semantics found from the first text with reference to the word in the second text, ri is the word vector corresponding to the word in the input first text, rj is the word vector corresponding to the word in the second text, m is the number of words in the second text, and n is the number of words in the first text. After obtaining the above P and R, the evaluation score can be calculated. The evaluation score F1 = 2(PR) / (P+R), which is used to represent the semantic relevance between the first text and the second text.

[0085] Step 302: Sort the multiple second texts obtained by the search based on semantic relevance.

[0086] Specifically, after determining the semantic relevance between each second text and the first text, the computer device can sort the multiple second texts in descending order according to the semantic relevance to obtain a semantic sorting result, in which the second text with the greatest semantic relevance is sorted first.

[0087] Step 303: Determine the second text whose ranking is higher than a second preset threshold as the context of the first text.

[0088] Specifically, the computer device may use the second text ranked higher than a second preset threshold as the context of the first text. The second preset threshold may be a ranking threshold in the semantic ranking results and may be set based on actual circumstances. For example, the second preset threshold may be 5, meaning that the computer device may use the second text ranked higher than 5 in the semantic relevance ranking results as the context of the first text.

[0089] In the above scheme, the second text obtained by the search is reordered according to semantic similarity so that the most semantically relevant second text is selected as the context of the first text, thereby improving the semantic similarity between the context of the first text and the first text, which is beneficial to improving the accuracy of subsequent named entity recognition.

[0090] For example, Figure 4 is a schematic diagram of a named entity recognition provided by an embodiment of the present disclosure, such as Figure 4 As shown in the figure, a complete named entity recognition process provided by an embodiment of the present disclosure is implemented by the reordering module 401, the named entity recognition module 402 and the search engine 403 in the figure. The specific process may include: the first text (x) input (input) can be a sentence including n words, and the first text can be input into the search engine 403 as a query. The search engine can obtain k related texts (Related Texts) as second texts through the search, and then the multiple second texts can be input into the reordering module 401; the reordering module 401 can determine the semantic relevance score based on each related text (Related Texts) and the query text (Query Texts) through the reordering function (Ranking function), and then implement re-ranking, that is, reordering the multiple second texts based on the semantic relevance between each second text and the first text, and then determining L second texts from the second texts as external contexts (ExternalContexts), that is, the context of the first text. Then the first text (x) and the context of the first text can be The concatenated text Input into the named entity recognition module 402; the named entity recognition module 402 may include the multimodal pre-trained embedding layer 4021 and the CRF layer 4022 in the figure, and the CRF layer 4022 outputs a prediction of the label y at each position of the first text. The label y can be understood as the named entity corresponding to the word at a position.

[0091] For example, Figure 5 Schematic diagram of a named entity recognition model provided by an embodiment of the present disclosure, showing a structural diagram of a named entity recognition model 500. The structure of the named entity recognition model 500 is similar to Figure 4 The structure of the named entity recognition module 402 in is the same as that in Figure 5 The named entity recognition model 500 may include a multimodal pre-trained embedding layer 501 and a CRF layer 502, wherein the multimodal pre-trained embedding layer 501 corresponds to Figure 4 The multimodal pre-trained embedding layer 4021 in CRF layer 502 corresponds to Figure 4 The CRF layer 4022 in .

[0092] like Figure 5 As shown, in the embodiment of the present disclosure, the task of named entity recognition can be solved as a sequence labeling problem, and a neural network model with a CRF layer is applied to the sequence labeling problem. Sequence labeling can be understood as a task of labeling each position in a given sentence sequence. In the sequence labeling model, the input text can be input into the multimodal pre-trained embedding layer 501 (that is, the pre-trained context word vector model based on the transformer structure) to obtain word features: v i =embed i (x), the word feature is input into the CRF layer 502 to obtain the conditional probability of each label sequence corresponding to the input text:

[0093]

[0094]

[0095] Where Ψ is the potential function, Ψ represents v i The label is y and v i The label of the previous word is y′, θ represents the model parameter, and y is v i A label of v, y′ is i The label of the previous word, W, is used to calculate v i The label of y is the score of b, which is used to calculate the score of the previous label of y is y′, y i-1 This is y′, y′ in the previous formula i It is v i Other possible labels, y′ i-1 is the possible label of the previous word. Y represents the set of all possible label sequences of the input text. The above word features can be the features represented by each word in the text, which can be represented in vector form and are generally extracted from word vectors.

[0096] The loss function (Loss) of the above model during training can be the negative log-likelihood loss function, which can be defined as The above model can be trained using this loss function.

[0097] In this solution, the concatenated text obtained by concatenating the first text and the context of the first text can be input into the above model, that is, the context of the first text is connected to the end of the first text to obtain the concatenated text, and the concatenated text is input into the multimodal pre-trained embedding layer 501 to obtain the word feature representation as The conditional probability of each label sequence corresponding to the first text obtained by inputting the word feature into the CRF layer 502 will also change, and the negative log-likelihood loss function can be redefined as The above model can be trained by re-defining the loss function. The concatenated text is input into the above model. The conditional probability of each label sequence corresponding to the input first text is calculated based on the context of the first text. The label set with the highest probability is determined as the final output label sequence. The label can be the result of named entity annotation of each word included in the first text. The label sequence can be a collection of multiple labels, that is, the named entities included in the first text are obtained.

[0098] Related technologies only extract word features of the language model from the input text, but are unable to obtain some information related to the input text. In the case of document context, such information is not available in many application scenarios. This solution uses a search engine to search and select the input text, which can determine a group of semantically related texts as the context of the input text, thereby enhancing the information of the input text. The input text and its context are modeled simultaneously through a pre-trained language model, that is, the results obtained by the search engine are used as the context of the input text and then spliced ​​with the input text and input into the model together. This greatly enhances the semantic information represented by the word features of the input text, thereby significantly improving the effect of named entity recognition.

[0099] Figure 6 This is a schematic diagram of the structure of a named entity recognition device provided by an embodiment of the present disclosure. The processing device can be understood as the above-mentioned computer device or a part of the functional modules in the above-mentioned computer device. Figure 6 As shown, the named entity recognition device 60 includes:

[0100] An acquisition module 61 is configured to acquire a first text to be processed;

[0101] A search module 62 is configured to search for a plurality of second texts related to the first text based on the first text;

[0102] a determination module 63, configured to determine at least one second text from the plurality of second texts as a context of the first text;

[0103] The recognition module 64 is configured to recognize named entities in the first text based on the context.

[0104] In one embodiment, the search module 62 includes:

[0105] A search submodule, configured to use at least part of the content of the first text as a search keyword to search for text related to the first text content;

[0106] The determination submodule is configured to determine a preset number of texts from the search results as second texts.

[0107] In one embodiment, the determining submodule is specifically configured to:

[0108] A text with a ranking higher than a first preset threshold is determined from the search results as the second text.

[0109] In one embodiment, the determining module 63 is configured to:

[0110] determining a semantic relevance between each second text and the first text;

[0111] sorting the plurality of second texts obtained by searching based on the semantic relevance;

[0112] A second text whose ranking is higher than a second preset threshold is determined as the context of the first text.

[0113] In one embodiment, the identification module 64 is configured to:

[0114] performing splicing processing on the first text and the context to obtain a spliced ​​text;

[0115] Entity recognition is performed on the concatenated text to obtain named entities included in the first text.

[0116] The device provided in this embodiment can perform the above Figure 1-Figure 5 The method of any embodiment has similar execution methods and beneficial effects, which will not be described in detail here.

[0117] The present disclosure also provides a computer device comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the above-mentioned Figure 1-Figure 5 The method of any embodiment.

[0118] For example, Figure 7This is a schematic diagram of the structure of a computer device in the embodiment of the present disclosure. Figure 7 , which shows a schematic diagram of the structure of a computer device 700 suitable for implementing the embodiments of the present disclosure. The computer device 700 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0119] like Figure 7 As shown, the computer device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the computer device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0120] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the computer device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The computer device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0121] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0122] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0124] The computer-readable medium may be included in the computer device, or may exist independently without being incorporated into the computer device.

[0125] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the computer device, the computer device is enabled to: obtain a first text to be processed; based on the first text, search and obtain multiple second texts related to the content of the first text; determine at least one second text from the multiple second texts as the context of the first text; and identify named entities in the first text based on the context.

[0126] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0129] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0130] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] The present disclosure also provides a computer-readable storage medium in which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1-Figure 5The method of any embodiment has similar execution methods and beneficial effects, which will not be described in detail here.

[0132] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0133] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying named entities, characterized in that: include: Get the first text to be processed; Based on the first text, searching from a search engine to obtain a plurality of second texts related to the content of the first text; Determining at least one second text from a plurality of second texts as a context of the first text, wherein the context of the first text is used to represent the content of the document in which the first text is located; identifying named entities in the first text based on the first text and the context; The step of determining at least one second text from a plurality of second texts as the context of the first text includes: Determining a semantic relevance between each second text and the first text based on word features of the plurality of second texts and word features of the first text, wherein the semantic relevance is characterized by precision and recall; Based on the semantic relevance, at least one second text is determined from a plurality of second texts as a context of the first text.

2. The method according to claim 1, characterized in that The step of searching for a plurality of second texts related to the first text based on the first text includes: Using at least part of the first text as a search keyword, searching for text related to the first text; A preset number of texts are determined from the search results as second texts.

3. The method according to claim 2, characterized in that The step of determining a preset number of texts from the search results as the second texts includes: A text with a ranking higher than a first preset threshold is determined from the search results as the second text.

4. The method according to claim 1, wherein The determining, based on the semantic relevance, at least one second text from the plurality of second texts as the context of the first text includes: sorting the plurality of second texts obtained by searching based on the semantic relevance; A second text whose ranking is higher than a second preset threshold is determined as the context of the first text.

5. The method according to any one of claims 1 to 4, characterized in that The identifying a named entity in the first text based on the first text and the context includes: performing splicing processing on the first text and the context to obtain a spliced ​​text; Entity recognition is performed on the concatenated text to obtain named entities included in the first text.

6. A named entity recognition device, characterized in that: include: An acquisition module, used for acquiring a first text to be processed; A search module, configured to search for a plurality of second texts related to the first text from a search engine based on the first text; a determination module, configured to determine at least one second text from a plurality of second texts as a context of the first text, wherein the context of the first text is used to represent the content of the document in which the first text is located; a recognition module, configured to recognize named entities in the first text based on the first text and the context; The determination module is further used to determine the semantic relevance between each second text and the first text based on the word features of the multiple second texts and the word features of the first text, wherein the semantic relevance is characterized by precision and recall; based on the semantic relevance, determine at least one second text from the multiple second texts as the context of the first text.

7. The device according to claim 6, characterized in that The search module includes: A search submodule, configured to use at least part of the content of the first text as a search keyword to search for text related to the first text content; The determination submodule is configured to determine a preset number of texts from the search results as second texts.

8. The device according to claim 6, characterized in that The determining module is configured to: sorting the plurality of second texts obtained by searching based on the semantic relevance; A second text whose ranking is higher than a second preset threshold is determined as the context of the first text.

9. A computer device, characterized in that: include: A memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images