Method, device and storage medium for nested named entity recognition

Abstract

The application discloses a nested named entity recognition method and device and a storage medium. The method comprises the following steps: performing named entity recognition on a target sentence by using a preset named entity recognition model to determine a plurality of first entities corresponding to the target sentence; performing recombination processing on first entities that do not belong to first entities without nested words in the plurality of first entities according to a preset recombination rule to obtain a word set composed of a plurality of words with different lengths; performing named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities; and determining an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, and in particular, to a nested named entity recognition method and device and storage medium. BACKGROUND

[0002] Named entity recognition is an important tool for information extraction, question answering systems, syntactic analysis and machine translation, and plays an important role in the process of natural language processing technology becoming practical. Compared with the traditional flat NER task, entity nesting occurs in actual business, corresponding to the nested NER task, that is, the recognized entity may have a nested situation. The entity embedded in a longer entity is regarded as a nested entity. Most named entity recognition (NER) systems only process flat entities and ignore the entities nested inside, so they cannot capture the fine-grained semantic information in the text.

[0003] In addition, based on the method of machine reading comprehension, the reading comprehension idea is used to fully mine the document-level context features to support entity recognition. First, for each type of entity, the entity recognition task is converted into a question answering task, and a question, text and entity answer triple is constructed. For the nested NER task, more accurate question design is required to meet the requirements. The method based on the Chinese nested named entity recognition corpus has good recognition results in some vertical fields and is efficient. The disadvantages are also very obvious. The construction and maintenance of the nested named entity recognition corpus require experts in the field to spend a lot of manpower and material resources to complete. In addition, there are problems such as long system construction period, poor portability and the need to establish different domain knowledge bases as auxiliary to improve the system recognition ability.

[0004] For the technical problems of low accuracy, high cost and long period of Chinese nested named entity recognition in the prior art, no effective solution has been proposed so far. SUMMARY

[0005] Embodiments of the present application provide a nested named entity recognition method, device and storage medium to at least solve the technical problems of low accuracy, high cost and long period of Chinese nested named entity recognition in the prior art.

[0006] According to an aspect of an embodiment of the present application, a method for nested named entity recognition is provided, including: performing named entity recognition on a target sentence by using a preset named entity recognition model to determine a plurality of first entities corresponding to the target sentence; performing recombination processing on first entities in the plurality of first entities that do not belong to first entities without nested words according to a preset recombination rule to obtain a word set composed of a plurality of words of different lengths; performing named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities; and determining an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities.

[0007] According to another aspect of an embodiment of the present application, a storage medium is also provided, including a stored program, wherein the program is executed by a processor when the program is running to perform the method of any one of the above.

[0008] According to another aspect of an embodiment of the present application, an apparatus for nested named entity recognition is also provided, including: a first determining module configured to perform named entity recognition on a target sentence by using a preset named entity recognition model to determine a plurality of first entities corresponding to the target sentence; a recombination processing module configured to perform recombination processing on first entities in the plurality of first entities that do not belong to first entities without nested words according to a preset recombination rule to obtain a word set composed of a plurality of words of different lengths; a second determining module configured to perform named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities; and a third determining module configured to determine an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities.

[0009] According to another aspect of an embodiment of the present application, an apparatus for nested named entity recognition is also provided, including: a processor; and a memory connected with the processor, configured to provide the processor with instructions for processing the following processing steps: performing named entity recognition on a target sentence by using a preset named entity recognition model to determine a plurality of first entities corresponding to the target sentence; performing recombination processing on first entities in the plurality of first entities that do not belong to first entities without nested words according to a preset recombination rule to obtain a word set composed of a plurality of words of different lengths; performing named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities; and determining an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities.

[0010] In the embodiment of the present application, the recognition system first uses a preset named entity recognition model to perform named entity recognition on the target sentence to determine a plurality of first entities, then performs recombination processing on the first entities that do not belong to the first entities without nested words according to a preset recombination rule to obtain a word set, secondly uses the named entity recognition model to perform named entity recognition on each word in the word set to determine a plurality of second entities, and finally determines an entity recognition result according to the plurality of first entities and the plurality of second entities. Thus, the recognition system can better solve the entity nesting problem through a simple and efficient post-processing process without changing the named entity recognition model, and has the advantages of simple structure and high accuracy. Compared with the existing rule-based or entity library-based method, the technical solution can save a large amount of manpower and material resources, and has better migration performance. In addition, the technical solution does not need an additional network for entity boundary detection and supervision, has lower time complexity and higher reasoning efficiency. Moreover, the extracted nested information can also be used as auxiliary information or auxiliary features to support downstream tasks and enhance the effect of other tasks. In addition, the technical solution has stronger migration performance and can be fine-tuned for different fields to solve the nested entity recognition problem. Thus, the technical problems of low accuracy, high cost and long cycle of Chinese nested named entity recognition in the prior art are solved. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of this application and help to explain the illustrative embodiments of the present application and their descriptions, but do not constitute improper limitations on the present application. In the drawings:

[0012] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to the first embodiment of the present application;

[0013] Figure 2 is a flowchart of the method of nested named entity recognition according to the first aspect of the first embodiment of the present application;

[0014] Figure 3 is another flowchart of the method of nested named entity recognition according to the first aspect of the first embodiment of the present application;

[0015] Figure 4 is a schematic diagram of the device for nested named entity recognition according to the second embodiment of the present application;

[0016] Figure 5 is a schematic diagram of the device for nested named entity recognition according to the third embodiment of the present application; DETAILED DESCRIPTION

[0017] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] First, some nouns or terms that appear in the description of the embodiments of this disclosure shall be interpreted as follows:

[0020] Named Entity Recognition (NER) refers to the identification of entities with specific meanings in text, mainly including names of people, places, organizations, and proper nouns.

[0021] Example 1

[0022] According to this embodiment, a method embodiment for nested named entity recognition is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] The method embodiments provided in this example can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Figure 1 A hardware block diagram of a computing device for implementing a method for nested named entity recognition is shown. Figure 1As shown, the computing device can include one or more processors (which can include, but are not limited to, processing devices such as microprocessors, MCUs, or programmable logic devices, FPGAs, etc.), a memory for storing data, and a transmission device for communication functions. In addition, a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera can be included. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only schematic, and does not limit the structure of the above-mentioned electronic device. For example, the computing device can further include more or fewer components than those shown in Figure 1 or have a different configuration than that shown in Figure 1 .

[0024] It should be noted that the one or more processors and / or other data processing circuits described above can be referred to herein generally as "data processing circuits". The data processing circuits can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuits can be a single independent processing module, or be incorporated in whole or in part into any one of the other elements of the computing device. As referred to in embodiments of the present application, the data processing circuits serve as a processor to control, for example, the selection of the variable resistance terminal path connected to the interface.

[0025] The memory can be used to store software programs and modules of application software, such as program instructions / data storage means corresponding to the method of nested named entity recognition in embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, i.e. implements the method of nested named entity recognition of the application program described above. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely disposed with respect to the processor, which can be connected to the computing device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device is used to receive or send data via a network. Specific examples of the above-mentioned network can include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (NIC), which can be connected to other network devices through a base station so as to be able to communicate with the Internet. In one example, the transmission device can be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

[0027] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of the computing device.

[0028] It should be noted here that, in some optional embodiments, the above... Figure 1 The computing device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computing devices.

[0029] Under the above operating environment, according to the first aspect of this embodiment, a method for nested named entity recognition is provided. Figure 2 A flowchart illustrating the method is shown below. (Refer to...) Figure 2 As shown, the method includes:

[0030] S202: Using a preset named entity recognition model, perform named entity recognition on the target statement to determine multiple first entities corresponding to the target statement;

[0031] S204: According to the preset recombination rules, recombine the first entities that do not contain nested words among the multiple first entities to obtain a word set composed of multiple words of different lengths;

[0032] S206: Using a named entity recognition model, perform named entity recognition on each word in the word set to identify multiple second entities; and

[0033] S208: Determine the entity recognition result of the target statement based on multiple first entities and multiple second entities.

[0034] Specifically, refer to Figure 3As shown, the recognition system obtains a text of a business scenario or a public data set, and takes the text as a target sentence. The recognition system uses a preset named entity recognition model to perform named entity recognition on the target sentence according to an entity category that the user wants to recognize, and determines a plurality of first entities corresponding to the target sentence. The named entity recognition model is a model that the recognition system has pre-trained using the business scenario. For example, the recognition system obtains a text of a business scenario as: “Beijing News, Beijing University Third Hospital publicly recruits outstanding graduates at home and abroad, and the start conference is held in Beijing.” The recognition system takes the text of the business scenario as a target sentence, obtains an entity type that the user wants to recognize in the text of the business scenario, for example, a location and an organization, and then uses a preset named entity recognition model to perform named entity recognition on the target sentence according to the pre-set entity type (for example, a location and an organization), and the result of the named entity recognition is: “Beijing News, Beijing University Third Hospital publicly recruits outstanding graduates at home and abroad, and the start conference is held in Beijing.” It should be understood that “ / B-LOC” in the result indicates that the single word is the first word of an entity, and the entity type of the entity is a location identifier. “ / I-LOC” indicates that the single word is a word other than the first word of an entity, and the entity type of the entity is a location identifier. “ / B-ORG” indicates that the single word is the first word of an entity, and the entity type of the entity is an organization identifier. “ / I-ORG” indicates that the single word is a word other than the first word of an entity, and the entity type of the entity is an organization identifier. “ / O” indicates that the entity type of the single word is neither a location nor an organization. Then the recognition system determines a plurality of first entities corresponding to the target sentence according to the result as: “Beijing”, “Beijing University Third Hospital”, and “Beijing”. The recognition system identifies these entities according to the location (LOCATION) and the organization (ORGANIZATION), and the specific identification is: {LOCATION: [“Beijing”, “Beijing”], ORGANIZATION: [“Beijing University Third Hospital”]} (S202).

[0035] Further, the recognition system recombines the first entities not containing nested words according to a preset recombination rule to obtain a word set composed of words of different lengths. The recombination rule is to split and recombine the entities according to different lengths. For example, the entities not containing nested words in the entities “Beijing”, “Peking University Third Hospital” and “Jing” are “Beijing” and “Jing”, and the entity not containing nested words is “Peking University Third Hospital”. The recognition system recombines the nested word “Peking University Third Hospital” according to the preset recombination rule. For example, the total length of “Peking University Third Hospital” is 8, and “Peking University Third Hospital” is split according to lengths of 2, 3,..., 8, so that the words of length 2 in “Peking University Third Hospital” are “Beijing”, “Jingda”, “University”, “Xuedi”, “Third”, “San Yi” and “Hospital”, the words of length 3 are “Beijingda”, “Jingda Xue”, “Xuedi”, “Xuedi”, “Third Yi”, “San Hospital”, the words of length 4 are “Beijing University”, “Jing University”, “Xuedi”, “Xuedi Yi”, “Third Hospital”, and so on, until the length of 8, the words of length 8 are “Beijing University Third Hospital”. Then the recognition system collects the words of different lengths to obtain the word set (S204).

[0036] Further, the recognition system uses a named entity recognition model to perform named entity recognition on each word in the word set to determine a plurality of second entities. Taking the words of length 2 as an example, the recognition system performs entity recognition on the words in the word set, and the result of entity recognition is: “North / B-LOC Beijing / I-LOC”, “Beijing / O”, “Da / O”, “Xue / O”, “Di / O”, “San / O”, “Yi / O”, and “Yuan / O”, and the recognition system determines the second entity word to be “Beijing”. The recognition system performs entity recognition on entity words of other lengths in the same manner, so the plurality of second entities obtained finally are: “Beijing”, “Peking University” and “Peking University Third Hospital”. The recognition system identifies these entities according to locations (LOCATION) and organizations (ORGANIZATION), specifically: {LOCATION: [“Beijing”], ORGANIZATION: [“Peking University”, “Peking University Third Hospital”]} (S206).

[0037] Further, after the first entity and the second entity are determined by the recognition system, the entity recognition result of the target sentence is determined and output according to the plurality of first entities and the plurality of second entities. For example, the plurality of first entities are "Beijing", "Beijing University Third Hospital" and "Jing", and the plurality of second entities are "Beijing", "Beijing University" and "Beijing University Third Hospital". The "Beijing" in the first entity is "Beijing" in "Beijing News News...", and the "Beijing" in the second entity is "Beijing" in "Beijing University Third Hospital". Therefore, the "Beijing" in the first entity and the "Beijing" in the second entity are different, so the recognition system identifies these entities according to the location (LOCATION) and the organization (ORGANIZATION), and finally obtains the entity recognition result: {LOCATION: ["Beijing", "Jing", "Beijing"], ORGANIZATION: ["Beijing University Third Hospital", "Beijing University"]} (S208).

[0038] As described in the background, the method based on machine reading comprehension uses the reading comprehension idea to fully mine the document-level context features to support entity recognition. First, for each type of entity, the entity recognition task is converted into a question and answer task to construct a question, text and entity answer triple. Specifically, for the nested NER task, more accurate question design is required to meet the requirements. In addition, the method based on the Chinese nested named entity recognition corpus has good recognition results in some vertical fields and is efficient. The disadvantages are also very obvious. The construction and maintenance of the nested named entity recognition corpus require experts in the field to spend a lot of manpower and material resources to complete. In addition, the system construction period is long, the portability is poor, and a knowledge base in different fields needs to be established as an auxiliary to improve the system recognition ability.

[0039] To solve the above technical problems, the technical scheme of the embodiment of the present application is as follows: the recognition system first uses a preset named entity recognition model to perform named entity recognition on a target sentence to determine a plurality of first entities, then performs recombination processing on first entities that do not belong to first entities without nested words according to a preset recombination rule to obtain a word set, then uses the named entity recognition model to perform named entity recognition on each word in the word set to determine a plurality of second entities, and finally determines an entity recognition result according to the plurality of first entities and the plurality of second entities. Thus, the recognition system can better solve the entity nesting problem through a simple and efficient post-processing process without changing the named entity recognition model, and has the advantages of simple structure and high accuracy. Compared with existing methods based on rules or entity libraries, the technical scheme can save a large amount of manpower and material resources, and has better migration performance. In addition, the technical scheme does not need an additional network for entity boundary detection and supervision, has lower time complexity, and has higher reasoning efficiency. Moreover, the extracted nested information can also be used as auxiliary information or auxiliary features to support downstream tasks and enhance the effect of other tasks. In addition, the technical scheme has stronger migration performance and can be fine-tuned for different fields to solve the nested entity recognition problem. Thus, the technical problems of low accuracy, high cost, and long cycle of Chinese nested named entity recognition in the prior art are solved.

[0040] Optionally, the operation of recombining the first entities that do not belong to the first entities without nested words according to the preset recombination rule to obtain the word set composed of a plurality of words of different lengths includes: determining whether the first entities that do not belong to the first entities without nested words meet a preset first determination condition, wherein the first determination condition is whether the entity is a single-character entity or belongs to a preset category of entities; and performing fine-grained segmentation on the first entities that do not meet the first determination condition to obtain the word set composed of a plurality of words of different lengths.

[0041] Specifically, referring to Figure 3As shown, the recognition system determines whether the first entity not belonging to the first entity without nested words in the plurality of first entities meets a preset first determination condition. The first determination condition is whether the entity is a single-character entity or belongs to a preset category of entities, which can be adjusted according to different scenarios and can include a person name entity, a special jargon entity, a sensitive word entity, an undesirable word entity, etc. For example, the content of the first determination condition is: a. the entity is a single-character entity; b. the entity belongs to a person name entity, a special jargon entity, a sensitive word entity, etc. (i.e., a preset category of entities). The recognition system determines whether the first entity not belonging to the first entity without nested words, such as "Beijing University Third Hospital", meets a or b in the content of the first determination condition. When the first entity meets one of a and b, it is considered that the first entity meets the preset first determination condition. If the first entity does not meet a or b, it is considered that the first entity does not meet the preset first determination condition. For example, the recognition system needs to determine whether "Beijing University Third Hospital" meets the preset first determination condition. Specifically, the entity "Beijing University Third Hospital" is not a single-character entity and does not belong to a person name entity, a special jargon entity, a sensitive word entity, etc. Therefore, "Beijing University Third Hospital" does not meet a or b, and the recognition system considers that "Beijing University Third Hospital" (i.e., the first entity) does not meet the preset first determination condition. Then the recognition system performs fine-grained segmentation on the first entity not meeting the first determination condition, such as "Beijing University Third Hospital", to obtain a word set composed of multiple words of different lengths. Thus, the recognition system determines whether the first entity not belonging to the first entity without nested words in the plurality of first entities meets the preset first determination condition, and then performs fine-grained segmentation on the first entity not meeting the first determination condition to obtain a word set composed of multiple words of different lengths. Since the single-character entity and the preset category of entities are no longer needed to be recognized, the recognition system compares the first entity not belonging to the first entity without nested words in the plurality of first entities with the single-character entity and the preset category of entities, and filters out the single-character entity and the preset category of entities. These words will no longer participate in subsequent segmentation and recognition operations, reducing the number of recognitions. Then the recognition system performs fine-grained segmentation on the first entity not meeting the first determination condition to obtain a word set composed of multiple words of different lengths. For each word in the word set, the named entity recognition model can recognize the entity it has after inputting it into the named entity recognition model. Therefore, the named entity recognition model can be used to recognize the entity of each word in the word set, further recognize the nested words, and improve the efficiency of the entire system.

[0042] Optionally, using a named entity recognition model, perform named entity recognition on each word in the word set to determine multiple second entities, including: judging whether the words in the word set meet a preset second judgment condition, where the second judgment condition is whether the words in the word set are included in multiple first entities; and using the named entity recognition model to perform entity recognition on the words in the word set that do not meet the second judgment condition to determine multiple second entities.

[0043] Specifically, referring to Figure 3 As shown, the recognition system judges whether the words in the word set meet a preset second judgment condition, where the second judgment condition is whether the words in the word set are included in multiple first entities. For example, multiple first entities are: "Beijing", "Peking University Third Hospital", and "Jing". The recognition system judges whether the words "Beijing", "Jingda", "University",..., "Peking University", "Jing University No.", "University Third",... "Peking University Third Hospital" in the word set are included in multiple first entities respectively. Thus, the recognition system determines that the word "Peking University Third Hospital" in the word set is included in multiple first entities according to multiple first entities, so the word "Peking University Third Hospital" in the word set meets the preset second judgment condition. In addition, it should be particularly noted that since "Beijing" in multiple first entities is "Beijing" in "Beijing News...", and "Beijing" in the word set is "Beijing" in "Peking University Third Hospital", the "Beijing" in the first entity and the "Beijing" in the word set have different identifications. Therefore, the recognition system determines that "Beijing" is not included in multiple first entities according to multiple first entities, that is, the word "Beijing" in the word set does not meet the second judgment condition. Similarly, other words in the word set, such as "Beijing", "University", "No.", "Third", "Third Medical", and "Peking University" in the word set, also apply to this judgment method.

[0044] After the recognition system determines whether all the words in the word set meet the preset second judgment condition, it determines that the word that meets the second judgment condition is "Peking University Third Hospital", and the other words in the word set do not meet the second judgment condition. Then, the recognition system uses the named entity recognition model to perform entity recognition on the words in the word set that do not meet the second judgment condition, and determines that the multiple second entities obtained are "Beijing" and "Peking University". Thus, the recognition system determines whether the words in the word set meet the second judgment condition that the words in the preset word set are included in multiple first entities, and then uses the named entity recognition model to perform entity recognition on the words in the word set that do not meet the second judgment condition to determine multiple second entities. Since the multiple first entities are entities recognized by the named entity recognition model, and the recombined word set may store words that are the same as the multiple first entities, the recognition system needs to determine whether each word in the word set exists in the multiple first entities, so as to screen out the words that are the same as the multiple first entities. The words screened out after the above judgment already belong to entities, so they will no longer participate in subsequent recognition, reducing the number of times of subsequent recognition using the named entity recognition model. Then, the named entity recognition model is used to perform entity recognition on the words in the word set that do not meet the second judgment condition respectively, so as to extract the nested information in the entities. In addition, the extracted nested information can also be used as auxiliary information or auxiliary features to support downstream tasks and enhance the effects of other tasks.

[0045] Optionally, the operation of determining the entity recognition result of the target statement according to the multiple first entities and the multiple second entities includes: determining the entity recognition result of the target statement according to the first entities that belong to the first entities without nested words, meet the first judgment condition among the multiple first entities, and the multiple second entities.

[0046] Specifically, as shown in Figure 3 Among the multiple first entities, the first entities that belong to the first entities without nested words are, for example, "Beijing" and "Jing", the first entities that meet the first judgment condition are, for example, "Peking University Third Hospital", and the multiple second entities are, for example, "Beijing" and "Peking University". Therefore, the entity recognition result of the target statement finally determined by the recognition system is that the entities with specific meanings in the target statement are: {LOCATION: ["Beijing", "Jing", "Beijing"], ORGANIZATION: ["Peking University Third Hospital", "Peking University"]}.

[0047] In addition, referring to Figure 3As shown, the recognition system will put the first entity (i.e., "Beijing") that does not contain nested words into the preset entity word dictionary, put the first entity (i.e., "Jing") that meets the first preset judgment condition into the preset entity word dictionary, put the reorganized word (i.e., "Beijing University Third Hospital") that meets the second preset judgment condition into the preset entity word dictionary, and put the multiple second entities (i.e., "Beijing" and "Beijing University") obtained by performing named entity recognition on the words in the word set that do not meet the second preset judgment condition using the NER model (corresponding to the named entity recognition model) into the preset entity word dictionary. Then the recognition system outputs the words in the entity word dictionary as the entity recognition result. Thus, by storing the entities in each step that do not need to be identified subsequently into the entity word dictionary, the loss of entity words in the subsequent identification process can be avoided, and the integrity of the entity recognition result is ensured.

[0048] Optionally, before the operation of judging whether the first entity that does not contain nested words in the multiple first entities meets the first preset judgment condition, the method further includes: respectively judging whether the multiple first entities are the first entity that does not contain nested words using a preset entity word library, wherein the entity word library is composed of multiple entity words that do not contain nested words.

[0049] Specifically, before the operation of judging whether the first entity that does not contain nested words in the multiple first entities meets the first preset judgment condition, the recognition system will respectively judge whether the multiple first entities are the first entity that does not contain nested words using a preset entity word library composed of multiple entity words that do not contain nested words. For example, the entity word library includes the entity "Beijing" that contains nested words, and the multiple first entities include "Beijing", "Jing", and "Beijing University Third Hospital". The recognition system judges that the first entity "Beijing" is the first entity that does not contain nested words, while "Jing" and "Beijing University Third Hospital" are not the first entity that does not contain nested words. Since the entity word library is composed of multiple entity words that do not contain nested words, the recognition system can quickly screen the first entity that does not contain nested words by comparing the preset entity word library with the multiple first entities. The screened first entity does not need to participate in the subsequent identification operation, thereby reducing the number of subsequent judgments and identifications and improving the overall recognition efficiency.

[0050] Optionally, the method further includes: recording the first entity that does not contain nested words in the multiple first entities into an entity word set; and the operation of judging whether the first entity that does not contain nested words in the multiple first entities meets the first preset judgment condition includes: judging whether each entity word in the entity word set is a single-character entity; and judging whether each entity word in the entity word set belongs to a preset category of entities, wherein the preset category of entities includes a personal name entity, a special jargon entity, and a sensitive entity.

[0051] Specifically, the recognition system enters the first entities that do not belong to the first entities without nested words among the multiple first entities, such as "Jing" and "Peking University Third Hospital", into the entity word set. Then, the recognition system determines whether the entity words in the entity word set, such as "Jing" and "Peking University Third Hospital", are single-character entities, and determines whether the entity words in the entity word set, such as "Peking University Third Hospital", belong to the entities of the preset categories. The entities of the preset categories include person name entities, proprietary phrase entities, and sensitive entities. Thus, by entering the first entities that do not belong to the first entities without nested words among the multiple first entities into the entity word set, the recognition system determines whether each entity word in the entity word set is a single-character entity and an entity of the preset category, and screens out the single-character entities and the entities of the preset category among them. The screened-out entities no longer need to participate in the subsequent recognition operations, thereby reducing the subsequent judgment and recognition times and improving the overall recognition efficiency.

[0052] Optionally, before performing the named entity recognition operation on the target statement using the preset named entity recognition model, the method further includes: obtaining the statement to be recognized; and performing a preprocessing operation on the statement to be recognized to obtain the target statement, where the preprocessing operation includes at least any one of the following: data cleaning, text correction, and removal of short texts.

[0053] Specifically, before performing the named entity recognition operation on the target statement using the preset named entity recognition model, the recognition system will also obtain the statement to be recognized and perform preprocessing operations such as data cleaning, text correction, and removal of short texts on the statement to be recognized, and then obtain the target statement. Thus, the recognition system clears the redundant punctuation, spaces, special symbols, numbers, etc. in the statement to be recognized through data cleaning, improving the recognition efficiency. And the recognition system corrects the misspelled words or terms in the statement to be recognized through text correction, improving the recognition accuracy. In addition, the recognition system improves the recognition efficiency by removing short texts with unclear semantics. Thus, the recognition system further improves the recognition efficiency by preprocessing the target statement.

[0054] In addition, as shown in Figure 1 According to the second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, where, when the program runs, the above-mentioned method is executed by a processor.

[0055] According to the embodiment, the recognition system first performs named entity recognition on the target sentence by using a preset named entity recognition model to determine a plurality of first entities, then performs recombination processing on the first entities that do not belong to the first entities without nested words according to a preset recombination rule to obtain a word set, secondly performs named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities, and finally determines an entity recognition result according to the plurality of first entities and the plurality of second entities. Thus, the recognition system can better solve the entity nesting problem through a simple and efficient post-processing process without changing the named entity recognition model, and has the advantages of simple structure and high accuracy. Compared with the existing rule-based or entity library-based method, the technical solution can save a large amount of manpower and material resources, and has better migration performance. In addition, the technical solution does not need an additional network for entity boundary detection and supervision, has lower time complexity and higher reasoning efficiency. The extracted nested information can also be used as auxiliary information or auxiliary features to support downstream tasks and enhance the effect of other tasks. In addition, the technical solution has stronger migration performance and can be fine-tuned for different fields to solve the nested entity recognition problem. Thus, the technical problems of low accuracy, high cost and long cycle of Chinese nested named entity recognition in the prior art are solved.

[0056] In addition, the technical solution has the following technical effects:

[0057] The present application provides a new method and device for nested named entity recognition, which can better solve the problem of entity nesting through a rich post-processing process without changing the basic model. In addition, the boundary detection problem of nested entities is solved without changing the basic model. The method and device provided by the present application have the advantages of simple structure and high accuracy. The extracted nested information can also be used as auxiliary information or auxiliary features to support downstream tasks and enhance the effect of other tasks. The method and device provided by the present application do not need an additional network for entity boundary detection and supervision, so the time complexity is lower and the reasoning efficiency is higher. Compared with the traditional rule-based or entity library-based method, the technical solution can save a large amount of manpower and material resources, and has better migration performance. In addition, the technical solution has stronger migration performance and can be fine-tuned for different fields to solve the nested entity recognition problem.

[0058] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action order described, because according to the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0059] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software on a general hardware platform as necessary, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disc), and includes a plurality of instructions for causing an end device (which can be a mobile phone, a computer, a server, or a network device) to execute the method described in each embodiment of the present application.

[0060] Embodiment 2

[0061] Figure 4 An apparatus 400 for nested named entity recognition according to the present embodiment is shown, which corresponds to the method according to the first aspect of the embodiment 1. Referring to the apparatus 400 shown in the figure, the apparatus 400 comprises: a first determining module 410 configured to perform named entity recognition on a target sentence by using a preset named entity recognition model to determine a plurality of first entities corresponding to the target sentence; a recombination processing module 420 configured to perform recombination processing on first entities in the plurality of first entities that do not belong to first entities without nested words according to a preset recombination rule to obtain a word set composed of a plurality of words of different lengths; a second determining module 430 configured to perform named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities; and a third determining module 440 configured to determine an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities. Figure 4

[0062] Optionally, the recombination processing module 420 comprises: a first judgment sub-module configured to judge whether the first entities in the plurality of first entities that do not belong to the first entities without nested words meet a preset first judgment condition, wherein the first judgment condition is whether the entity is a single-character entity or belongs to a preset category of entities; and a segmentation sub-module configured to perform fine-grained segmentation on the first entities that do not meet the first judgment condition to obtain the word set composed of a plurality of words of different lengths.

[0063] Optionally, the second determining module 430 comprises: a second judgment sub-module configured to judge whether the words in the word set meet a preset second judgment condition, wherein the second judgment condition is whether the words in the word set are contained in the plurality of first entities; and a first determining sub-module configured to perform entity recognition on the words in the word set that do not meet the second judgment condition by using the named entity recognition model to determine the plurality of second entities.

[0064] ​Optionally, the third determining module 440 comprises a second determining sub-module configured to determine the entity recognition result of the target sentence according to the first entities that do not contain nested words, the first entities that meet the first judging condition, and the second entities in the plurality of first entities.

[0065] Optionally, before the operation of judging whether the first entities that do not contain nested words meet the preset first judging condition, the device 400 further comprises a judging module configured to respectively judge whether the plurality of first entities belong to the first entities that do not contain nested words by using a preset entity word library, wherein the entity word library is composed of a plurality of first entities that do not contain nested words.

[0066] Optionally, the device 400 further comprises an inputting module configured to input the first entities that do not contain nested words into an entity word set; and the first judging sub-module comprises a first judging unit configured to judge whether each entity word in the entity word set is a single-character entity, and a second judging unit configured to judge whether each entity word in the entity word set belongs to a preset category of entities, wherein the preset category of entities comprises a person name entity, a special dialogue entity, and a sensitive entity.

[0067] Optionally, before the operation of performing named entity recognition on the target sentence by using the preset named entity recognition model, the device 400 further comprises an obtaining module configured to obtain a to-be-recognized sentence, and a preprocessing module configured to perform a preprocessing operation on the to-be-recognized sentence to obtain the target sentence, wherein the preprocessing operation comprises at least any one of the following: data cleaning, text error correction, and short text removal.

[0068] According to the embodiment, the recognition system first performs named entity recognition on the target sentence by using a preset named entity recognition model to determine a plurality of first entities, then performs recombination processing on the first entities that do not belong to the first entities without nested words according to a preset recombination rule to obtain a word set, secondly performs named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities, and finally determines the entity recognition result according to the plurality of first entities and the plurality of second entities. Thus, the recognition system can better solve the entity nesting problem through a simple and efficient post-processing process without changing the named entity recognition model, and has the advantages of simple structure and high accuracy. Compared with the existing rule-based or entity library-based method, the technical solution can save a lot of manpower and material resources, and has better migration performance. In addition, the technical solution does not need an additional network for entity boundary detection and supervision, has lower time complexity and higher reasoning efficiency. The extracted nested information can also be used as auxiliary information or auxiliary features to support downstream tasks and enhance the effect of other tasks. In addition, the technical solution has stronger migration performance and can be fine-tuned for different fields to solve the nested entity recognition problem. Thus, the technical problems of low accuracy, high cost and long cycle of Chinese nested named entity recognition in the prior art are solved.

[0069] Embodiment 3

[0070] Figure 5 An apparatus 500 for nested named entity recognition according to the embodiment is shown, which corresponds to the method according to the first aspect of embodiment 1. Referring to Figure 5 As shown, the apparatus 500 comprises a processor 510 and a memory 520 connected with the processor 510, for providing the processor 510 with instructions for processing the following processing steps: performing named entity recognition on a target sentence by using a preset named entity recognition model to determine a plurality of first entities corresponding to the target sentence; performing recombination processing on the first entities that do not belong to the first entities without nested words according to a preset recombination rule to obtain a word set composed of a plurality of words of different lengths; performing named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities; and determining an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities.

[0071] Optionally, the operation of reorganizing the first entities that do not belong to the first entities without nested words according to the preset reorganization rule to obtain the word set composed of the words of different lengths comprises: judging whether the first entities that do not belong to the first entities without nested words meet a preset first judgment condition, wherein the first judgment condition is whether the entities are single-character entities or belong to preset categories of entities; and performing fine-grained segmentation on the first entities that do not meet the first judgment condition to obtain the word set composed of the words of different lengths.

[0072] Optionally, the operation of performing named entity recognition on each word in the word set by using the named entity recognition model to determine the second entities comprises: judging whether the words in the word set meet a preset second judgment condition, wherein the second judgment condition is whether the words in the word set are contained in the first entities; and performing entity recognition on the words in the word set that do not meet the second judgment condition by using the named entity recognition model to determine the second entities.

[0073] Optionally, the operation of determining the entity recognition result of the target sentence according to the first entities and the second entities comprises: determining the entity recognition result of the target sentence according to the first entities that belong to the first entities without nested words, the first entities that meet the first judgment condition, and the second entities.

[0074] Optionally, before the operation of judging whether the first entities that do not belong to the first entities without nested words meet the preset first judgment condition, the memory 520 is further configured to provide the processor 510 with instructions to process the following processing steps: respectively judging whether the first entities belong to the first entities without nested words by using a preset entity word library, wherein the entity word library is composed of a plurality of entities without nested words.

[0075] Optionally, the memory 520 is further configured to provide the processor 510 with instructions to process the following processing steps: recording the first entities that do not belong to the first entities without nested words in an entity word set; and the operation of judging whether the first entities that do not belong to the first entities without nested words meet the preset first judgment condition comprises: judging whether each entity word in the entity word set is a single-character entity; and judging whether each entity word in the entity word set belongs to a preset category of entities, wherein the preset category of entities includes a personal name entity, a proprietary jargon entity, and a sensitive entity.

[0076] Optionally, before the operation of performing named entity recognition on the target sentence by using the preset named entity recognition model, the memory 520 is further configured to provide the processor 510 with instructions to process the following processing steps: obtaining a to-be-recognized sentence; and performing a preprocessing operation on the to-be-recognized sentence to obtain the target sentence, wherein the preprocessing operation comprises at least any one of the following: data cleaning, text error correction, and short text removal.

[0077] According to the embodiment, the recognition system first performs named entity recognition on the target sentence by using a preset named entity recognition model to determine a plurality of first entities, then performs recombination processing on the first entities that do not belong to the first entities without nested words according to a preset recombination rule to obtain a word set, secondly performs named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities, and finally determines an entity recognition result according to the plurality of first entities and the plurality of second entities. Thus, the recognition system can better solve the nested entity problem through a simple and efficient post-processing process without changing the named entity recognition model, and has the advantages of simple structure and high accuracy. Compared with the existing rule-based or entity library-based method, the technical solution can save a large amount of manpower and material resources, and has better migration performance. In addition, the technical solution does not need an additional network for entity boundary detection and supervision, has lower time complexity and higher reasoning efficiency. The extracted nested information can also be used as auxiliary information or auxiliary features to support downstream tasks and enhance the effect of other tasks. In addition, the technical solution has stronger migration performance and can be fine-tuned for different fields to solve the nested entity recognition problem. Thus, the technical problems of low accuracy, high cost and long cycle of Chinese nested named entity recognition in the prior art are solved.

[0078] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0079] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0080] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0081] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0082] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0083] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0084] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method of nested named entity recognition, characterized by, The method comprises the following steps: performing named entity recognition on the target sentence by using a preset named entity recognition model to determine a plurality of first entities corresponding to the target sentence; performing recombination processing on the first entities that do not belong to the first entities without nested words according to a preset recombination rule to obtain a word set composed of a plurality of words of different lengths; The operation of performing recombination processing on the first entities that do not belong to the first entities without nested words according to a preset recombination rule to obtain a word set composed of a plurality of words of different lengths comprises: determining whether the first entities that do not belong to the first entities without nested words meet a preset first determination condition, wherein the first determination condition is whether the entity is a single-character entity or belongs to a preset category of entities; and performing fine-grained segmentation on the first entities that do not meet the first determination condition to obtain a word set composed of a plurality of words of different lengths; performing named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities; and determining an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities.

2. The method of claim 1, wherein, The operation of performing named entity recognition on each word in the word set by using the named entity recognition model to determine a plurality of second entities comprises: determining whether the words in the word set meet a preset second determination condition, wherein the second determination condition is whether the words in the word set are contained in the plurality of first entities; and performing entity recognition on the words in the word set that do not meet the second determination condition by using the named entity recognition model to determine the plurality of second entities.

3. The method of claim 1, wherein, The operation of determining an entity recognition result of the target sentence according to the plurality of first entities and the plurality of second entities comprises: determining the entity recognition result of the target sentence according to the first entities without nested words, the first entities that meet the first determination condition in the plurality of first entities, and the plurality of second entities.

4. The method of claim 1, wherein, Before the operation of determining whether the first entities that do not belong to the first entities without nested words meet a preset first determination condition, the method further comprises: respectively determining whether the plurality of first entities belong to the first entities without nested words by using a preset entity word library, wherein the entity word library is composed of a plurality of entity words without nested words.

5. The method of claim 4, wherein, The method further comprises: recording the first entities that do not belong to the first entities without nested words in an entity word set; and the operation of determining whether the first entities that do not belong to the first entities without nested words meet a preset first determination condition comprises: determining whether each entity word of the entity word set is a single-character entity; and determining whether each entity word of the entity word set belongs to a preset category of entities, wherein the preset category of entities includes a personal name entity, a proprietary dialogue entity, and a sensitive entity.

6. The method of claim 1, wherein, Before the operation of performing named entity recognition on the target sentence by using a preset named entity recognition model, the method further comprises: Acquire a to-be-recognized sentence; and perform a preprocessing operation on the to-be-recognized sentence to obtain the target sentence, wherein the preprocessing operation includes at least any one of the following: data cleaning, text error correction, and short text removal.

7. A storage medium, characterized by The storage medium includes a stored program, wherein the program is executed by a processor to perform the method of any one of claims 1 to 6 when the program is running.

Images