Named entity recognition method, device, electronic device and storage medium
By using the Elasticsearch database and relevance calculation in named entity recognition, the problem of low efficiency of entity recognition without setting association rules in the existing technology is solved, and efficient named entity recognition is achieved.
Patent Information
- Application Number
- CN202210542109.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-17
- Publication Date
- 2025-09-19
- Estimated Expiration
- 2042-05-17
AI Technical Summary
Existing named entity recognition methods are based on rules and dictionary libraries, and are unable to recognize entities without set association rules, resulting in low recognition efficiency.
Use Elasticsearch's entity retrieval database to query, calculate the character relevance and word order relevance of candidate entities, filter recognition results by entity similarity, and improve recognition efficiency.
Through entity retrieval database query and relevance calculation, efficient recognition of entities without set association rules is achieved, which improves the efficiency of named entity recognition.
Smart Images

Figure CN114881037B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to artificial intelligence technology, and in particular to a named entity recognition method, device, electronic device and storage medium. Background Art
[0002] Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP). NER is an important foundational tool for many NLP tasks, including information extraction, question-answering systems, syntactic analysis, and machine translation.
[0003] However, the current named entity recognition method is based on rules and dictionary libraries, and can only recognize entities for which association rules have been set. It cannot recognize entities for which no association rules are set in the dictionary, and the efficiency of named entity recognition is low. Summary of the Invention
[0004] The present invention provides a method, device, electronic device and storage medium for named entity recognition, the main purpose of which is to improve the efficiency of named entity recognition.
[0005] Obtaining entity text to be identified, and using the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities;
[0006] Calculating character relevance of the candidate entity to obtain entity character relevance;
[0007] Calculating the word order relevance of the candidate entity to obtain the entity word order relevance;
[0008] Calculate the entity similarity of each candidate entity based on the entity character relevance and the entity word order relevance;
[0009] All the candidate entities are screened and identified using the entity similarity to obtain an identification result.
[0010] Optionally, calculating the word order relevance of the candidate entity to obtain the entity word order relevance includes:
[0011] Calculating the cumulative loss gain of the candidate entity to obtain the word order relevance of the candidate entity;
[0012] The candidate entity word order relevance is normalized to obtain the entity word order relevance.
[0013] Optionally, calculating the discounted cumulative gain of the candidate entity to obtain the candidate entity word order relevance includes:
[0014] Determine whether each character starting from the first character in the candidate entity is identical to the characters in the corresponding order in the text to be recognized;
[0015] Determine the word order correlation coefficient corresponding to each character in the candidate entity according to the judgment result of whether the characters are the same;
[0016] The word order correlation coefficient of all characters in the candidate entity is used for calculation to obtain the word order correlation of the candidate entity.
[0017] Optionally, determining the word order correlation coefficient corresponding to each character in the candidate entity according to the result of judging whether the characters are the same includes:
[0018] When the characters in the candidate entity are the same as the characters in the corresponding order in the text to be recognized, the word correlation coefficient corresponding to the characters in the candidate entity is a first preset value;
[0019] When the characters in the candidate entity are different from the characters in the corresponding order in the text to be recognized, the word correlation coefficient corresponding to the character in the candidate entity is a second preset value, wherein the first preset value is greater than the second preset value.
[0020] Optionally, normalizing the candidate entity word order relevance to obtain the entity word order relevance includes:
[0021] Calculating the cumulative loss gain of the entity text to be identified to obtain the word order relevance of the entity to be identified;
[0022] Determining the first preset value as the word order correlation coefficient of each character in the entity text to be recognized;
[0023] Calculating the word order correlation coefficient of all characters in the entity text to be identified to obtain the word order correlation of the entity to be identified;
[0024] The candidate entity word order relevance is normalized according to the to-be-identified entity word order relevance to obtain the entity word order relevance.
[0025] Optionally, the using the entity similarity to screen and identify all the candidate entities to obtain an identification result includes:
[0026] Select the maximum value among all the entity similarities to obtain the target entity similarity;
[0027] The candidate entity corresponding to the target entity similarity is determined as the recognition result.
[0028] Optionally, obtaining the entity text to be identified includes:
[0029] Get the question text;
[0030] Segmenting the text to be recognized to obtain multiple segmented texts;
[0031] The segmented texts existing in a preset non-entity text dictionary are filtered to obtain the entity text to be identified.
[0032] In order to solve the above problems, the present invention further provides a named entity recognition device, comprising:
[0033] A candidate entity query module is used to obtain entity text to be identified, and use the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities;
[0034] A similarity calculation module is used to calculate the character relevance of the candidate entities to obtain entity character relevance; calculate the word order relevance of the candidate entities to obtain entity word order relevance; and calculate based on the entity character relevance and the entity word order relevance to obtain entity similarity of each candidate entity;
[0035] The entity recognition module is used to screen and recognize all the candidate entities using the entity similarity to obtain a recognition result.
[0036] In order to solve the above problem, the present invention further provides an electronic device, comprising:
[0037] a memory storing at least one computer program; and
[0038] The processor executes the computer program stored in the memory to implement the above-mentioned named entity recognition method.
[0039] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned named entity recognition method.
[0040] The embodiment of the present invention calculates the character relevance of the candidate entities to obtain the entity character relevance; calculates the word order relevance of the candidate entities to obtain the entity word order relevance; calculates based on the entity character relevance and the entity word order relevance to obtain the entity similarity of each candidate entity; queries the candidate entities through an Elasticsearch-based entity retrieval database, and then screens and identifies the candidate entities through character relevance and word order relevance, which is not affected by the entity association rules in the database and has higher efficiency in entity recognition; therefore, the named entity recognition method, device, electronic device and readable storage medium proposed in the embodiment of the present invention improve the efficiency of named entity recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 A flowchart of a method for named entity recognition provided by one embodiment of the present invention;
[0042] Figure 2 A schematic diagram of modules of a named entity recognition device provided by one embodiment of the present invention;
[0043] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a method for named entity recognition provided by one embodiment of the present invention;
[0044] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION
[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0046] An embodiment of the present invention provides a method for named entity recognition. The execution subject of the named entity recognition method includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the named entity recognition method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.
[0047] Reference Figure 1FIG. 1 is a flow chart of a method for named entity recognition provided by an embodiment of the present invention. In the embodiment of the present invention, the method for named entity recognition includes:
[0048] S1. Obtain entity text to be identified, and use the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities.
[0049] In detail, the entity text to be identified in the implementation of the present invention is the text that needs to be subject to entity recognition.
[0050] Optionally, in an embodiment of the present invention, the method for obtaining entity text to be identified includes:
[0051] Step I: Get the question text;
[0052] Optionally, the question text in the embodiment of the present invention may be a question text entered by the user, such as "What does Xiaomi do?" In order to more accurately match the answer to the question text entered by the user, it is necessary to identify the entities in the text to be identified, so that the answer matched to the subsequent question text is more accurate. For example, if the entity corresponding to "Xiaomi Company" is identified as "Xiaomi Technology Co., Ltd.", then "What does Xiaomi Company do" can be improved to "What does Xiaomi Technology Co., Ltd.", which makes the question text more accurately expressed, and thus the corresponding answer matching more accurately.
[0053] Step II: segmenting the text to be recognized to obtain multiple segmented texts;
[0054] Optionally, since not all words in the text to be recognized require entity recognition, it is necessary to segment the text to be recognized, thereby filtering out common words in the text to be recognized that do not require entity recognition.
[0055] Step III: Filter the segmented texts in the preset non-entity text dictionary to obtain the entity text to be identified.
[0056] Specifically, the non-entity text dictionary in the embodiment of the present invention is a dictionary database constructed from non-entity text that does not require entity recognition.
[0057] For example, there are three segmented texts: segmented text A, segmented text B, and segmented text C. Segmented text C is in the non-entity text font. Then, segmented text A and segmented text B are both entity texts to be identified.
[0058] Furthermore, the entity retrieval database in the embodiment of the present invention is an Elasticsearch database built based on the elastic search engine. Compared with ordinary database retrieval speed, the retrieval results do not necessarily need to include the searched fields, and the retrieval results are more comprehensive.
[0059] Optionally, the entity index structure defined in the entity retrieval database in the embodiment of the present invention can be defined according to requirements, and usually has at least three items: entityName (entity standard name), entityType (entity classification) and entityAlias (entity alias).
[0060] In another embodiment of the present invention, the entity to be identified can be stored in a blockchain node, utilizing the high throughput characteristics of the blockchain node to improve data access efficiency.
[0061] S2. Calculate character relevance for the candidate entity to obtain entity character relevance.
[0062] Optionally, the font character relevance in the embodiment of the present invention utilizes the bm25 algorithm to calculate the score of the candidate entity in the entity retrieval database.
[0063] In another embodiment of the present invention, the entity character relevance may also be the TF-IDF value of the candidate entity in the entity retrieval database.
[0064] S3. Calculate the word order relevance of the candidate entity to obtain the entity word order relevance;
[0065] Optionally, in an embodiment of the present invention, calculating the word order relevance of the candidate entity to obtain the entity word order relevance includes:
[0066] Step A: Calculate the cumulative loss gain of the candidate entity to obtain the word order relevance of the candidate entity;
[0067] Optionally, the character similarity in the embodiment of the present invention only measures the character relevance between the candidate entity and the entity text to be identified, and does not consider the word order relevance between the candidate entity and the entity text to be identified through the embodiment of the present invention. Therefore, the embodiment of the present invention calculates the discounted cumulative gain of the candidate entity as the evaluation criterion for word order relevance.
[0068] Specifically, in an embodiment of the present invention, the discounted cumulative gain of the candidate entity is calculated to obtain the word order relevance of the candidate entity, including:
[0069] Determine whether each character starting from the first character in the candidate entity is identical to the characters in the corresponding order in the text to be recognized;
[0070] Determine the word order correlation coefficient corresponding to each character in the candidate entity according to the judgment result of whether the characters are the same;
[0071] Calculate using the word order correlation coefficients of all characters in the candidate entity to obtain the word order correlation degree of the candidate entity.
[0072] Specifically, in the embodiment of the present invention, determining the word order correlation coefficient corresponding to each character in the candidate entity according to the judgment result of whether the characters are the same includes:
[0073] When the character in the candidate entity is the same as the character in the corresponding order in the text to be recognized, the word correlation coefficient corresponding to this character in the candidate entity is the first preset value;
[0074] When the character in the candidate entity is different from the character in the corresponding order in the text to be recognized, the word correlation coefficient corresponding to this character in the candidate entity is the second preset value, where the first preset value is greater than the second preset value.
[0075] Optionally, when the character in the candidate entity is the same as the character in the corresponding order in the text to be recognized, the word correlation coefficient corresponding to this character in the candidate entity is 1, and when the character in the candidate entity is different from the character in the corresponding order in the text to be recognized, the second preset value of the word correlation coefficient corresponding to this character in the candidate entity is 0.
[0076] For example: the first preset value is 1, the second preset value is 0, the text of the entity to be recognized is "A B", and the candidate entity is "C B", then the word order correlation coefficient corresponding to the character "C" in the candidate entity is 0, and the word order correlation coefficient corresponding to the character "B" in the candidate entity is 1.
[0077] In another embodiment of the present invention, calculating the discounted cumulative gain of the candidate entity to obtain the word order correlation degree of the candidate entity includes:
[0078] Judge whether each character starting from the last character in the candidate entity is the same as the character in the corresponding order in the text of the entity to be recognized;
[0079] Determine the word order correlation coefficient corresponding to each character in the candidate entity according to the judgment result of whether the characters are the same;
[0080] Calculate using the word order correlation coefficients of all characters in the candidate entity to obtain the word order correlation degree of the candidate entity.
[0081] In another embodiment of the present invention, calculating the discounted cumulative gain of the candidate entity to obtain the word order correlation degree of the candidate entity includes:
[0082] Determine whether each character in the candidate entity exists in the entity text to be identified;
[0083] Determine the word order correlation coefficient corresponding to each character in the candidate entity according to the judgment result of whether the character exists;
[0084] The word order correlation coefficient of all characters in the candidate entity is used for calculation to obtain the word order correlation of the candidate entity.
[0085] Specifically, in an embodiment of the present invention, determining the word order correlation coefficient corresponding to each character in the candidate entity according to the judgment result of whether the character exists includes:
[0086] When each character in the candidate entity exists in the entity text to be identified, the word correlation coefficient corresponding to the character in the candidate entity is a first preset value;
[0087] When each character in the candidate entity does not exist in the entity text to be identified, the word correlation coefficient corresponding to the character in the candidate entity is a second preset value, wherein the first preset value is greater than the second preset value.
[0088] Optionally, in an embodiment of the present invention, the candidate entity word order relevance of the candidate entity is calculated using the following formula:
[0089]
[0090] Where i is the order of characters in the candidate entity, rel i is the word order correlation coefficient of the character in order i in the candidate entity, p is the number of characters in the candidate entity, DCG p is the candidate entity word order relevance of the candidate entity.
[0091] Step B: normalize the candidate entity word order relevance to obtain the entity word order relevance.
[0092] Optionally, in order to reduce data computing resources in an embodiment of the present invention, the candidate entity word order relevance is normalized to obtain the entity word order relevance.
[0093] Specifically, in an embodiment of the present invention, the candidate entity word order relevance is normalized to obtain the entity word order relevance, including:
[0094] Calculating the cumulative loss gain of the entity text to be identified to obtain the word order relevance of the entity to be identified;
[0095] Determining the first preset value as the word order correlation coefficient of each character in the entity text to be recognized;
[0096] The word order correlation coefficient of all characters in the entity text to be identified is used for calculation to obtain the word order correlation of the entity to be identified.
[0097] Optionally, the embodiment of the present invention uses the following formula to calculate the word order relevance of the entity to be identified:
[0098]
[0099] Wherein, j is the order of characters in the entity text to be identified, rel is the word order correlation coefficient of any character in the entity text to be identified, n is the number of characters in the entity text to be identified, IDCG n is the word order relevance of the candidate entity.
[0100] The candidate entity word order relevance is normalized according to the to-be-identified entity word order relevance to obtain the entity word order relevance.
[0101] Optionally, the embodiment of the present invention uses the following formula for normalization:
[0102]
[0103] Where X is the entity word order relevance.
[0104] S4, calculating based on the entity character relevance and the entity word order relevance to obtain the entity similarity of each candidate entity;
[0105] Optionally, in an embodiment of the present invention, the entity character relevance and the entity word order relevance are multiplied to obtain the entity relevance.
[0106] For example, if the entity character relevance corresponding to candidate entity A is 0.8 and the entity word order relevance corresponding to candidate entity A is 0.5, then the entity relevance corresponding to candidate entity A is 0.8*0.5=0.4.
[0107] S5. Screen and identify all the candidate entities using the entity similarity to obtain an identification result.
[0108] In detail, in the embodiment of the present invention, all the candidate entities are screened and identified using the entity similarity to obtain an identification result, including:
[0109] Select the maximum value among all the entity similarities to obtain the target entity similarity;
[0110] The candidate entity corresponding to the target entity similarity is determined as the recognition result.
[0111] like Figure 2, which is a functional module diagram of the named entity recognition device of the present invention.
[0112] The named entity recognition device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the named entity recognition device can include a candidate entity query module 101, a similarity calculation module 102, and an entity recognition module 103. The modules described herein can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can perform fixed functions, and are stored in the memory of the electronic device.
[0113] In this embodiment, the functions of each module / unit are as follows:
[0114] The candidate entity query module 101 is used to obtain entity text to be identified, and use the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities;
[0115] The similarity calculation module 102 is used to calculate the character relevance of the candidate entities to obtain the entity character relevance; calculate the word order relevance of the candidate entities to obtain the entity word order relevance; and calculate based on the entity character relevance and the entity word order relevance to obtain the entity similarity of each candidate entity;
[0116] The entity recognition module 103 is used to screen and recognize all the candidate entities using the entity similarity to obtain a recognition result.
[0117] In detail, each module in the named entity recognition device 100 according to the embodiment of the present invention adopts the same Figure 1 The technical means are the same as the named entity recognition method described in , and can produce the same technical effects, so I will not go into details here.
[0118] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing the method for named entity recognition according to the present invention.
[0119] The electronic device may include a processor 10 , a memory 11 , a communication bus 12 , and a communication interface 13 , and may further include a computer program stored in the memory 11 and executable on the processor 10 , such as a named entity recognition program.
[0120] Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example: SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a named entity recognition program, etc., but can also be used to temporarily store data that has been output or is to be output.
[0121] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing programs or modules (such as named entity recognition programs) stored in the memory 11, as well as calling data stored in the memory 11, to perform various functions of the electronic device and process data.
[0122] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Such a bus may be classified as an address bus, a data bus, a control bus, etc. The communication bus 12 is configured to enable communication between the memory 11 and at least one processor 10. For ease of illustration, the figure shows only one thick line, but this does not imply that there is only one bus or only one type of bus.
[0123] Figure 3 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 3The structure shown does not limit the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0124] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure classification circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.
[0125] Optionally, the communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices.
[0126] Optionally, the communication interface 13 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.
[0127] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.
[0128] The named entity recognition program stored in the memory 11 of the electronic device is a combination of multiple computer programs. When running in the processor 10, it can achieve the following:
[0129] Obtaining entity text to be identified, and using the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities;
[0130] Calculating character relevance of the candidate entity to obtain entity character relevance;
[0131] Calculating the word order relevance of the candidate entity to obtain the entity word order relevance;
[0132] Calculate the entity similarity of each candidate entity based on the entity character relevance and the entity word order relevance;
[0133] All the candidate entities are screened and identified using the entity similarity to obtain an identification result.
[0134] Specifically, the specific implementation method of the processor 10 for the above computer program can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.
[0135] Furthermore, if the module / unit integrated into the electronic device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or volatile. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, or read-only memory (ROM).
[0136] An embodiment of the present invention may further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program may implement:
[0137] Obtaining entity text to be identified, and using the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities;
[0138] Calculating character relevance of the candidate entity to obtain entity character relevance;
[0139] Calculating the word order relevance of the candidate entity to obtain the entity word order relevance;
[0140] Calculate the entity similarity of each candidate entity based on the entity character relevance and the entity word order relevance;
[0141] All the candidate entities are screened and identified using the entity similarity to obtain an identification result.
[0142] Furthermore, the computer-usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.
[0143] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.
[0144] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.
[0145] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.
[0146] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.
[0147] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0148] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.
[0149] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.
[0150] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.
[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for named entity recognition, characterized in that: The method comprises: Obtaining entity text to be identified, and using the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities; Calculating a score of the candidate entity in the entity search database, or obtaining a TF-IDF value of the candidate entity in the entity search database, and using the score or TF-IDF value as the entity character relevance of the candidate entity; Calculating the discounted cumulative gain of the candidate entity to obtain the candidate entity word order relevance, including: determining whether each character starting from the first character in the candidate entity is identical to the characters of the corresponding order in the to-be-identified entity text, determining the word order relevance coefficient corresponding to each character in the candidate entity based on the determination result of whether the characters are identical, and calculating using the word order relevance coefficients of all characters in the candidate entity to obtain the candidate entity word order relevance; Normalizing the candidate entity word order relevance to obtain the entity word order relevance; Calculate the entity similarity of each candidate entity based on the entity character relevance and the entity word order relevance; All the candidate entities are screened and identified using the entity similarity to obtain an identification result.
2. The named entity recognition method according to claim 1, wherein The determining of the word order correlation coefficient corresponding to each character in the candidate entity according to the result of judging whether the characters are the same includes: When the characters in the candidate entity are the same as the characters in the corresponding order in the entity text to be recognized, the word correlation coefficient corresponding to the characters in the candidate entity is a first preset value; When the characters in the candidate entity are different from the characters in the corresponding order in the entity text to be identified, the word correlation coefficient corresponding to the character in the candidate entity is a second preset value, wherein the first preset value is greater than the second preset value.
3. The named entity recognition method according to claim 2, wherein: Normalizing the candidate entity word order relevance to obtain the entity word order relevance includes: Determining the first preset value as the word order correlation coefficient of each character in the entity text to be recognized; Calculating the word order correlation coefficient of all characters in the entity text to be identified to obtain the word order correlation of the entity to be identified; The candidate entity word order relevance is normalized according to the to-be-identified entity word order relevance to obtain the entity word order relevance.
4. The named entity recognition method according to claim 1, wherein The method of screening and identifying all candidate entities by using the entity similarity to obtain an identification result includes: Select the maximum value among all the entity similarities to obtain the target entity similarity; The candidate entity corresponding to the target entity similarity is determined as the recognition result.
5. A named entity recognition device, characterized in that: include: A candidate entity query module is used to obtain entity text to be identified, and use the entity text to be identified to query a pre-built Elasticsearch-based entity retrieval database to obtain one or more candidate entities; A similarity calculation module is used to calculate the score of the candidate entity in the entity search database, or obtain the TF-IDF value of the candidate entity in the entity search database, and use the score or TF-IDF value as the entity character relevance of the candidate entity; The similarity calculation module is further used to calculate the discounted cumulative gain of the candidate entity to obtain the candidate entity word order relevance, including: determining whether each character starting from the first character in the candidate entity is the same as the characters of the corresponding order in the to-be-identified entity text, determining the word order relevance coefficient corresponding to each character in the candidate entity based on the determination result of whether the characters are the same, and using the word order relevance coefficients of all characters in the candidate entity to perform calculations to obtain the candidate entity word order relevance; The similarity calculation module is further used to normalize the candidate entity word order relevance to obtain the entity word order relevance; The similarity calculation module is further configured to calculate the entity similarity of each candidate entity based on the entity character relevance and the entity word order relevance; The entity recognition module is used to screen and recognize all the candidate entities using the entity similarity to obtain a recognition result.
6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the named entity recognition method according to any one of claims 1 to 4.
7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for named entity recognition according to any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Chinese place name inquiry method, device and equipment
CN109145161A
Object sorting method and system
CN113343101A