Entity link association method, system, terminal device and storage medium

Abstract

The present invention discloses an entity linking association method, system, terminal device, and storage medium. The method comprises: receiving a query text; screening the query text to obtain abbreviation clusters; and inputting the abbreviation clusters into a pre-created database for processing to obtain entities corresponding to the query text. The present invention solves the problem of associating abbreviations with full names and improves the accuracy of entity linking.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to an entity link association method, system, terminal device and storage medium. Background Art

[0002] With the development of internet media, online public opinion has undergone significant changes in terms of data volume, complexity, and release speed. Linking company abbreviations, products, stocks, and other information in public opinion to the correct full company name is crucial for public opinion analysis. However, media outlets often use a variety of names for the same company, and the same abbreviation may represent multiple different companies. Entity linking and disambiguation are necessary to analyze company aliases mentioned in public opinion and determine the company's true full name. Currently, the primary method for linking abbreviations mentioned in public opinion is to measure textual similarity between entities and the company's own strengths, but the accuracy of the resulting entities is limited. Summary of the Invention

[0003] The main purpose of the embodiments of the present invention is to provide an entity link association method, system, terminal device and storage medium, which solve the problem of associating the abbreviation and full name and improve the accuracy of entity linking.

[0004] To achieve the above-mentioned purpose, an embodiment of the present invention provides an entity link association method, which includes:

[0005] Receive the text to be queried;

[0006] Screening the text to be queried to obtain abbreviation clusters;

[0007] The abbreviation clusters are input into a pre-created database for processing to obtain entities corresponding to the text to be queried.

[0008] Optionally, before the step of clustering the abbreviations into a pre-created database for processing to obtain entities corresponding to the text to be queried, the step includes:

[0009] establishing the database;

[0010] Wherein, the step of establishing the database includes:

[0011] Get a sample set;

[0012] Based on the sample set, generate a product library and a knowledge graph;

[0013] Recalling the name of each sample in the sample set to obtain a name set;

[0014] Establishing a full name candidate library based on the knowledge graph, the name set, and the sample set;

[0015] The product library, the name set, and the full name candidate library are used as the database.

[0016] Optionally, the step of generating a knowledge graph based on the sample set includes:

[0017] Filtering the sample set to obtain similar co-occurring abbreviation-full name pairs in the sample set;

[0018] Generating the knowledge graph based on the similar co-occurring short-full name pairs;

[0019] The full name candidate library is established based on the knowledge graph, the name set, and the similar co-occurring abbreviation and full name pairs.

[0020] Optionally, the step of recalling the name of each sample in the sample set to obtain a name set includes:

[0021] Recall the name of each sample in the sample set, and score the name of each sample in the sample set to obtain an entity score;

[0022] Processing the entity score to obtain a normalized score;

[0023] The name set is obtained based on the normalized score and the name of each sample in the sample set.

[0024] Optionally, after the step of establishing a full name candidate library based on the knowledge graph, the name set, and the sample set, the following steps are performed:

[0025] When the preset time is met, the sample set is updated and the execution step is returned to: generating a product library and a knowledge graph based on the sample set.

[0026] Optionally, the step of generating a product library based on the sample set includes:

[0027] Screening samples in the sample set to obtain co-occurrence samples;

[0028] The product library is generated based on the co-occurrence samples.

[0029] Optionally, the step of establishing the full name candidate library based on the knowledge graph, the name set, and the similar co-occurring abbreviation-full name pairs includes:

[0030] Based on the knowledge graph and the name set, calculating a first similar full name and a first similarity score;

[0031] Based on the knowledge graph and the similar co-occurring short and full name pairs, a second similar full name and a second similarity score are calculated;

[0032] Based on the first similar full name, the first similarity score, the second similar full name, and the second similarity score, the full name candidate library is established.

[0033] In addition, to achieve the above-mentioned purpose, the present invention further provides an entity link association system, the system comprising:

[0034] A text receiving module, used for receiving the text to be queried;

[0035] A text screening module, used to screen the text to be queried to obtain abbreviation clusters;

[0036] The entity query module is used to input the abbreviation clusters into a pre-created database for processing to obtain the entity corresponding to the text to be queried.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes: a memory, a processor, and an entity link association method stored in the memory and runnable on the processor, and the entity link association program implements the steps of the entity link association method described above when executed by the processor.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a program for entity link association is stored. When the program for entity link association is executed by a processor, the steps of the entity link association method described above are implemented.

[0039] The entity linking association method, system, terminal device, and storage medium proposed in embodiments of the present invention receive a query text; filter the query text to obtain abbreviation clusters to organize the query text and improve the efficiency of querying corresponding entities; input the abbreviation clusters into a pre-created database for processing to obtain entities corresponding to the query text, thereby achieving the purpose of querying corresponding entities based on the query text and facilitating subsequent correlation analysis based on the entities corresponding to the query text. Through the above-mentioned methods, the present invention solves the problem of associating abbreviations with full names and improves the accuracy of entity linking. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the entity link association device of the present invention belongs;

[0041] Figure 2 Schematic diagram of the process of the first embodiment of the entity link association method of the present invention;

[0042] Figure 3 Schematic diagram of the flow of the third embodiment of the entity link association method of the present invention;

[0043] Figure 4A flowchart of a specific embodiment of the entity link association method of the present invention is shown;

[0044] Figure 5 Schematic diagram of another embodiment of the entity link association method of the present invention;

[0045] Figure 6 Schematic diagram of a flow chart of another specific embodiment of the entity link association method of the present invention;

[0046] Figure 7 Schematic diagram of the functional modules of the entity link association system of the present invention.

[0047] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] The main solution of the embodiment of the present invention is: receiving a text to be queried; screening the text to be queried to obtain abbreviation clusters; inputting the abbreviation clusters into a pre-created database for processing to obtain entities corresponding to the text to be queried.

[0050] Technical terms involved in the embodiments of the present invention:

[0051] NER (Named Entity Recognition) is an important foundational tool for applications such as information extraction, question-answering systems, syntactic analysis, and machine translation. Generally speaking, the task of named entity recognition is to identify three major categories (entity, time, and number) and seven subcategories (personal names, organization names, place names, time, date, currency, and percentage) in the text to be processed.

[0052] EL: (Entity Linking), which means linking the query text to a standard name in the database.

[0053] SPARK: Spark is a general-purpose engine that can be used to perform various operations, including SQL queries, text processing, machine learning, etc. Before the emergence of Spark, we generally needed to learn a variety of engines to handle these needs separately.

[0054] Hive: Hive is a data warehouse tool based on Hadoop that extracts, transforms, and loads data. It is a mechanism for storing, querying, and analyzing large amounts of data stored in Hadoop. The Hive data warehouse tool maps structured data files into database tables and provides SQL query functionality, converting SQL statements into MapReduce tasks for execution.

[0055] ES: Elasticsearch is a near real-time search platform. There is only a slight delay from indexing a document to its availability for search. ES calculates the similarity based on the degree of matching after word segmentation, which is called the ES Match score.

[0056] Entity linking: mapping certain strings in a text to corresponding entities in the knowledge base.

[0057] With the development of internet media, online public opinion has undergone tremendous changes in terms of data volume, complexity, and release speed. Mastering public opinion monitoring not only provides businesses with real-time customer risk warnings but also enables them to identify business opportunities within massive amounts of news. Converting this vast amount of news text into risk and marketing signals can significantly aid businesses in intelligent investment, digital risk management, and precision marketing. Linking company abbreviations, products, and stocks mentioned in public opinion to the correct full company names is crucial for further analysis of events and sentiment. Existing technologies primarily link abbreviations mentioned in public opinion to entities based on textual similarity between entities and the company's own strengths. This results in limited optimization potential and a low accuracy rate.

[0058] The present invention provides a solution to solve the problem of associating the abbreviation and full name, and improves the accuracy of entity linking.

[0059] Specifically, refer to Figure 1 , Figure 1 This is a schematic diagram of the functional modules of a terminal device to which the entity link association device of the present invention belongs. The entity link association device can be a device independent of the terminal device, capable of image processing and network model training, and can be hosted on the terminal device in the form of hardware or software. The terminal device can be a smart mobile terminal with data processing capabilities, such as a mobile phone or tablet computer, or a fixed terminal device or server with data processing capabilities.

[0060] In this embodiment, the terminal device to which the entity link association apparatus belongs includes at least an output module 110 , a processor 120 , a memory 130 and a communication module 140 .

[0061] The memory 130 stores the operation method and the entity link association program; the output module 110 can be a display screen, etc. The communication module 140 can include a WIFI module, a mobile communication module, and a Bluetooth module, etc., and communicates with an external device or server through the communication module 140.

[0062] When the entity link association program in the memory 130 is executed by the processor, the following steps are implemented:

[0063] Receive the text to be queried;

[0064] Screening the text to be queried to obtain abbreviation clusters;

[0065] The abbreviation clusters are input into a pre-created database for processing to obtain entities corresponding to the text to be queried.

[0066] Furthermore, when the entity link association program in the memory 130 is executed by the processor, the following steps are also implemented:

[0067] establishing the database;

[0068] Wherein, the step of establishing the database includes:

[0069] Get a sample set;

[0070] Based on the sample set, generate a product library and a knowledge graph;

[0071] Recalling the name of each sample in the sample set to obtain a name set;

[0072] Establishing a full name candidate library based on the knowledge graph, the name set, and the sample set;

[0073] The product library, the name set, and the full name candidate library are used as the database.

[0074] Furthermore, when the entity link association program in the memory 130 is executed by the processor, the following steps are also implemented:

[0075] Filtering the sample set to obtain similar co-occurring abbreviation-full name pairs in the sample set;

[0076] Generating the knowledge graph based on the similar co-occurring short-full name pairs;

[0077] The full name candidate library is established based on the knowledge graph, the name set, and the similar co-occurring abbreviation and full name pairs.

[0078] Furthermore, when the entity link association program in the memory 130 is executed by the processor, the following steps are also implemented:

[0079] Recall the name of each sample in the sample set, and score the name of each sample in the sample set to obtain an entity score;

[0080] Processing the entity score to obtain a normalized score;

[0081] The name set is obtained based on the normalized score and the name of each sample in the sample set.

[0082] Furthermore, when the entity link association program in the memory 130 is executed by the processor, the following steps are also implemented:

[0083] When the preset time is met, the sample set is updated and the execution step is returned to: generating a product library and a knowledge graph based on the sample set.

[0084] Furthermore, when the entity link association program in the memory 130 is executed by the processor, the following steps are also implemented:

[0085] Screening samples in the sample set to obtain co-occurrence samples;

[0086] The product library is generated based on the co-occurrence samples.

[0087] Furthermore, when the entity link association program in the memory 130 is executed by the processor, the following steps are also implemented:

[0088] Based on the knowledge graph and the name set, calculating a first similar full name and a first similarity score;

[0089] Based on the knowledge graph and the similar co-occurring short and full name pairs, a second similar full name and a second similarity score are calculated;

[0090] The full name candidate library is established based on the first similar full name, the first similarity score, the second similar full name, and the second similarity score.

[0091] This embodiment, through the above-mentioned scheme, receives a query text; filters the query text to obtain abbreviation clusters; and inputs the abbreviation clusters into a pre-created database for processing to obtain entities corresponding to the query text. This invention solves the problem of associating abbreviations with full names and improves the accuracy of entity linking.

[0092] Based on the above terminal device architecture but not limited to the above architecture, an embodiment of the method of the present invention is proposed.

[0093] Reference Figure 2 , Figure 2 Schematic diagram of the first embodiment of the entity linking association method of the present invention. The entity linking association method includes:

[0094] Step S101: receiving a text to be searched.

[0095] Step S102: screening the text to be queried to obtain abbreviation clusters.

[0096] The execution subject of the method of this embodiment can be an entity link association device, or an entity link association terminal device or server. This embodiment takes the entity link association device as an example. The entity link association device can be integrated into a terminal device such as a smart phone, tablet computer, etc. with data processing capabilities. The entity link association device has a search engine.

[0097] To resolve the issue of associating abbreviations with full names and improve the accuracy of entity linking, it is necessary to obtain the query text and perform NER processing on it to obtain the NER results for the query text. The query text can be news, public opinion, or other information. The NER results for the query text include all company names included in the processed text.

[0098] Furthermore, the query text is screened to obtain abbreviation clusters.

[0099] Specifically, based on the co-occurrence alias similarity, the NER results of the query text are screened to obtain similar co-occurrence abbreviation pairs of the processed text; the similar co-occurrence abbreviation pairs are further screened to obtain abbreviation clusters, wherein the similarity is the number of co-occurrence articles, and the similarity is determined by at least the following methods:

[0100] The first method is to consider two abbreviations similar if their edit distance is equal to the difference in the number of characters in the two abbreviations, ensuring that the characters of the shorter abbreviation appear in the order of the full name, for example, "AB" and "AaBb Company".

[0101] The second method: If the edit distance between two abbreviations is 1 and the lengths of the two abbreviations are equal, it can be determined that only one character in the two abbreviations is different. The different characters must have the same pinyin, and the abbreviations are considered similar. For example, "ABC" and "Abc".

[0102] The third method: If the difference in the number of characters between the two abbreviations minus the edit distance is 2, the shorter abbreviation can be considered similar if two characters are different from the longer abbreviation. The two different characters must represent the company type (group, company, stock, etc.). For example, "A Group" and "Abc Company" are considered similar.

[0103] For example, the NER results of the processed text include the following abbreviations:

[0104] A, Ab, AAb, BB, Bk, etc. A, Ab, and AAb are classified as one-category sets, BB and Bk are classified as two-category sets, and the one-category sets and the two-category sets are referred to as clustering.

[0105] If the corresponding entities are directly associated with the text to be processed, the speed of obtaining the corresponding entities is slow and the accuracy is low. However, by processing the text to be processed, abbreviation clustering is obtained, which improves the processing efficiency of associating the corresponding entities with the text to be processed.

[0106] Step S103: clustering the abbreviations into a pre-created database for processing to obtain entities corresponding to the text to be queried.

[0107] The abbreviation cluster is input into a pre-created database for processing to obtain entities corresponding to the text to be queried; wherein the abbreviation cluster includes at least one abbreviation set, and each abbreviation set includes at least one abbreviation.

[0108] Specifically, the abbreviation clusters are input into a pre-created database for processing. Since abbreviation clusters include at least one abbreviation set, the database is searched sequentially in descending order of length for each abbreviation set. Since longer text contains more information, longer abbreviations are less ambiguous, so the database is searched sequentially in descending order of length. The database includes a product library, a full name candidate library, and a name set.

[0109] For example, the abbreviation cluster includes a first set and a second set. The first set includes A, Ab, and AAb, and the second set includes BB and Bk. When querying based on the first set, AAb is first matched against the product library, full name candidate library, and name set in the database. If a match is found, it is output. If no match is found, Ab is then matched against the product library, full name candidate library, and name set in the database. If a match is found, it is output. If no match is found, A is then matched against the database, and so on.

[0110] Therefore, the abbreviation clusters are input into a pre-created database for processing to obtain the entities corresponding to the text to be queried, which improves the average analysis speed, solves the performance problem of company entity association, integrates the co-occurrence features and entity links of the product library, and greatly improves the accuracy of the association between the abbreviation and full name of the public opinion cloud company.

[0111] This embodiment, through the above-mentioned solution, receives a query text; filters the query text to obtain a cluster of abbreviations; and inputs the cluster of abbreviations into a pre-created database for processing to obtain the entity corresponding to the query text. This invention improves average analysis speed, solves performance issues related to company entity association, integrates co-occurrence features and entity links in the product library, and significantly improves the accuracy of associating company abbreviations and full names in the public opinion cloud.

[0112] Based on the above Figure 2The embodiment shown is a second embodiment of the entity linking association method of the present invention. In this embodiment, step S103: inputting the abbreviation cluster into a pre-created database for processing to obtain the entity corresponding to the query text includes:

[0113] Step S104: Obtain a sample set.

[0114] Step S105: Generate a product library and a knowledge graph based on the sample set.

[0115] As an implementation method, in this embodiment, named entity recognition and entity linking are performed on a large amount of news offline to build a database, so as to achieve the association between the abbreviation and the full name based on the database.

[0116] For entity linking research in academia, the entity in the knowledge base is generally embedded, the attention mechanism is used to obtain the representation of the context, and finally the pair-wise coherence score between the entity and the context is calculated to complete EL.

[0117] Different from the entity linking in academia, the properties of candidate entities are all companies with no difference, and the excluded candidates may also be names of people, book titles, place names, etc. In this embodiment, EL can be used and simplified, and other entities in the news can be used as context. If two company aliases appear in the same news at the same time, it is co-occurrence, and two related abbreviations and full names are almost inevitable to co-occur with a certain amount of corpus accumulation. Then co-occurrence can be equivalent to the coherence score, which is another important feature in addition to the semantic similarity between the two entities. Therefore, the required database and search engine are constructed using the existing news library.

[0118] First, obtain a sample set and generate a product library and knowledge graph based on the sample set.

[0119] Specifically, we took 1.3 million named entity recognition abbreviations from 1 million news articles, used Spark and Hive to analyze and store the 60 million co-occurring alias pairs and the number of co-occurring chapters as a sample set, and imported it into ES for easy query.

[0120] Furthermore, a product library is generated based on the sample set.

[0121] Specifically, based on whether the company entity and the abbreviation co-occur, the samples in the sample set are screened to obtain co-occurrence samples; based on the co-occurrence samples, a product library is generated.

[0122] If the company entity and abbreviation of the sample in the sample set co-occur, the company entity and abbreviation are regarded as co-occurrence samples; if the company entity and abbreviation of the sample in the sample set do not co-occur, the company entity and abbreviation are not regarded as co-occurrence samples.

[0123] For example, a company entity with 1.3 million abbreviations is searched in the software copyright and trademark database. If the searched company entity and abbreviation co-occur, the company entity and abbreviation are stored in one of the product databases.

[0124] Since the construction of this part of the abbreviation and full name not only refers to news but also uses other corporate data public libraries, it is considered to be the most confident library, and online entity links will be queried first in the future.

[0125] Furthermore, a knowledge graph is generated based on the sample set.

[0126] Specifically, the sample set is screened to obtain similar co-occurring abbreviation-full name pairs of the sample set; and a knowledge graph is generated based on the similar co-occurring abbreviation-full name pairs.

[0127] More specifically, the samples in the sample set are screened based on similarity to obtain similar co-occurring abbreviation pairs; similar co-occurring abbreviation pairs are further screened based on similarity to obtain similar co-occurring abbreviation-full name pairs; and a knowledge graph is generated based on similar co-occurring abbreviation-full name pairs. Similarity judgment includes at least the following methods:

[0128] The first method is to consider two abbreviations similar if their edit distance is equal to the difference in the number of characters in the two abbreviations, ensuring that the characters of the shorter abbreviation appear in the order of the full name, for example, "AB" and "AaBb Company".

[0129] The second method: If the edit distance between two abbreviations is 1 and the lengths of the two abbreviations are equal, the abbreviations are considered similar. It can be determined that there is only one different character in the two abbreviations, and the different characters must have the same pinyin. For example, "ABC" and "Abc".

[0130] The third method: If the difference in the number of characters between the two abbreviations minus the edit distance is 2, it can be determined that the shorter abbreviation has two characters that are different from the longer abbreviation. The two different characters must represent the company type (group, company, stock, etc.), and the abbreviations are considered similar. For example, "A Group" and "Abc Company".

[0131] For example, we calculated 60 million co-occurring aliases, filtered out similar co-occurring abbreviation pairs based on similarity, and then further filtered out similar co-occurring abbreviation-full name pairs, where the similarity is the number of co-occurring articles. We then used 1.3 million similar co-occurring abbreviation-full name pairs to generate a knowledge graph.

[0132] Therefore, based on the sample set, a product library and knowledge graph are generated. The product library includes commonly used products and their corresponding company names. Based on the capacity of the product library, the product library has the characteristic of high search efficiency.

[0133] Step S106: Recall the name of each sample in the sample set to obtain a name set.

[0134] Step S107: establishing a full name candidate library based on the knowledge graph, the name set, and the sample set.

[0135] Recall the names of each sample in the sample set to obtain a name set; establish a full name candidate library based on the knowledge graph, name set, and sample set.

[0136] Specifically, a full name candidate library is established based on the knowledge graph, name set, and similar co-occurring abbreviation-full name pairs.

[0137] More specifically, based on the knowledge graph and the name set, the first similar full name and the first similarity score are calculated; based on the knowledge graph and similar co-occurring abbreviation-full name pairs, the second similar full name and the second similarity score are calculated.

[0138] Furthermore, a full name candidate library is established based on the first similar full name, the first similarity score, the second similar full name, and the second similarity score.

[0139] Specifically, a first weight is set for the first similarity score according to the first similar full name, and a second weight is set for the second similarity score according to the second similar full name; and a full name candidate library is established based on the first weight, the first similarity score, the second weight, and the second similarity score.

[0140] For example, a first weight is set for the first similarity score according to the first similar full name, and a second weight is set for the second similarity score according to the second similar full name; based on the first weight, the first similarity score, the second weight, and the second similarity score, a TOP10 full name candidate library is established.

[0141] Therefore, a full name candidate library is established based on the knowledge graph, name set, and sample set; the capacity of the full name candidate library is larger than that of the product library. Based on the capacity of the full name candidate library, the full name candidate library has the characteristic of high search efficiency.

[0142] Step S108: Use the product library, the name set, and the full name candidate library as the database.

[0143] In this embodiment, the database consists of a product library, a name set, and a full name candidate library.

[0144] When the preset time is met, the sample set is updated and the process returns to step S105: generating a product library and a knowledge graph based on the sample set.

[0145] The preset time can be set according to actual conditions. In this embodiment, 00:00-05:00 is used as the preset time.

[0146] Therefore, when the preset time is met, the sample set is updated based on information on the network, and the process returns to step S105: generating a product library and knowledge graph based on the sample set. This updates the database, allowing company aliases and products that are not included in the knowledge graph but are breaking news to be quickly processed and added back to the knowledge graph, achieving dynamic entity linking.

[0147] This embodiment, through the above-mentioned scheme, receives a text to be queried; screens the text to be queried to obtain a cluster of abbreviations; and inputs the cluster of abbreviations into a pre-created database for processing to obtain the entity corresponding to the text to be queried. The present invention uses the existing industrial and commercial registered enterprise database as a knowledge base, constructs company attributes using basic enterprise information, trademarks, and software copyrights, accumulates NER recognition results by analyzing massive amounts of public opinion text, formulates text and co-occurrence similarity calculation rules, and establishes a knowledge graph of companies, aliases, products, etc. in conjunction with the industrial and commercial enterprise knowledge base, thereby realizing graph-based company entity linking.

[0148] Reference Figure 3 , Figure 3 This is a flow chart of the third embodiment of the entity link association method of the present invention. Based on the above second embodiment, in this embodiment, step S106: recall the name of each sample in the sample set, and the obtained name set includes:

[0149] Step S1061 : Recall the name of each sample in the sample set, and score the name of each sample in the sample set to obtain an entity score.

[0150] As an implementation method, in this embodiment, the name of each sample in the sample set is recalled, and the name of each sample in the sample set is scored based on similarity to obtain an entity score; wherein the similarity is scored based on the following dimensions:

[0151] The first category represents the company's own comprehensive strength: the company's registered capital, number of relevant documents (bid success, patents, qualification certificates, etc.).

[0152] The second category represents the correlation between the abbreviation and the company: the similarity between the abbreviation and the full name, the similarity of the main components of the abbreviation and the full name (excluding place names, business formats, types, etc.), a certain penalty if the character order of the abbreviation and the full name is inconsistent, and the ES match score.

[0153] Specifically, the name of each sample in the sample set is recalled using the business registration database of the enterprise, and the name of each sample in the sample set is scored according to similarity to obtain an entity score.

[0154] Since the business registration database of enterprises is highly accurate, the entity scores obtained have a high accuracy rate.

[0155] Step S1062: Process the entity score to obtain a normalized score.

[0156] Step S1063 : Obtain the name set based on the normalized score and the name of each sample in the sample set.

[0157] The entity scores are processed to obtain a normalized score; based on the normalized score and the name of each sample in the sample set, a name set is obtained.

[0158] Specifically, the entity score of the sample includes scores of multiple dimensions, and the entity score is normalized to obtain a normalized score; the entity names of the sample are sorted according to the normalized score to obtain a name set.

[0159] This embodiment, through the above-described scheme, receives a query text; filters the query text to obtain a cluster of abbreviations; and inputs the cluster of abbreviations into a pre-created database for processing to obtain entities corresponding to the query text. This invention solves the problem of associating abbreviations with full names. Because the business registration database of enterprises is highly accurate, the resulting entity scores are highly accurate, and thus the resulting name set is also highly accurate, improving the accuracy of entity linking.

[0160] Reference Figure 4 , Figure 4 The flowchart of a specific embodiment of the entity link association method of the present invention is shown in FIG. Based on the third embodiment described above, in this embodiment, the full name of the entity corresponding to the enterprise abbreviation is recalled in ES according to the enterprise abbreviation, wherein ES recall (a) indicates that the recalled full name contains all the characters in the abbreviation, and ES recall (b) indicates that the recalled full name contains at least one character in the abbreviation.

[0161] Furthermore, the full name of the entity corresponding to the corporate abbreviation is scored according to the preset standards to obtain a total score; wherein the preset standards are as follows:

[0162] (1) Score the full name of the entity based on the registered capital to obtain the registered capital score.

[0163] Based on the preset grade weights, the registered capital of the company corresponding to the full name of the entity is scored, wherein the registered capital is divided into grades in advance and each grade is assigned a corresponding weight as the above-mentioned preset grade weights.

[0164] (2) Score the full name of the entity based on the relevant documents to obtain the relevant document score.

[0165] Based on the preset document grade weights, the documents of the company corresponding to the full name of the entity are scored, wherein the company's relevant documents, such as patents, certification reports, etc., are assigned corresponding grade weights in advance as the above-mentioned preset document grade weights.

[0166] (3) Score the entity full name based on the matching of the short and full name to obtain the short and full name direct matching score.

[0167] The full name of the entity is scored according to the matching degree between the company abbreviation and the full name of the entity to obtain the direct matching score of the abbreviation and full name.

[0168] (4) Score the entity full name according to the principal components of the abbreviation and full name to obtain the principal component matching score of the abbreviation and full name.

[0169] The principal components of the entity full name are identified according to the principal component recognition algorithm, and corresponding weights are assigned to each component of the entity full name. The string similarity is calculated to score the entity full name.

[0170] (5) Score the full name of the entity according to Reverse to obtain the Reverse score.

[0171] If the order of characters in the company abbreviation is different from the full name, the full name will be penalized and its reverse score will be lowered. For example, if the company abbreviation is AD, the full name 1 is ABCD, and the full name 2 is DBCA, then the reverse score of full name 1 will be higher than that of full name 2 because the order of characters A and D in full name 1 is the same as in the abbreviation.

[0172] (6) Scoring the full name of the entity based on ES to obtain the ES match score. ES itself will segment the query text and then calculate the similarity between the segmented results and the database one by one to obtain a similarity score. The similarity calculation score provided by ES is used as the ES Match score.

[0173] Furthermore, the registered capital score, related document score, abbreviation and full name direct match score, abbreviation and full name principal component match score, reverse score, and ES match score are normalized to obtain the total score.

[0174] Among them, the full names of the entities recalled based on the corporate abbreviations have a corresponding total score. The full names of the entities of the corporate abbreviations are sorted according to the total scores and assigned corresponding grades to obtain a name set.

[0175] Therefore, the full entity name of the enterprise is scored through multiple dimensions to make the score more in line with the reality and improve the accuracy of the matching of the abbreviation and full name.

[0176] This embodiment, through the above-mentioned solution, receives a query text; filters the query text to obtain a cluster of abbreviations; and inputs the cluster of abbreviations into a pre-created database for processing to obtain the entity corresponding to the query text. This invention improves average analysis speed, solves performance issues related to company entity association, integrates co-occurrence features and entity links in the product library, and significantly improves the accuracy of associating company abbreviations and full names in the public opinion cloud.

[0177] Reference Figure 5 , Figure 5 This is a flow chart of another specific embodiment of the entity linking association method of the present invention. Based on the above second embodiment, in this embodiment, text similarity and co-occurrence similarity are calculated based on the abbreviation, associated abbreviations, and full name. The full name is then sorted based on text similarity and co-occurrence similarity to obtain the top 10 results.

[0178] First, based on the knowledge graph and name set, the text similarity of the abbreviation and full name is calculated.

[0179] Specifically, the first similarity between the abbreviation and the full name and the number of the first full names are calculated based on the knowledge graph, where the number of the first full names is calculated as: similarity * number of abbreviations + 1.

[0180] The second similarity between the associated abbreviation and the full name and the number of second full names are calculated based on the knowledge graph. The number of second full names is calculated as follows: similarity of associated abbreviations * number of co-occurrences + 1.

[0181] Secondly, based on the knowledge graph and similar co-occurring abbreviation-full name pairs, the co-occurrence similarity is calculated.

[0182] Specifically, the third similarity of the co-occurrence of the abbreviation and the full name and the number of the third full names are calculated based on the knowledge graph, where the number of the third full names is calculated as: 3*the number of co-occurrences of the abbreviation and the full name + 1.

[0183] The fourth similarity between the associated abbreviation and the full name and the fourth number of full names are calculated based on the knowledge graph. The calculation method for the number of fourth full names is: 3*the number of co-occurrences of the associated abbreviation and full name + 1.

[0184] Furthermore, based on the knowledge graph, the first similarity, the second similarity, the third similarity, the fourth similarity, the number of first full names, the number of second full names, the number of third full names, and the number of fourth full names, a fusion score is calculated, and based on the fusion score, the name set, and similar co-occurring abbreviation-full name pairs, a TOP10ES is constructed as the above-mentioned full name candidate library.

[0185] This embodiment, through the above-mentioned solution, receives a query text; filters the query text to obtain a cluster of abbreviations; and inputs the cluster of abbreviations into a pre-created database for processing to obtain the entity corresponding to the query text. This invention improves average analysis speed, solves performance issues related to company entity association, integrates co-occurrence features and entity links in the product library, and significantly improves the accuracy of associating company abbreviations and full names in the public opinion cloud.

[0186] Figure 6 This is a flow chart of another specific embodiment of the entity linking and association method of the present invention. In this embodiment, during the process of analyzing public opinion, entities in the news, including people and companies, are first identified, and then risks and marketing events are extracted.

[0187] Specifically, we first collected 1.3 million named entity recognition abbreviations from 1 million news articles, used Spark and Hive to analyze and store the 60 million co-occurring alias pairs and the number of co-occurring chapters, and then imported them into ES for easy query.

[0188] Next, we searched the software copyright and trademark database for 1.3 million company names. Any co-occurrences between the company names and the searched names were stored in ES, a knowledge base. Because these names were constructed based not only on news but also on other publicly available corporate data repositories, we considered them the most reliable and prioritized them for subsequent online entity links.

[0189] Secondly, using the business registration database of enterprises, we modified the ranking of recalled corporate entities by their abbreviations, which can be queried through an online interface. The ranking mainly integrates the following two categories and six dimensions, and the similarity is the score of the six dimensions:

[0190] The first category represents the company's own comprehensive strength: the company's registered capital, number of relevant documents (bid success, patents, qualification certificates, etc.).

[0191] The second category represents the correlation between the abbreviation and the company: the similarity between the abbreviation and the full name, the similarity of the main components of the abbreviation and the full name (excluding place names, business formats, types, etc.), a certain penalty if the character order of the abbreviation and the full name is inconsistent, and the ES match score.

[0192] Furthermore, the similarity of 6 million co-occurring alias pairs was calculated to filter out similar co-occurring abbreviation pairs, and then further filtered to obtain similar co-occurring abbreviation-full name pairs. The similarity is the number of co-occurring articles. Similarity includes the following three categories:

[0193] The first type: The edit distance between two abbreviations is equal to the difference in the number of characters in the two abbreviations, which ensures that the characters of the shorter abbreviation appear in the full name in sequence.

[0194] Category 2: The edit distance between the two abbreviations is 1, and the lengths of the two abbreviations are equal. It can be judged that there is only one different character in the two abbreviations, and the different characters are required to have the same pinyin.

[0195] Category 3: The difference in the number of characters between the two abbreviations minus the edit distance is 2. It can be determined that the shorter abbreviation has two characters that are different from the longer abbreviation. The different two characters are required to be the company type (group, company, stock, etc.).

[0196] Furthermore, we use similar co-occurring abbreviation-full name pairs to construct a similar abbreviation knowledge graph of 1.3 million abbreviations, and then calculate the similarity of the abbreviation recall company entity and the abbreviation of similar co-occurring abbreviation-full name pairs separately. After fusing the similarity scores, we get a new abbreviation-full name candidate ranking as the Top 5 result.

[0197] Finally, when searching for abbreviations and full names online, the NER result of a news article is input. First, the abbreviation clusters of the news article are calculated. Second, within the abbreviation clusters, it is assumed that longer abbreviations have less ambiguity. For each abbreviation set, the product database ES and the top five full name candidate database are searched in descending order of length. If no results are found in either search, the online interface is queried within the company entities recalled using the abbreviations.

[0198] This embodiment, through the above-mentioned scheme, receives a query text; filters the query text to obtain abbreviation clusters; and inputs the abbreviation clusters into a pre-created database for processing to obtain the entities corresponding to the query text. This invention identifies the identities of natural persons mentioned in news and returns the companies to which they belong. It also matches corporate entity aliases to full company names based on the news context, thereby improving average analysis speed.

[0199] Reference Figure 7 , Figure 7 Schematic diagram of the functional modules of the entity linking association system of the present invention. The entity linking association system includes:

[0200] A text receiving module 10 is used to receive the text to be queried;

[0201] A text screening module 20 is used to screen the text to be queried to obtain abbreviation clusters;

[0202] The entity query module 30 is used to input the abbreviation clusters into a pre-created database for processing to obtain the entities corresponding to the text to be queried.

[0203] For the principle and implementation process of implementing entity link association in this embodiment, please refer to the above embodiments and will not be repeated here.

[0204] In addition, an embodiment of the present invention also proposes a terminal device, which includes a memory, a processor, and an entity link association program stored on the memory and runnable on the processor. When the entity link association program is executed by the processor, the steps of the entity link association method described above are implemented.

[0205] Since this entity link association program adopts all the technical solutions of all the aforementioned embodiments when executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0206] In addition, an embodiment of the present invention further proposes a computer-readable storage medium, on which an entity link association program is stored. When the entity link association program is executed by a processor, the steps of the entity link association method described above are implemented.

[0207] Since this entity link association program adopts all the technical solutions of all the aforementioned embodiments when executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0208] Compared to existing technologies, the present invention provides an entity linking association method, system, terminal device, and storage medium. These methods receive query text, filter the query text, and generate abbreviation clusters. These abbreviation clusters are then input into a pre-created database for processing to obtain entities corresponding to the query text. This solves the problem of associating abbreviations with full names and improves the accuracy of entity linking.

[0209] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or method that includes a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or method. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, article, or method that includes the element.

[0210] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0211] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present invention.

[0212] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An entity link association method, characterized in that: The method comprises the following steps: Receive the text to be queried; Screening the text to be queried to obtain abbreviation clusters; Input the abbreviation cluster into a pre-created database for processing to obtain entities corresponding to the text to be queried; Before the step of clustering the abbreviations into a pre-created database for processing to obtain the entity corresponding to the text to be queried, the following steps are included: establishing the database; Wherein, the step of establishing the database includes: Get a sample set; Based on the sample set, generate a product library and a knowledge graph; Recalling the name of each sample in the sample set to obtain a name set; Establishing a full name candidate library based on the knowledge graph, the name set, and the sample set; The product library, the name set, and the full name candidate library are used as the database; The step of generating a product library based on the sample set includes: Screening samples in the sample set to obtain co-occurrence samples; generating the product library based on the co-occurrence samples; The step of establishing a full name candidate library based on the knowledge graph, the name set, and the sample set includes: Based on the knowledge graph and the name set, calculating a first similar full name and a first similarity score; Based on the knowledge graph and similar co-occurring short-full name pairs, a second similar full name and a second similarity score are calculated, wherein the similar co-occurring short-full name pairs are obtained by screening the sample set; The full name candidate library is established based on the first similar full name, the first similarity score, the second similar full name, and the second similarity score.

2. The entity link association method according to claim 1, characterized in that: The step of generating a knowledge graph based on the sample set includes: Filtering the sample set to obtain similar co-occurring abbreviation-full name pairs in the sample set; The knowledge graph is generated based on the similar co-occurring abbreviation and full name pairs.

3. The entity link association method according to claim 1, characterized in that: The step of recalling the names of the samples in the sample set to obtain a name set includes: Recall the name of each sample in the sample set, and score the name of each sample in the sample set to obtain an entity score; Processing the entity score to obtain a normalized score; The name set is obtained based on the normalized score and the name of each sample in the sample set.

4. The entity link association method according to claim 1, characterized in that: After the step of establishing a full name candidate library based on the knowledge graph, the name set, and the sample set, the following steps are performed: When the preset time is met, the sample set is updated and the execution step is returned to: generating a product library and a knowledge graph based on the sample set.

5. An entity link association system, characterized in that: include: A text receiving module, used for receiving the text to be queried; A text screening module, used to screen the text to be queried to obtain abbreviation clusters; An entity query module is used to input the abbreviation cluster into a pre-created database for processing to obtain the entity corresponding to the text to be queried; Establishing module, used to establish the database, specifically for: Get a sample set; Based on the sample set, generate a product library and a knowledge graph; Recalling the name of each sample in the sample set to obtain a name set; Establishing a full name candidate library based on the knowledge graph, the name set, and the sample set; The product library, the name set, and the full name candidate library are used as the database; The establishment module is further configured to screen samples in the sample set to obtain co-occurrence samples; and generate the product library based on the co-occurrence samples; The establishing module is further configured to calculate a first similar full name and a first similarity score based on the knowledge graph and the name set; Based on the knowledge graph and similar co-occurring short-full name pairs, a second similar full name and a second similarity score are calculated, wherein the similar co-occurring short-full name pairs are obtained by screening the sample set; The full name candidate library is established based on the first similar full name, the first similarity score, the second similar full name, and the second similarity score.

6. A terminal device, characterized in that: The terminal device includes a memory, a processor, and an entity link association method stored in the memory and executable on the processor. When the entity link association program is executed by the processor, the steps of the entity link association method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an entity link association program, and when the entity link association program is executed by a processor, the steps of the entity link association method according to any one of claims 1 to 4 are implemented.

Images