An entity matching method, system, device and medium based on word frequency

Abstract

The application relates to an entity matching method, system, device and medium based on word frequency, wherein the method comprises the following steps: performing word segmentation on aliases in a plurality of entity data to obtain a first word segmentation list and an alias word set, counting word frequency data of words in the alias word set, removing city words existing in the first word segmentation list to obtain a second word segmentation list, judging whether the words in the second word segmentation list are common words according to the word frequency data, processing the words in the second word segmentation list according to the judgment result, obtaining alias keywords corresponding to the aliases, and querying whether the alias keywords exist in a text corpus; if the alias keywords exist, an entity corresponding to the alias keywords is recognized in the text corpus. Through the application, the problems of missed recognition caused by alias simplification and alias synonym misjudgment are solved. The alias keywords are obtained based on the word frequency of the words in the aliases, entity matching is more accurate, and the accuracy of entity matching is improved.

Description

Technical Field

[0001] This application relates to the field of data recognition, and in particular to a word frequency-based entity matching method, system, device, and medium. Background Technology

[0002] With the continuous development of information technology, artificial intelligence is now widely used in the industry for text information extraction. In text information extraction, based on the possibility of encountering entities such as companies of interest within the corpus, a program is needed to automatically identify the entities appearing in each text to obtain the text containing these entities. A common approach is to manually assign aliases to each entity and then search for these aliases in each text. If an alias appears in a text, it means that the entity corresponding to that alias appears in that text. However, this approach has several drawbacks. Firstly, the aliases may not be concise enough, leading to missed identifications. For example, if an entity is manually assigned the alias "San Kuai Technology," but only "San Kuai" appears in the text, then there's also the problem of misidentifying synonyms. For instance, if an entity is manually assigned the alias "Jin Yi," but "Ji Yi" appears in the text, searching solely based on the alias "Jin Yi" will incorrectly identify the entity.

[0003] Currently, no effective solutions have been proposed to address the issues of insufficiently concise aliases leading to missed identification and misjudgment of alias synonyms in related technologies. Summary of the Invention

[0004] This application provides a word frequency-based entity matching method, system, device, and medium to at least solve the problems of insufficiently concise aliases leading to missed identification and misjudgment of alias synonyms in related technologies.

[0005] In a first aspect, embodiments of this application provide an entity matching method based on word frequency, the method comprising:

[0006] The aliases in several entity data are segmented to obtain a first segmentation list of several aliases, and a set of alias names containing all the first segmentation lists;

[0007] The frequency of words in the aforementioned set of alternative nouns in the training corpus is statistically analyzed to obtain word frequency data;

[0008] Based on the city word list, remove the city words that exist in the first word segmentation list to obtain the second word segmentation list;

[0009] Iterate through the words in the second word segmentation list and determine whether the words are common words based on the word frequency data;

[0010] If yes, then remove the word from the second word segmentation list; otherwise, stop the traversal.

[0011] Based on the traversal results, the words in the second word segmentation list are processed to obtain the alias keywords of the corresponding aliases;

[0012] The system queries the text corpus to determine if an alias keyword exists. If it does, the system identifies the entity corresponding to the alias keyword in the text corpus.

[0013] In some embodiments, the words in the second word segmentation list are processed according to the traversal results to obtain the alias keywords corresponding to the aliases, including:

[0014] If there are words in the second word segmentation list that have not been removed, then the words that have not been removed will be combined to obtain the alias keywords of the corresponding alias;

[0015] If all words in the second word segmentation list are removed, then the last word in the second word segmentation list that was judged as a common word is taken and put back into the second word segmentation list. Based on the first word segmentation list corresponding to the second word segmentation list, the alias keyword of the corresponding alias is obtained.

[0016] In some embodiments, the alias keywords corresponding to the alias are obtained based on the first word segmentation list corresponding to the second word segmentation list, including:

[0017] If the second-to-last word in the second word segmentation list that is judged to be a common word exists, then the word is combined with the words in the second word segmentation list to obtain the alias keyword of the corresponding alias;

[0018] If the second-to-last word in the second word segmentation list does not exist, then the city word in the first word segmentation list corresponding to the second word segmentation list is combined with the word in the second word segmentation list to obtain the alias keyword of the corresponding alias.

[0019] In some embodiments, the frequency of occurrence of words in the alternative noun set in the training corpus is counted to obtain word frequency data, including:

[0020] The frequency of words in the aforementioned set of alternative nouns is obtained by statistically analyzing their occurrence frequency in the training corpus.

[0021] The words in the set of alternative nouns are sorted according to the word frequency to obtain word frequency data, wherein each row in the word frequency data is a word and the word frequency of the word.

[0022] In some embodiments, traversing the words in the second word segmentation list and determining whether a word is a common word based on the word frequency data includes:

[0023] Based on the order of words in the second word segmentation list, traverse the words in the word segmentation list from back to front;

[0024] Set a preset threshold and determine whether the word is a common word based on the word frequency data;

[0025] If the frequency of the word is greater than the preset threshold, then the word is a common word;

[0026] If the frequency of a word is less than the preset threshold, then the word is an uncommon word.

[0027] In some embodiments, aliases in several entity data are segmented to obtain a first segmented list of the aliases, including:

[0028] Based on the logical order of words in the entity data, the aliases are segmented to obtain a first segmented word list, wherein the order of words in the first segmented word list follows the logical order of the words.

[0029] In some embodiments, the method further includes, before segmenting aliases in several entity data, the following:

[0030] Entity data is constructed using a trained entity recognition model or manual annotation methods, wherein the entity data includes entity ID, full entity name, and alias.

[0031] Secondly, embodiments of this application provide a word frequency-based entity matching system, the system including a word segmentation module, a word frequency module, a preprocessing module, a traversal module, a processing module, and a matching module;

[0032] The word segmentation module segments the aliases in several entity data structures to obtain a first word segmentation list of several aliases and a set of alias names of all the aliases;

[0033] The word frequency module counts the frequency of occurrence of words in the noun set in the training corpus, and sorts the words according to the frequency of occurrence to obtain word frequency data;

[0034] The preprocessing module removes city words from the first word segmentation list based on the city word list to obtain the second word segmentation list;

[0035] The traversal module traverses the words in the second word segmentation list and determines whether the word is a common word based on the word frequency data. If it is, the word is removed from the second word segmentation list; otherwise, the traversal stops.

[0036] The processing module processes the words in the second word segmentation list according to the traversal results to obtain the alias keywords of the corresponding aliases;

[0037] The matching module queries the text corpus to see if the alias keyword exists. If it does, the entity corresponding to the alias keyword is identified in the text corpus.

[0038] Thirdly, embodiments of this application provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the word frequency-based entity matching method as described in the first aspect above.

[0039] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the word frequency-based entity matching method as described in the first aspect above.

[0040] Compared to related technologies, the entity matching method, system, device, and medium provided in this application embodiment are based on word frequency. This method involves segmenting aliases in several entity data sets to obtain a first segmentation list and an alias set. The frequency of words in the alias set in the training corpus is statistically analyzed to obtain word frequency data. Based on a city word list, city words in the first segmentation list are removed to obtain a second segmentation list. The word frequency data is used to determine whether words in the second segmentation list are common words. If so, the words are removed from the second segmentation list; otherwise, the traversal stops. The words in the second segmentation list are processed based on the traversal results to obtain alias keywords corresponding to the aliases. The existence of the alias keywords is then checked in the text corpus. If they exist, the entity corresponding to the alias keywords is identified in the text corpus. This solves the problems of insufficiently concise aliases leading to missed identification and misjudgment of alias synonyms. It achieves more accurate entity matching by obtaining alias keywords based on word frequency in aliases, thus improving the accuracy of entity matching. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1 This is a flowchart of the steps in the alias-based entity matching method in related technologies;

[0043] Figure 2 This is a flowchart of the steps of the word frequency-based entity matching method according to an embodiment of this application;

[0044] Figure 3 This is a flowchart illustrating the steps involved in a method for obtaining alias keywords based on word frequency.

[0045] Figure 4 Here is a flowchart of another method for obtaining alias keywords based on word frequency;

[0046] Figure 5 This is a structural block diagram of a word frequency-based entity matching system according to an embodiment of this application;

[0047] Figure 6 This is a flowchart illustrating the steps of a word frequency-based entity matching method according to a specific embodiment of this application;

[0048] Figure 7 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application.

[0049] Figure descriptions: 51. Word segmentation module; 52. Word frequency module; 53. Preprocessing module; 54. Traversal module; 55. Processing module; 56. Matching module. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0051] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0052] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0053] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0054] In related technologies, Figure 1 This is a flowchart based on the steps of alias-based entity matching methods in related technologies, such as... Figure 1 As shown, the method includes the following steps:

[0055] Step S102: Manually assign an alias to each entity to obtain a list of aliases;

[0056] Step S104: Retrieve the target text based on the alias list;

[0057] Step S106: If an alias from the alias list appears in the target text, then the target text contains an entity corresponding to that alias.

[0058] Therefore, simply matching entities from aliases to target text has the following problems:

[0059] The alias is not concise enough. For example, if the alias is "San Kuai Technology", but the target text contains "San Kuai" (without "Technology"), there will be a problem of missed identification.

[0060] Aliases and synonyms. For example, "Jin Yi" is an alias for "Jin Yi Technology", but the text only contains "Further". Simply searching based on the alias "Jin Yi" will incorrectly identify "Jin Yi Technology". In this case, there will be a problem of misidentification of synonyms.

[0061] Therefore, to solve the above problems existing in the related technologies, the present invention provides a method, system, device and medium for entity matching based on word frequency. The following are the term explanations involved in the present invention;

[0062] Corpus: A collection containing multiple texts.

[0063] Word frequency: The number of times a certain word appears in the corpus is called the word frequency of the word.

[0064] Common word: A word with a word frequency higher than the common word threshold. The common word threshold can be adjusted according to the situation and is generally set to 200.

[0065] Uncommon word: A word with a word frequency less than or equal to the common word threshold.

[0066] Entity: An organization such as an enterprise or a non-profit organization.

[0067] Word segmentation: Converting a Chinese text into a list of words separated by delimiters, such as converting "Beijing Yunci Information Technology Co., Ltd." into "Beijing Yunci Information Technology Co., Ltd.".

[0068] The embodiment of the present application provides a method for entity matching based on word frequency. Figure 2 It is a step flow chart of the method for entity matching based on word frequency according to the embodiment of the present application, as Figure 2 shown. The method includes the following steps:

[0069] Step S202: Perform word segmentation on the aliases in several entity data to obtain a first word segmentation list of several aliases and a set of alias words containing all the first word segmentation lists;

[0070] Step S204: Count the occurrence frequencies of the words in the set of alias words in the training corpus to obtain word frequency data;

[0071] Step S206: Remove the city words existing in the first word segmentation list according to the city word list to obtain a second word segmentation list;

[0072] Step S208: Traverse the words in the second word segmentation list, and judge whether the word is a common word according to the word frequency data. If so, remove the word from the second word segmentation list. If not, stop traversing;

[0073] Step S210: Process the words in the second word segmentation list according to the traversal result to obtain the alias keywords corresponding to the aliases;

[0074] Step S212: Query whether the alias keywords exist in the text corpus. If so, identify the entity corresponding to the alias keywords in the text corpus.

[0075] Specifically, Table 1 is a comparison table of alias keywords obtained according to the embodiments of this application and alias settings in related technologies.

[0076] Table 1

[0077]

[0078] As shown in Table 1, some entities, aliases, and alias keywords are listed. Through experiments, it can be concluded that the accuracy of entity matching based on manually set aliases in related technologies is only 85.8%; however, the entity matching accuracy based on the alias keywords obtained by the word frequency-based entity matching method of this application reaches 91.5%.

[0079] Therefore, by using steps S202 to S212 in the embodiments of this application, the problems of insufficient alias leading to missed identification and misjudgment of alias synonyms are solved. The alias keywords are obtained based on the word frequency of words in the alias, and entity matching is performed more accurately, thereby improving the accuracy of entity matching.

[0080] In some of these embodiments, Figure 3 This is a flowchart illustrating the steps involved in obtaining alias keywords based on word frequency, such as... Figure 3 As shown, in step S210, the words in the second word segmentation list are processed according to the traversal results to obtain the alias keywords for the corresponding aliases, including:

[0081] Step S302: If there are words that have not been removed in the second word segmentation list, combine the words that have not been removed to obtain the alias keywords of the corresponding alias;

[0082] Step S304: If all words in the second word segmentation list are removed, take the last word in the second word segmentation list that was judged as a common word and put it back into the second word segmentation list. Based on the first word segmentation list corresponding to the second word segmentation list, obtain the alias keyword of the corresponding alias.

[0083] In some of these embodiments, Figure 4 Here is a flowchart of another method for obtaining alias keywords, such as... Figure 4 As shown, in step S304, based on the first word segmentation list corresponding to the second word segmentation list, the alias keywords for the corresponding aliases are obtained, including:

[0084] Step S402: If the second-to-last word in the second word segmentation list is a common word, then combine the word with the words in the second word segmentation list to obtain the alias keyword of the corresponding alias;

[0085] Step S404: If the second-to-last word in the second word segmentation list that is judged as a common word does not exist, then the city word in the first word segmentation list corresponding to the second word segmentation list is combined with the word in the second word segmentation list to obtain the alias keyword of the corresponding alias.

[0086] In some embodiments, step S204, which involves counting the frequency of words in the alternative noun set in the training corpus to obtain word frequency data, includes:

[0087] The frequency of words in the set of alternative nouns is obtained by counting the occurrence frequency of words in the training corpus.

[0088] The words in the noun set are sorted according to their frequency to obtain word frequency data. Each row in the word frequency data is a word and its frequency.

[0089] In some embodiments, step S208, traversing the words in the second word segmentation list and determining whether a word is a common word based on word frequency data, includes:

[0090] Based on the order of words in the second word segmentation list, traverse the words in the word segmentation list from back to front;

[0091] Set a preset threshold to determine whether a word is a common word based on word frequency data;

[0092] If the frequency of a word is greater than a preset threshold, then the word is a common word;

[0093] If the frequency of a word is less than a preset threshold, then the word is considered an uncommon word.

[0094] In some embodiments, step S202 involves segmenting the aliases in several entity data to obtain a first segmented list of aliases, including:

[0095] Based on the logical order of words in the entity data, the aliases are segmented to obtain the first segmented word list, where the order of words in the first segmented word list follows the logical order of words.

[0096] Specifically, for the alias "Beijing Yunce Information", the logical word order is "Beijing", "Yunce", "Information", therefore its corresponding first word segmentation list is "Beijing Yunce Information".

[0097] It should be noted that the second word segmentation list obtained from the first word segmentation list also follows the logical order of the words.

[0098] In some embodiments, before step S202, which involves word segmentation of aliases in the entity data, the method further includes:

[0099] Entity data is constructed using a trained entity recognition model or manual annotation methods. The entity data includes entity ID, full name, and alias.

[0100] Specific examples are as follows:

[0101] For a certain entity data, the entity ID is "89757"; the full name of the entity is "Beijing Yunce Information Technology Co., Ltd".; the alias is "Beijing Yunce Information".

[0102] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0103] This application provides an entity matching system based on word frequency. Figure 5 This is a structural block diagram of a word frequency-based entity matching system according to an embodiment of this application, such as... Figure 5 As shown, the system includes a word segmentation module 51, a word frequency module 52, a preprocessing module 53, a traversal module 54, a processing module 55, and a matching module 56;

[0104] The word segmentation module 51 segments the aliases in several entity data structures to obtain a first word segmentation list of several aliases and a set of alias names for all aliases;

[0105] The word frequency module 52 counts the frequency of words in the noun set in the training corpus and sorts the words according to their frequency to obtain word frequency data;

[0106] Preprocessing module 53 removes city words from the first word segmentation list based on the city word list to obtain the second word segmentation list;

[0107] The traversal module 54 traverses the words in the second word segmentation list. It determines whether a word is a common word based on word frequency data. If it is, the word is removed from the second word segmentation list; otherwise, the traversal stops.

[0108] The processing module 55 processes the words in the second word segmentation list according to the traversal results to obtain the alias keywords of the corresponding aliases;

[0109] The matching module 56 queries the text corpus to see if the alias keyword exists. If it does, the entity corresponding to the alias keyword is identified in the text corpus.

[0110] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0111] In this embodiment, the word segmentation module 51 segments the aliases in several entity data to obtain a first word segmentation list and an alias set. The word frequency module 52 counts the frequency of words in the alias set in the training corpus to obtain word frequency data. The preprocessing module 53 removes city words from the first word segmentation list based on the city word list to obtain a second word segmentation list. The traversal module 54 determines whether the words in the second word segmentation list are common words based on the word frequency data. If so, the words are removed from the second word segmentation list; otherwise, the traversal stops. The processing module 55 processes the words in the second word segmentation list based on the traversal results to obtain the alias keywords of the corresponding aliases. The matching module 56 queries the text corpus to see if the alias keywords exist. If they exist, the entity corresponding to the alias keywords is identified in the text corpus. This solves the problems of insufficient conciseness of aliases leading to missed identification and misjudgment of alias synonyms. It realizes the acquisition of alias keywords based on the word frequency of words in aliases, enabling more accurate entity matching and improving the accuracy of entity matching.

[0112] This application provides a specific embodiment of an entity matching method based on word frequency. Figure 6 This is a flowchart illustrating the steps of a word frequency-based entity matching method according to a specific embodiment of this application, as follows: Figure 6 As shown, the method includes the following steps:

[0113] Step S602: Segment the aliases.

[0114] The aliases are segmented into words to obtain a list of segmented words for each alias, as well as a set of all alias words.

[0115] Step S604: Count the word frequency.

[0116] On all the corpora, the word frequency of each word in the set of alternative names is counted, and the words are sorted from highest to lowest frequency and output to a word frequency file. Each line in the file contains a word and its frequency. If the frequency of an alternative name is greater than the common word threshold, then the alternative name is a common word; if the frequency is less than the common word threshold, then it is an uncommon word.

[0117] Step S606: Remove city words from the word segmentation list.

[0118] For each alias's word segmentation list, remove the city-related words.

[0119] Specifically, the list of city terms could be: Beijing, Shanghai, Guangzhou, Shenzhen, Tianjin, Chengdu, Hangzhou, Suzhou, Chongqing, Wuhan, Nanjing, Dalian, Shenyang, Changsha, Zhengzhou, Xi'an, Qingdao, Wuxi, Jinan, Ningbo, Foshan, Nantong, Harbin, Dongguan, Fuzhou, Changchun, Shijiazhuang, Yantai, Hefei, Tangshan, Changzhou, Taiyuan, Kunming, Weifang, Nanchang, Quanzhou, Wenzhou, Shaoxing, Jiaxing, Xiamen, Guiyang, Zibo, Xuzhou, Nanning, Yangzhou, Hohhot, Ordos, Urumqi, Jinhua, Taizhou, Zhenjiang, Weihai, Zhuhai, Dongying, Daqing, Zhongshan, Yancheng, Baotou, Baoding, Jining, Taizhou, Langfang, Lanzhou, Luoyang, Yichang, Cangzhou, Linyi, Tai'an, Anshan, Handan, Huizhou, Jiangmen, Xiangyang, Huzhou, Jilin, Wuhu, Dezhou, Liaocheng, Zhangzhou, Zhuzhou, Huai'an, Yulin, Changde, Xianyang, Hengyang, Binzhou, Liuzhou, Zunyi, Heze, Nanyang, Xinxiang, Zhanjiang, Yueyang, Chenzhou, Xuchang, Lianyungang, Zaozhuang, Maoming, Zhoukou, Suqian.

[0120] Step S608: Iterate through the words in the word segmentation list.

[0121] For each alias's word segmentation list, iterate through the list elements from back to front, removing common words when they are encountered and stopping when uncommon words are encountered.

[0122] Step S610: Determine alias keywords based on word frequency.

[0123] If there are words in the word segmentation list of the alias that have not been removed, then the combination of the words that have not been removed will be the alias keywords corresponding to the alias.

[0124] If all words in the word segmentation list of the alias are removed, the first word in the list is added to the list, and then the word after the first word in the word segmentation list is added; if there are no words after the first word in the word segmentation list, the original city name that has been removed from the alias is added, and the final result is the alias keyword corresponding to each alias.

[0125] Step S612: Perform entity matching based on alias keywords.

[0126] For each text, query whether each alias keyword exists in the text. If it exists, it means that the corresponding entity has been identified in the text.

[0127] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0128] Steps S602 to S612 in the specific embodiments of this application solve the problems of insufficient alias leading to missed identification and misjudgment of alias synonyms. They enable the generation of alias keywords based on word frequency in aliases, allowing for more accurate entity matching and improving the accuracy of entity matching.

[0129] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0130] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0131] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0132] Furthermore, in conjunction with the word frequency-based entity matching method in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the word frequency-based entity matching methods in the above embodiments.

[0133] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a word frequency-based entity matching method. The display screen of the computer device may be a liquid crystal display (LCD) or an e-ink display. The input device of the computer device may be a touch layer covering the display screen, or buttons, a trackball, or a touchpad located on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0134] In one embodiment, Figure 7 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 7 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 7As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores an operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network connection, the internal memory provides an environment for the operation of the operating system and computer programs, the computer programs are executed by the processor to implement a word frequency-based entity matching method, and the database stores data.

[0135] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0136] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0137] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A word frequency-based entity matching method, characterized in that, The method includes: The aliases in several entity data are segmented to obtain a first segmentation list of several aliases, and a set of alias names containing all the first segmentation lists; The frequency of words in the aforementioned set of alternative nouns in the training corpus is statistically analyzed to obtain word frequency data; Based on the city word list, remove the city words that exist in the first word segmentation list to obtain the second word segmentation list; Iterate through the words in the second word segmentation list and determine whether the words are common words based on the word frequency data; If yes, then remove the word from the second word segmentation list; otherwise, stop the traversal. The words in the second word segmentation list are processed according to the traversal results. If there are words in the second word segmentation list that have not been removed, the words that have not been removed are combined to obtain the alias keywords of the corresponding aliases. If all words in the second word segmentation list are removed, then the last word in the second word segmentation list that was judged as a common word is taken and put back into the second word segmentation list; Based on the first word segmentation list corresponding to the second word segmentation list, the alias keywords for the corresponding alias are obtained as follows: if the second-to-last word in the second word segmentation list that is judged to be a common word exists, then the word is combined with the words in the second word segmentation list to obtain the alias keywords for the corresponding alias; if the second-to-last word in the second word segmentation list that is judged to be a common word does not exist, then the city word in the first word segmentation list corresponding to the second word segmentation list is combined with the words in the second word segmentation list to obtain the alias keywords for the corresponding alias. The system queries the text corpus to determine if an alias keyword exists. If it does, the system identifies the entity corresponding to the alias keyword in the text corpus.

2. The method according to claim 1, characterized in that, The frequency of words in the aforementioned set of alternative nouns in the training corpus was statistically analyzed to obtain word frequency data, including: The frequency of words in the aforementioned set of alternative nouns is obtained by statistically analyzing their occurrence frequency in the training corpus. The words in the set of alternative nouns are sorted according to the word frequency to obtain word frequency data, wherein each row in the word frequency data is a word and the word frequency of the word.

3. The method according to claim 1, characterized in that, Traversing the words in the second word segmentation list and determining whether a word is a common word based on the word frequency data includes: Based on the order of words in the second word segmentation list, traverse the words in the word segmentation list from back to front; Set a preset threshold and determine whether the word is a common word based on the word frequency data; If the frequency of the word is greater than the preset threshold, then the word is a common word; If the frequency of a word is less than the preset threshold, then the word is an uncommon word.

4. The method according to claim 1, characterized in that, The aliases in several entity data are segmented into words, and the first word segmentation list of the aliases is obtained, including: Based on the logical order of words in the entity data, the aliases are segmented to obtain a first segmented word list, wherein the order of words in the first segmented word list follows the logical order of the words.

5. The method according to claim 1, characterized in that, Before segmenting aliases in several entity data, the method further includes: Entity data is constructed using a trained entity recognition model or manual annotation methods, wherein the entity data includes entity ID, full entity name, and alias.

6. A word frequency-based entity matching system, characterized in that, The system includes a word segmentation module, a word frequency module, a preprocessing module, a traversal module, a processing module, and a matching module; The word segmentation module segments the aliases in several entity data structures to obtain a first word segmentation list of several aliases and a set of alias names of all the aliases; The word frequency module counts the frequency of occurrence of words in the noun set in the training corpus, and sorts the words according to the frequency of occurrence to obtain word frequency data; The preprocessing module removes city words from the first word segmentation list based on the city word list to obtain the second word segmentation list; The traversal module traverses the words in the second word segmentation list and determines whether the word is a common word based on the word frequency data. If it is, the word is removed from the second word segmentation list; otherwise, the traversal stops. The processing module processes the words in the second word segmentation list according to the traversal results. If there are words in the second word segmentation list that have not been removed, the words that have not been removed are combined to obtain the alias keywords of the corresponding alias. If all words in the second word segmentation list are removed, the last word judged as a common word in the second word segmentation list is taken and put back into the second word segmentation list. Based on the first word segmentation list corresponding to the second word segmentation list, the alias keywords for the corresponding alias are obtained as follows: if the second-to-last word judged as a common word in the second word segmentation list exists, the word is combined with the words in the second word segmentation list to obtain the alias keywords for the corresponding alias; if the second-to-last word judged as a common word in the second word segmentation list does not exist, the city word in the first word segmentation list corresponding to the second word segmentation list is combined with the words in the second word segmentation list to obtain the alias keywords for the corresponding alias. The matching module queries the text corpus to see if the alias keyword exists. If it does, the entity corresponding to the alias keyword is identified in the text corpus.

7. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the word frequency-based entity matching method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a computer program that, when executed, implements the word frequency-based entity matching method as described in any one of claims 1 to 5.

Images