Entity nickname generation method and apparatus, computer device, and storage medium
By using corpus-based word segmentation and frequency gain calculation, and employing greedy and Apriori algorithms to generate entity abbreviations, this solves the problem of high effort and cost in generating entity abbreviations in existing technologies, and achieves efficient and accurate automated generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2021-11-08
- Publication Date
- 2026-04-10
AI Technical Summary
In existing technologies, generating entity abbreviations requires a significant amount of effort and cost.
By segmenting the full name of the target entity based on a pre-set corpus, calculating the frequency gain of the constituent words, and using a greedy algorithm and the Apriori algorithm to generate entity abbreviations, the entity names are automatically processed to generate accurate abbreviations.
It enables the automatic and simple generation of entity abbreviations, improving generation efficiency and accuracy while reducing the cost of manual processing.
Smart Images

Figure CN116090448B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to the technical field of natural language processing, and in particular to an entity abbreviation generation method and device, computer equipment and a storage medium. BACKGROUND
[0002] Data intelligent report is a visual presentation of data intelligent analysis, and visualizes the result of data mining. In a traditional report drawing scheme, relevant personnel need to make reports and analyze on the basis of certain business scenarios, which requires a large amount of time and effort. Especially in various statistical chart displays, abbreviations or short names of original entity names are often used. Moreover, short names are used in many scenarios in the expression of enterprise names, government agency names and social group names. With the access of various industries to big data applications, the number of entity classes has increased rapidly, and it takes a lot of effort and cost to generate reasonable entity short names manually. SUMMARY
[0003] In view of the above problems, embodiments of the present application provide an entity abbreviation generation method, device, computer equipment and storage medium, to solve the problem that generating entity abbreviations requires a lot of effort and cost in the prior art.
[0004] According to an aspect of an embodiment of the present application, an entity abbreviation generation method is provided, which comprises:
[0005] performing word segmentation on a target entity full name based on a pre-set corpus to obtain each component word constituting the target entity full name; wherein the corpus comprises at least one word and the number of occurrences corresponding to each word;
[0006] According to the number of occurrences corresponding to each word, calculate the frequency gain corresponding to each component word according to a preset rule;
[0007] determine the target entity abbreviation according to the frequency gain corresponding to each component word.
[0008] In an optional manner, before the performing word segmentation on a target entity full name based on a pre-set corpus to obtain each component word constituting the target entity full name, further comprising:
[0009] combining each two entity full names in the set entity library respectively to obtain at least one pair of entity full name combinations;
[0010] acquiring a common continuous character subset and a unique continuous character subset in each of the entity full name combinations, wherein the common continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is a difference set of each entity full name in the entity full name combination and the common continuous character subset;
[0011] performing correlation analysis on each of the common continuous character subsets to obtain a support degree of each of the common continuous character subsets;
[0012] establishing a word segmentation library based on the common continuous character subsets with a support degree greater than a preset support degree threshold and each of the unique continuous character subsets;
[0013] segmenting entity full names in the entity library based on the word segmentation library, and generating a corpus based on a segmentation result, wherein the corpus includes at least one word and an occurrence frequency of each of the words.
[0014] In an optional manner, the performing correlation analysis on each of the common continuous character subsets to obtain a support degree of each of the common continuous character subsets further includes:
[0015] performing correlation analysis on each of the common continuous character subsets based on an Apriori algorithm to obtain a support degree of each of the common continuous character subsets.
[0016] In an optional manner, the segmenting entity full names in the entity library based on the word segmentation library, and generating a corpus based on a segmentation result further includes:
[0017] segmenting each of the entity full names in the entity library based on the word segmentation library to obtain each word constituting each of the entity full names;
[0018] establishing a corpus based on each word constituting each of the entity full names.
[0019] In an optional manner, the calculating a frequency gain of each of the constituent words according to the occurrence frequency of each of the words according to a preset rule further includes:
[0020] determining an occurrence frequency of each of the constituent words according to the occurrence frequency of each of the words;
[0021] eliminating each of the constituent words one by one, and calculating a word occurrence frequency of an entity constituted by the constituent words remaining after elimination and an occurrence frequency of each of the constituent words according to the occurrence frequency of each of the constituent words, respectively;
[0022] calculating a frequency gain of each of the constituent words based on a greedy algorithm according to the word occurrence frequency.
[0023] In an alternative mode, after the appearance frequency of each of the component words is obtained according to the corresponding appearance times of each of the component words, the method further comprises:
[0024] The appearance frequency of each of the component words is logarithmically processed.
[0025] In an alternative mode, the target entity nickname is determined according to the frequency gain of each of the component words, further comprising:
[0026] Each of the component words is sorted in ascending order of the frequency gain of each of the component words based on the frequency gain of each of the component words.
[0027] Each of the component words is combined and removed in ascending order of the frequency gain, and the word appearance frequency and of the entity composed of the component words remaining after the combination and removal is calculated according to the appearance times of each of the component words.
[0028] The frequency gain of each of the component words is calculated based on a greedy algorithm according to the word appearance frequency and.
[0029] The target entity nickname is constructed based on the component word combination whose frequency gain is less than a preset frequency gain threshold.
[0030] According to another aspect of an embodiment of the present application, an entity nickname generation device is provided, comprising:
[0031] A word segmentation module is configured to segment a target entity full name based on a pre-set corpus to obtain each component word constituting the target entity full name; wherein the corpus comprises at least one word and the corresponding appearance times of each of the words.
[0032] A calculation module is configured to calculate the frequency gain of each of the component words according to the corresponding appearance times of each of the words according to a preset rule.
[0033] A processing module is configured to determine the target entity nickname according to the frequency gain of each of the component words.
[0034] In an alternative mode, the entity nickname generation device further comprises a combination module, an extraction module, an analysis module and a construction module; wherein,
[0035] The combination module is configured to combine each of the two entity full names in the set entity library to obtain at least one pair of entity full name combinations.
[0036] The extraction module is configured to traverse each pair of the entity full name combination, and obtain a common continuous character subset and a unique continuous character subset in each pair of the entity full name combination; the common continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is a difference set of each entity full name in the entity full name combination and the common continuous character subset;
[0037] The analysis module is configured to perform correlation analysis on each common continuous character subset to obtain a support degree of each common continuous character subset.
[0038] The establishment module is configured to establish a word segmentation library based on the common continuous character subset with a support degree greater than a preset support degree threshold and each unique continuous character subset.
[0039] The word segmentation module is further configured to perform word segmentation on the entity full name in the entity library based on the word segmentation library, and generate a corpus based on a word segmentation result; the corpus includes at least one word and an occurrence frequency corresponding to each word.
[0040] According to another aspect of an embodiment of the present application, a computer device is provided, which includes a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete communication with each other through the communication bus;
[0041] The memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform the following operations:
[0042] The target entity full name is segmented based on a preset corpus to obtain each component word of the target entity full name; the corpus includes at least one word and an occurrence frequency corresponding to each word.
[0043] According to the occurrence frequency corresponding to each word, a frequency gain corresponding to each component word is calculated according to a preset rule.
[0044] The target entity abbreviation is determined according to the frequency gain corresponding to each component word.
[0045] In an optional manner, the executable instruction causes the processor to perform the following operations:
[0046] Each two entity full names in the set entity library are combined respectively to obtain at least one pair of entity full name combinations.
[0047] Iterate through each pair of entity full name combinations to obtain a common continuous character subset and a unique continuous character subset in each pair of entity full name combinations; wherein, the common continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is the difference between each entity full name in the entity full name combination and the common continuous character subset;
[0048] A correlation analysis is performed on each of the aforementioned common continuous character subsets to obtain the support of each of the aforementioned common continuous character subsets;
[0049] A word segmentation library is established based on the common continuous character subsets whose support is greater than a preset support threshold and each of the unique continuous character subsets;
[0050] The full names of entities in the entity database are segmented based on the word segmentation library, and a corpus is generated based on the segmentation results; wherein, the corpus includes at least one word and the number of occurrences of each word.
[0051] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing at least one executable instruction that causes a computer device / apparatus to perform the following operations:
[0052] The full name of the target entity is segmented based on a pre-set corpus to obtain the constituent words that make up the full name of the target entity; wherein, the corpus includes at least one word and the number of occurrences of each word;
[0053] Based on the frequency of occurrence of each of the aforementioned words, the frequency gain of each of the constituent words is calculated according to a preset rule.
[0054] The abbreviation of the target entity is determined based on the frequency gain corresponding to each of the constituent words.
[0055] In an alternative approach, the executable instructions cause the computer device / apparatus to perform the following operations:
[0056] Combine each pair of full names of entities in the set entity library to obtain at least one pair of full name combinations;
[0057] Iterate through each pair of entity full name combinations to obtain a common continuous character subset and a unique continuous character subset in each pair of entity full name combinations; wherein, the common continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is the difference between each entity full name in the entity full name combination and the common continuous character subset;
[0058] A correlation analysis is performed on each of the aforementioned common continuous character subsets to obtain the support of each of the aforementioned common continuous character subsets;
[0059] establish a word segmentation library based on the common continuous character subset and each of the unique continuous character subset whose support is greater than a preset support threshold;
[0060] segment the full names of entities in the entity library based on the word segmentation library, and generate a corpus according to a segmentation result; wherein the corpus includes at least one word and an occurrence frequency corresponding to each of the word.
[0061] The embodiment of the present application quantifies the key degree of a word by the occurrence frequency of the word, calculates the frequency gain corresponding to the word based on the occurrence frequency of the word, and then retains the key word in the original entity name according to the frequency gain corresponding to the word to generate an entity nickname, so that the entity nickname can be automatically and simply generated with high accuracy.
[0062] The above description is only a summary of the technical scheme of the embodiment of the present application, in order to more clearly understand the technical means of the embodiment of the present application, and can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation of the present application is described below. BRIEF DESCRIPTION OF DRAWINGS
[0063] The accompanying drawings are included to provide a further understanding of the application and are incorporated herein and constitute a part of the detailed description. The drawings illustrate embodiments of the application and, together with the description, serve to explain the principles of the application. In the drawings:
[0064] Figure 1 A flowchart of an entity nickname generation method provided by an embodiment of the present application is shown;
[0065] Figure 2 A flowchart of an entity nickname generation method provided by another embodiment of the present application is shown;
[0066] Figure 3 A distribution diagram of the frequency of word usage in an embodiment of the present application is shown;
[0067] Figure 4 A process diagram of an entity nickname generation method in an embodiment of the present application is shown;
[0068] Figure 5 A structure diagram of an entity nickname generation device provided by an embodiment of the present application is shown Figure 1 ;
[0069] Figure 6 A structure diagram of an entity nickname generation device provided by an embodiment of the present application is shown Figure 2 ;
[0070] Figure 7 A structure diagram of a computer device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION
[0071] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein.
[0072] Figure 1 A flowchart of an embodiment of a method for generating an entity abbreviation according to the present application is shown, which is executed by a computer device. The computer device refers to a device with computing capability, including but not limited to terminal devices (such as mobile phones, tablet computers, etc.), wearable smart devices (such as smart watches, smart bracelets, smart earphones, etc.), smart home devices (such as smart televisions, smart speakers, etc.), Internet of Vehicles devices (such as smart cars, vehicle-mounted terminals, etc.), servers, etc. As shown, the method includes the following steps: Figure 1
[0073] Step S110: performing word segmentation on the target entity full name based on a pre-set corpus to obtain each component word constituting the target entity full name; wherein the corpus includes at least one word and the number of occurrences of each word.
[0074] The target entity full name is the entity full name for which an abbreviation is needed, and the entity includes but is not limited to enterprises, government agencies, social groups, etc. The corpus is established based on the entity full names in the set entity library after word segmentation and other processing, and includes at least one word and the number of occurrences of each word.
[0075] In an embodiment, before the step of performing word segmentation on the target entity full name based on a pre-set corpus to obtain each component word constituting the target entity full name, the method further includes:
[0076] combining each two entity full names in the set entity library to obtain at least one pair of entity full name combinations;
[0077] traversing each pair of entity full name combinations to obtain a common continuous character subset and a unique continuous character subset in each pair of entity full name combinations; wherein the common continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is the difference set of each entity full name in the entity full name combination and the common continuous character subset;
[0078] performing relevance analysis on each common continuous character subset to obtain the support degree of each common continuous character subset;
[0079] establishing a word segmentation library based on the common continuous character subsets with a support degree greater than a preset support degree threshold and each unique continuous character subset;
[0080] The full name of the entity in the entity library is segmented based on the segmentation library, and a corpus is generated according to a segmentation result; wherein the corpus includes at least one word and a corresponding occurrence frequency of the word.
[0081] The combination of each two entity names in the entity library is to combine any two entity names in the entity library into a pair of entity name combinations. For example, if the entity library contains four entity names, six pairs of entity name combinations are obtained after combining any two entity names into a pair of entity name combinations. For each pair of entity name combinations, a string simultaneously located in each entity name of the pair of entity name combinations and having a length greater than a preset length threshold is extracted, and the string is taken as a common continuous character subset in the pair of entity name combinations. Meanwhile, each entity name of the pair of entity name combinations is respectively subjected to a difference set operation with the common continuous character subset in the pair of entity name combinations, a character subset belonging to a single entity name of the pair of entity name combinations is obtained, and the character subset is taken as a unique continuous character subset in the pair of entity name combinations. The length threshold can be set according to actual needs, for example, 1 or 2 characters. Taking the pair of entity name combinations including China Mobile Beijing Branch and China Mobile Shanghai Branch as an example, the common continuous character subset in the pair of entity name combinations is “China Mobile” and “Branch”, and the unique continuous character subset in the pair of entity name combinations is “Beijing” and “Shanghai” after character extraction in the above manner. Then, the Apriori algorithm can be used to analyze the association of each common continuous character subset, and the support degree of each common continuous character subset is obtained. The higher the support degree of the common continuous character subset, the higher the probability of the occurrence of the common continuous character subset, which can be used as a standard for word segmentation. Then, a word segmentation library can be established based on the common continuous character subset with a support degree greater than a preset support degree threshold and each unique continuous character subset. Specifically, the common continuous character subset with a high probability of occurrence and each unique continuous character subset in the pair of entity name combinations are added to the word segmentation library, so that the word segmentation library is used as a standard for word segmentation. The support degree threshold can be set according to actual needs, which is not limited here. The entity names in the entity library are segmented based on the word segmentation library, and a corpus is generated based on the segmentation result, which further includes: segmenting each entity name in the entity library based on the word segmentation library to obtain each word constituting each entity name; and establishing a corpus based on each word constituting each entity name. It should be noted that the segmentation result at least includes the words constituting each entity name. After segmenting each entity name in the entity library based on the word segmentation library, the words constituting each entity name are obtained, and the number of trips corresponding to each word is obtained based on the number of occurrences of each word in different entity names, that is, the sum of the number of occurrences of each word in different entity names is determined as the number of trips corresponding to the word.Thus, by using the word segmentation method based on the public continuous subset, the professional terms can be avoided from being split, and without relying on the public corpus, the speed of word segmentation and the rationality of the word segmentation result can be greatly improved.
[0082] Step S120: According to the occurrence frequency of each word, the frequency gain of each component word is calculated according to a preset rule.
[0083] According to the occurrence frequency of each word, the frequency gain of each component word is calculated according to a preset rule.
[0084] According to the occurrence frequency of each word, the occurrence frequency of each component word is determined.
[0085] According to the occurrence frequency of each word, the occurrence frequency of each component word is determined.
[0086] According to the occurrence frequency of each word, the occurrence frequency of each component word is determined.
[0087] Here, since the corpus includes the occurrence times of each of the terms, after the full name of the target entity is segmented based on the corpus, the occurrence times of each of the component terms constituting the full name of the target entity can be obtained accordingly. When each of the component terms is not removed one by one, according to the occurrence times of each of the component terms constituting the full name of the target entity and the occurrence times of each of the terms, the occurrence frequencies of each of the component terms at this time can be calculated, that is, for each component term, the occurrence times of the component term are divided by the sum of the occurrence times of each of the terms, and then the word frequency sum of the full name of the target entity at this time, that is, the sum of the occurrence frequencies of each of the component terms, can be obtained. When each of the component terms is removed one by one, according to the occurrence times of each of the component terms constituting the full name of the target entity and the occurrence times of each of the terms, the occurrence frequencies of each of the component terms at this time can also be calculated, that is, for each remaining component term, the occurrence times of the component term are divided by the target times, and then the word frequency sum of the entity constituted by the remaining component terms after removal at this time, that is, the sum of the occurrence frequencies of each of the remaining component terms, can be obtained. Here, after each of the component terms is removed one by one, according to the word frequency sum of the entity constituted by the remaining component terms after removal and the word frequency sum of the full name of the target entity, the frequency gain of each of the component terms can be calculated based on the greedy algorithm. In this way, the key degree of a term is quantified by the occurrence times of the term, and the frequency gain of the term is calculated based on the occurrence times of the term, which is fast and reasonable.
[0088] In an embodiment, after the occurrence frequencies of each of the component terms are obtained according to the occurrence times of each of the terms, the method further includes: performing logarithmic processing on the occurrence frequencies of each of the component terms. It can be understood that due to the characteristics of terms in daily use, the frequency of use of terms usually presents a power-law distribution, that is, a small part of terms often appear in daily use, and the use frequency of most terms is very low. In order to reduce the influence of the power-law distribution characteristics on the calculation, the occurrence frequencies of each of the component terms need to be logarithmically processed.
[0089] Step S130: determining the short name of the target entity according to the frequency gain of each of the component terms.
[0090] Specifically, based on the frequency gain corresponding to each of the constituent words, each of the constituent words is sorted in ascending order of the frequency gain; each of the constituent words is combined and removed in ascending order of the frequency gain, and a word frequency sum corresponding to an entity formed by the constituent words remaining after the combination and removal is calculated according to the number of occurrences corresponding to each of the constituent words; based on the word frequency sum, the frequency gain of each combination of the constituent words is calculated based on a greedy algorithm; and the target entity abbreviation is constructed based on the combination of the constituent words with the frequency gain less than a preset frequency gain threshold.
[0091] Here, based on the frequency gain corresponding to each of the constituent words, each of the constituent words can be sorted in ascending order of the frequency gain, that is, the constituent words with smaller frequency gain are arranged in front, and the constituent words with larger frequency gain are arranged in back. The combination and removal of each of the constituent words in ascending order of the frequency gain can be that the combination and removal of the constituent words with smaller frequency gain is performed first, such as the combination of two constituent words or the combination of three constituent words, etc. Then, when the combination and removal of each of the constituent words is performed, the occurrence frequency corresponding to each of the constituent words at this time can also be calculated according to the number of occurrences corresponding to each of the constituent words forming the full name of the target entity and the number of occurrences corresponding to each of the words, that is, for each of the remaining constituent words, the number of occurrences corresponding to the constituent word is divided by the target number, and the target number is the sum of the number of occurrences corresponding to each of the words minus the number of occurrences corresponding to the constituent word currently combined and removed, and thus the word frequency sum corresponding to the entity formed by the constituent words remaining after the combination and removal at this time can be obtained, that is, the sum of the occurrence frequencies corresponding to each of the remaining constituent words. Here, after the combination and removal of each of the constituent words, based on the word frequency sum corresponding to the entity formed by the constituent words remaining after the combination and removal and the word frequency sum corresponding to the full name of the target entity, the frequency gain of each combination of the constituent words can be calculated based on a greedy algorithm.
[0092] In summary, the entity abbreviation generation method provided in the above embodiments quantifies the key degree of a word based on the number of occurrences of the word, calculates the frequency gain corresponding to the word based on the number of occurrences of the word, and then retains the key word in the original entity name according to the frequency gain corresponding to the word to generate an entity abbreviation, thereby automatically and simply generating an entity abbreviation with high accuracy.
[0093] Based on the same inventive concept as the foregoing embodiments, the foregoing embodiments are specifically described below through specific examples, and in this embodiment, the frequency gain is referred to as a probability gain.
[0094] In each component word of the entity full name, there are two types of words, key words and non-key words, the key words usually contain the core of the entity full name, and the content composed of the key words can replace the entity full name to a certain extent, so that the entity full name can be abbreviated to obtain the entity abbreviation. Generally, the repeatedly appearing words are well known and belong to common words. However, some words have low frequency but express key meanings and belong to specific words. Based on the above idea, the present application proposes an entity abbreviation generation method based on a probability gain model, that is, the probability gain is calculated after the entity name is segmented, and the optimal entity abbreviation is obtained by maximizing the probability gain. In the text preprocessing part of the traditional key word extraction scheme, the traditional segmentation technology is used, and it is difficult to control the fine granularity of segmentation, for example, “Beijing University” will be segmented into “Beijing” and “University”, and the existing solution is to specially process the common special names. Since most of the entity names of institutions are special names and are strongly related to a specific field, similar to “China Mobile Suzhou Cloud Capability Center”, the traditional text preprocessing will most likely segment “Cloud Capability Center” into “Cloud Capability” and “Center”. Therefore, a threshold needs to be set in the data preprocessing process to prevent the splitting of professional terms. When performing the segmentation operation, compared with the traditional segmentation scenario, the existing technology mainly uses the hidden Markov segmentation method based on a large amount of text set, while the present application is aimed at segmenting the institution entity name library. Since the text set is mainly composed of professional terms, it has certain speciality, and the present application further proposes a segmentation scheme based on continuous public subsets. This segmentation scheme can greatly improve the speed of segmentation training and the rationality of the segmentation result without relying on a public corpus. Figure 2 A flowchart of an embodiment of the entity abbreviation generation method of the present application is shown in FIG. 1, which comprises the following steps: Figure 2
[0095] Step S210: obtaining an entity name and segmenting the entity name;
[0096] Step S220: initializing an entity word corpus and counting the frequency of words;
[0097] Step S230: traversing each segmented word of the target entity name;
[0098] Step S240: calculating the frequency gain of each segmented word and updating the entity word corpus;
[0099] Step S250: selecting the maximum frequency gain and reassembling the words to obtain the entity abbreviation.
[0100] The specific process is as follows:
[0101] Step 1, obtaining all entity full names in the entity table in the database to obtain an entity full name set C={c1, c2,…, c n}.
[0102] Step2、Entity name text data preprocessing;
[0103] Step2.1、For each entity full name c i , perform word segmentation to obtain the corresponding entity component words Since most of the entity names are proper nouns, and are strongly related to a specific field, we need to set a threshold to filter out the splitting of professional terms during data preprocessing. Specifically, for the entity name set {c1, c2, …, c n}, combine the entity names two by two {(c1, c2), …, (c n-1 , c n )}, the number of combinations is Iterate through the two-by-two combination of entity names {(c1, c2), …, (c n-1 , c n )}, and calculate the common continuous subsets with a string length greater than 1 in the two-by-two combination. For example, there are where k is the number of common continuous subsets with a string length greater than 1, and represents the common continuous subset operation of the two entity name strings.
[0104] Through set difference operation, we get the unique continuous subsets of the two entity names and where u and t are the number of unique continuous subsets of c1 and c2, respectively, and represents the set difference operation of the common continuous subsets of the entity name strings c1 and .
[0105] In the two-by-two entity full name set , use the Apriori algorithm to analyze the association of the common continuous subsets in all combinations, and calculate the support of each common continuous subset where the support of the common continuous subset is calculated as follows The higher the support of the common continuous subset, the higher the probability that this part of the subset will appear simultaneously, and it can be used as a standard for word segmentation. For a small number of accidental common continuous subsets, this scheme does not use them as the standard for word segmentation. Select the support greater than a certain threshold ξ>σ to construct the word segmentation library containing all the common continuous subsets and unique continuous subsets of the entity names Iterate through the entity full name set C={c1, c2, …, c n}, and use the word segmentation library subset as the standard to segment the entity full name string to obtain the entity component words where and sc 1 i ∈c1.
[0106] Step 2.2: Construct an entity corpus S = {s1, s2, ..., s} m},in
[0107] Step 2.3: Calculate the entity name set {c1, c2, ..., c n The frequency of words appearing in the} category.
[0108] Count the occurrence frequency of entity words And calculate the frequency p of each entity word. i ,in And p i ∈(0,1). Due to the characteristics of words in daily use, the frequency of word usage usually follows a power-law distribution, such as... Figure 3 As shown, a small number of words appear frequently in daily use, while most words are used very infrequently.
[0109] To reduce the impact of the power-law distribution characteristics on the calculation, this embodiment uses the word occurrence frequency p i Logarithmic transformation is performed, as shown in the formula below:
[0110]
[0111] Obtain the word frequency set
[0112] Step 3: Based on the word frequency set P from Step 2 * Calculate the word occurrence probability of the target entity name and h. x , where h x The calculation formula is as follows:
[0113]
[0114] Step 4: Based on the segmented set of the target entity name. Perform the following operations.
[0115] Step 4.1: Analyze the constituent words of the entity name one by one. Remove from the word segmentation set of the target entity name, and repeat steps Step 2.2 and Step 2.3.
[0116] Step 4.2: Calculate the constituent words of the entity names to be removed. The probability of subsequent words and On the one hand, the word probability is reduced due to the deletion of the component word, and on the other hand, the appearance frequency of the entity word is reduced due to the deletion of the component word with high appearance frequency Then The numerical value increases, and the promotion is large.
[0117] Step 4.3, combination of multiple redundant words. In the calculation of the single word probability gain, the implementation mode of traversing and removing the entity name is adopted, but in the combination of removing the entity name part, due to the reason of calculation complexity, the embodiment adopts the method of combining and optimizing the removal of the entity name based on the greedy algorithm to calculate the entity component word The probability gain is calculated as follows: Wherein, h i Indicates The word probability of the component word. The probability gain is sorted to obtain the probability gain set ΔH, and is stored.
[0118] Step 4.4, the component words in the probability gain set ΔH are combined from small to large, and the entity component words whose probability gain is less than a certain threshold ε are reserved to obtain the component word set C * ={c1, c2, …, c n}-{c α , …, c γ}.
[0119] Step 5, entity abbreviation reconstruction. According to the component word set C * obtained in Step 4, the entity abbreviation is combined and reconstructed, and is output. Repeat the above steps to generate the abbreviation of all entity names. Figure 4 The process diagram of the entity abbreviation generation method embodiment of the application is shown in FIG. 1. Figure 4 As shown in FIG. 1, the target entity name "China Mobile Suzhou Cloud Capability Center" is processed by the above-mentioned entity abbreviation generation method, and the corresponding entity abbreviation "Cloud Capability Center" is obtained.
[0120] In summary, the entity abbreviation generation method provided in the above embodiment constructs a corpus by tokenizing the entity name, defines the key degree of the word by using the appearance frequency of the statistical word, calculates the probability gain by the iterative method, retains the key word in the original entity name, and generates the entity abbreviation in the form of word reconstruction. Compared with the prior art, the method has the following advantages: it can automatically simplify various entity names; it does not need additional business logic and business information support; it tokenizes the entity name to construct a corpus, quantifies the key degree of each word by the appearance probability of the word in the corpus; and it generates the entity abbreviation in the form of word reconstruction, does not need training set for training, and belongs to unsupervised learning method.
[0121] Figure 5A structural diagram of an embodiment of the entity acronym generation apparatus of the present application is shown. As shown in the figure, Figure 5 The apparatus 300 comprises a word segmentation module 310, a calculation module 320 and a processing module 330; wherein,
[0122] The word segmentation module 310 is configured to perform word segmentation on a target entity full name based on a pre-set corpus to obtain each component word constituting the target entity full name; wherein the corpus comprises at least one word and the occurrence frequency of each word.
[0123] The calculation module 320 is configured to calculate the frequency gain of each component word according to the occurrence frequency of each word and according to a pre-set rule.
[0124] The processing module 330 is configured to determine the target entity acronym according to the frequency gain of each component word.
[0125] The target entity full name is the entity full name for which an acronym is needed, and the entity includes but is not limited to enterprises, government agencies, social groups, etc. The corpus is established based on the entity full name in the set entity library after word segmentation and other processing, and comprises at least one word and the occurrence frequency of each word.
[0126] In an optional manner, the calculation module 320 is specifically configured to determine the occurrence frequency of each component word according to the occurrence frequency of each word; remove each component word one by one, and calculate the word occurrence frequency of the entity constituted by the remaining component words after removal and the occurrence frequency of each component word, respectively; and calculate the frequency gain of each component word based on the greedy algorithm according to the word occurrence frequency and.
[0127] Here, since the corpus includes the occurrence times of each of the terms, after the full name of the target entity is segmented based on the corpus, the occurrence times of each of the component terms constituting the full name of the target entity can be obtained accordingly. When each of the component terms is not removed one by one, according to the occurrence times of each of the component terms constituting the full name of the target entity and the occurrence times of each of the terms, the occurrence frequencies of each of the component terms at this time can be calculated, i.e., for each component term, the occurrence times of the component term are divided by the sum of the occurrence times of each of the terms, and then the word frequency sum of the full name of the target entity at this time, i.e., the sum of the occurrence frequencies of each of the component terms, can be obtained. When each of the component terms is removed one by one, according to the occurrence times of each of the component terms constituting the full name of the target entity and the occurrence times of each of the terms, the occurrence frequencies of each of the component terms at this time can also be calculated, i.e., for each of the remaining component terms, the occurrence times of the component term are divided by the target times, and the target times are the sum of the occurrence times of each of the terms minus the occurrence times of the component term currently removed, and then the word frequency sum of the entity constituted by the remaining component terms after removal at this time, i.e., the sum of the occurrence frequencies of each of the remaining component terms, can be obtained. Here, after each of the component terms is removed one by one, according to the word frequency sum of the entity constituted by the remaining component terms after removal and the word frequency sum of the full name of the target entity, the frequency gain of each of the component terms can be calculated based on the greedy algorithm. In this way, the key degree of a term is quantified by the occurrence times of the term, and the frequency gain of the term is calculated based on the occurrence times of the term, which is fast and reasonable.
[0128] In an embodiment, the computing module 320 is further configured to, after obtaining the occurrence frequencies of each of the component terms according to the occurrence times of each of the terms, perform logarithmic processing on the occurrence frequencies of each of the component terms. It can be understood that, due to the characteristics of terms in daily use, the frequency of use of a term usually presents a power-law distribution, i.e., a small part of terms often appear in daily use, and the frequency of use of most terms is very low. In order to reduce the influence of the power-law distribution characteristics on the calculation, the occurrence frequencies of each of the component terms need to be logarithmically processed.
[0129] In an embodiment, the processing module 330 is specifically configured to: sort each of the component words in ascending order of frequency gain corresponding to each of the component words; combine and remove each of the component words in ascending order of frequency gain, and calculate a word frequency sum of an entity formed by the component words remaining after the combination and removal according to a frequency of occurrence corresponding to each of the component words; calculate the frequency gain of each combination of the component words based on a greedy algorithm according to the word frequency sum; and construct the target entity abbreviation based on the combination of the component words with the frequency gain less than a preset frequency gain threshold. Here, each of the component words can be sorted in ascending order of frequency gain corresponding to each of the component words, that is, the component word with smaller frequency gain is placed in front, and the component word with larger frequency gain is placed in back. The combination and removal of each of the component words in ascending order of frequency gain can be that the combination and removal of the component words with smaller frequency gain is performed first, such as the combination of two component words or the combination of three component words, and so on. When the combination and removal of each of the component words is performed, the frequency of occurrence corresponding to each of the component words can also be calculated according to the frequency of occurrence corresponding to each of the component words forming the full name of the target entity and the frequency of occurrence corresponding to each of the words, that is, for each of the component words remaining after the combination and removal, the frequency of occurrence corresponding to the component word is divided by a target number, the target number is the sum of the frequencies of occurrence corresponding to each of the words minus the frequency of occurrence corresponding to the component word removed by the current combination and removal, and then the word frequency sum of the entity formed by the component words remaining after the combination and removal can be obtained, that is, the sum of the frequencies of occurrence corresponding to each of the component words remaining after the combination and removal. Here, the frequency gain of each combination of the component words can be calculated based on a greedy algorithm according to the word frequency sum of the entity formed by the component words remaining after the combination and removal and the word frequency sum corresponding to the full name of the target entity after the combination and removal of each of the component words.
[0130] In an alternative manner, as shown in FIG. 3B, the entity abbreviation generation apparatus further includes a combination module 340, an extraction module 350, an analysis module 360, and a library construction module 370. Figure 6
[0131] The combination module 340 is configured to combine each two entity full names in the set entity library to obtain at least one entity full name combination.
[0132] The extraction module 350 is configured to traverse each of the entity full name combinations to obtain a public continuous character subset and a unique continuous character subset in each of the entity full name combinations. The public continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is a difference set of each of the entity full names in the entity full name combination and the public continuous character subset.
[0133] The analysis module 360 is configured to perform relevance analysis on each of the public continuous character subsets to obtain a support degree of each of the public continuous character subsets.
[0134] The library building module 370 is configured to build a word segmentation library based on the public continuous character subsets with the support degree greater than the preset support degree threshold and each of the unique continuous character subsets.
[0135] The word segmentation module 310 is further configured to perform word segmentation on the full name of the entity in the entity library based on the word segmentation library, and generate a corpus according to a word segmentation result. The corpus includes at least one word and a corresponding occurrence frequency of each of the words.
[0136] The combination of each two entity names in the entity library is to combine any two entity names in the entity library into a pair of entity name combinations. For example, if the entity library contains four entity names, six pairs of entity name combinations are obtained after combining any two entity names into a pair of entity name combinations. For each pair of entity name combinations, a string with a length greater than a preset length threshold that is simultaneously located in each entity name of the pair of entity name combinations is extracted, and the string is taken as a common continuous character subset in the pair of entity name combinations. Meanwhile, a set difference operation is performed between each entity name of the pair of entity name combinations and the common continuous character subset in the pair of entity name combinations, respectively, to obtain a character subset belonging to a single entity name of the pair of entity name combinations, and the character subset is taken as a unique continuous character subset in the pair of entity name combinations. The length threshold can be set according to actual needs, for example, 1 or 2 characters. Taking the pair of entity name combinations including China Mobile Beijing Branch and China Mobile Shanghai Branch as an example, the common continuous character subset in the pair of entity name combinations is “China Mobile” and “Branch”, and the unique continuous character subset in the pair of entity name combinations is “Beijing” and “Shanghai” after character extraction in the above manner. Then, the Apriori algorithm can be used to analyze the association of each common continuous character subset to obtain the support degree of each common continuous character subset. The higher the support degree of the common continuous character subset, the higher the probability of the occurrence of the common continuous character subset, which can be used as a standard for word segmentation. Then, a word segmentation library can be established based on the common continuous character subset with a support degree greater than a preset support degree threshold and each unique continuous character subset. Specifically, the common continuous character subset with a high probability of occurrence and each unique continuous character subset in the pair of entity name combinations are added to the word segmentation library, so that the word segmentation library is used as a word segmentation standard. The support degree threshold can be set according to actual needs, which is not limited here. Here, the word segmentation module 310 is specifically configured to: segment each entity name in the entity library based on the word segmentation library to obtain each word constituting each entity name; and establish a corpus based on each word constituting each entity name. It should be noted that the word segmentation result at least includes the words constituting each entity name. Here, after segmenting each entity name in the entity library based on the word segmentation library, the words constituting each entity name can be obtained, and the number of occurrences of each word in different entity names is used to obtain the number of trips corresponding to each word, that is, the sum of the number of occurrences of each word in different entity names is determined as the number of trips corresponding to the word. In this way, the word segmentation method based on the common continuous subset can avoid splitting professional terms, and greatly improve the speed of word segmentation and the rationality of the word segmentation result without relying on a common corpus.
[0137] In summary, the entity nickname generation apparatus provided by the above embodiments quantifies the key degree of a word by the occurrence times of the word, calculates the frequency gain corresponding to the word based on the occurrence times of the word, and then retains the key word in the original entity name according to the frequency gain corresponding to the word to generate the entity nickname, so that the entity nickname is automatically and simply generated with high accuracy.
[0138] Figure 7 The structural schematic diagram of the computer device embodiment of the present application is shown, and the specific embodiments of the present application do not limit the specific implementation of the computer device.
[0139] As shown in Figure 7 The computer device can include a processor 402, a communications interface 404, a memory 406, and a communications bus 408.
[0140] The processor 402, the communications interface 404, and the memory 406 can communicate with each other through the communications bus 408. The communications interface 404 is configured to communicate with network elements such as clients or other servers. The processor 402 is configured to execute the program 410, and specifically can execute the related steps in the above entity nickname generation method embodiments.
[0141] Specifically, the program 410 can include program code including computer executable instructions.
[0142] The processor 402 can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors included in the computer device can be the same type of processor, such as one or more CPUs; or can be different types of processors, such as one or more CPUs and one or more ASICs.
[0143] The memory 406 is configured to store the program 410. The memory 406 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory.
[0144] The program 410 can specifically be invoked by the processor 402 to make the computer device perform the following operations:
[0145] performing word segmentation on the full name of the target entity based on a preset corpus to obtain each component word of the full name of the target entity; wherein the corpus comprises at least one word and a corresponding occurrence frequency of each word;
[0146] According to the occurrence frequency of each word, the frequency gain corresponding to each component word is calculated according to a preset rule;
[0147] According to the frequency gain corresponding to each component word, the short name of the target entity is determined.
[0148] In an optional manner, the program 410 is called by the processor 402 to enable the computer device to perform the following operations:
[0149] Each of the full names of the entities in the set entity library is combined to obtain at least one pair of full name combinations;
[0150] Each pair of full name combinations is traversed to obtain a common continuous character subset and a unique continuous character subset in each pair of full name combinations; wherein the common continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is a difference set of each full name in the full name combination and the common continuous character subset;
[0151] Each of the common continuous character subsets is subjected to relevance analysis to obtain the support degree of each common continuous character subset;
[0152] Based on the common continuous character subset with a support degree greater than a preset support degree threshold and each unique continuous character subset, a word segmentation library is established;
[0153] Based on the word segmentation library, the full names of the entities in the entity library are subjected to word segmentation, and a corpus is generated based on the word segmentation result; wherein the corpus comprises at least one word and a corresponding occurrence frequency of each word.
[0154] In an optional manner, the program 410 is called by the processor 402 to enable the computer device to perform the following operations:
[0155] Based on the Apriori algorithm, the relevance of each common continuous character subset is analyzed to obtain the support degree of each common continuous character subset.
[0156] In an optional manner, the program 410 is called by the processor 402 to enable the computer device to perform the following operations:
[0157] Based on the word segmentation library, each full name of the entity in the entity library is subjected to word segmentation to obtain each word of each full name of the entity;
[0158] According to the appearance times of each of the words in the full names of the entities, a corpus is established.
[0159] In an alternative way, the program 410 is invoked by the processor 402 to make the computer device perform the following operations:
[0160] According to the appearance times of each of the words, the appearance times of each of the component words are determined.
[0161] The component words are removed one by one, and the word frequency sums of the entities formed by the component words remaining after the removal are calculated according to the appearance times of the component words.
[0162] According to the word frequency sums, the frequency gains of the component words are calculated based on a greedy algorithm.
[0163] In an alternative way, the program 410 is invoked by the processor 402 to make the computer device perform the following operations:
[0164] The appearance frequencies of the component words are logarithmically processed.
[0165] In an alternative way, the program 410 is invoked by the processor 402 to make the computer device perform the following operations:
[0166] Based on the frequency gains of the component words, the component words are sorted in ascending order of the frequency gains.
[0167] The component words are combined and removed in ascending order of the frequency gains, and the word frequency sums of the entities formed by the component words remaining after the combined removal are calculated according to the appearance times of the component words.
[0168] According to the word frequency sums, the frequency gains of the combinations of the component words are calculated based on a greedy algorithm.
[0169] Based on the combinations of the component words whose frequency gains are less than a preset frequency gain threshold, the target entity nickname is constructed.
[0170] In summary, in the computer device provided by the above embodiments, the key degree of a word is quantified by the appearance times of the word, the frequency gain of the word is calculated based on the appearance times of the word, and then the key words in the original entity name are retained to generate an entity nickname according to the frequency gain of the word, so that the entity nickname is automatically and simply generated with high accuracy.
[0171] The embodiment of the present application provides a computer readable storage medium, the storage medium stores at least one executable instruction, and the executable instruction enables a computer device / apparatus to execute the entity abbreviation generation method in any method embodiment.
[0172] The executable instruction can be used to enable the computer device / apparatus to perform the following operations:
[0173] The target entity full name is segmented based on a preset corpus, so that each component word of the target entity full name is obtained; wherein the corpus includes at least one word and the occurrence frequency corresponding to each word;
[0174] According to the occurrence frequency corresponding to each word, the frequency gain corresponding to each component word is calculated according to a preset rule;
[0175] The target entity abbreviation is determined according to the frequency gain corresponding to each component word.
[0176] In an optional manner, the executable instruction enables the computer device / apparatus to perform the following operations:
[0177] Each of two entity full names in the set entity library is combined to obtain at least one pair of entity full name combinations;
[0178] Each pair of the entity full name combinations is traversed to obtain a public continuous character subset and a unique continuous character subset in each pair of the entity full name combinations; wherein the public continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is a difference set of each entity full name in the entity full name combination and the public continuous character subset;
[0179] The public continuous character subsets are analyzed for relevance to obtain the support degree of each public continuous character subset;
[0180] The corpus is established based on the public continuous character subset with a support degree greater than a preset support degree threshold and each unique continuous character subset;
[0181] The entity full names in the entity library are segmented based on the segmentation library, and a corpus is generated according to the segmentation result; wherein the corpus includes at least one word and the occurrence frequency corresponding to each word.
[0182] In an optional manner, the analysis of the public continuous character subsets for relevance to obtain the support degree of each public continuous character subset further includes:
[0183] The Apriori algorithm is used to analyze the association of each of the public continuous character subsets, and support degrees of each of the public continuous character subsets are obtained.
[0184] In an alternative manner, the full name of each entity in the entity library is segmented based on the segmentation library, and a corpus is generated according to the segmentation result, further comprising:
[0185] Each of the full names of the entities in the entity library is segmented based on the segmentation library, and each of the words constituting each of the full names of the entities is obtained.
[0186] A corpus is established according to each of the words constituting each of the full names of the entities.
[0187] In an alternative manner, the frequency gain of each of the constituent words is calculated according to the preset rule based on the corresponding number of occurrences of each of the words, further comprising:
[0188] The number of occurrences of each of the constituent words is determined according to the corresponding number of occurrences of each of the words.
[0189] Each of the constituent words is removed one by one, and the word frequency sum of the entity constituted by the constituent words remaining after removal is calculated according to the number of occurrences of each of the constituent words.
[0190] The frequency gain of each of the constituent words is calculated based on the greedy algorithm according to the word frequency sum.
[0191] In an alternative manner, the executable instructions cause the computer device / apparatus to perform the following operations:
[0192] The frequency of occurrence of each of the constituent words is logarithmically processed.
[0193] In an alternative manner, the executable instructions cause the computer device / apparatus to perform the following operations:
[0194] Each of the constituent words is sorted in ascending order of frequency gain based on the frequency gain of each of the constituent words.
[0195] Each of the constituent words is combined and removed in ascending order of frequency gain, and the word frequency sum of the entity constituted by the constituent words remaining after combined removal is calculated according to the number of occurrences of each of the constituent words.
[0196] The frequency gain of each of the constituent word combinations is calculated based on the greedy algorithm according to the word frequency sum.
[0197] The target entity abbreviation is constructed based on the constituent word combination with a frequency gain less than a preset frequency gain threshold.
[0198] In summary, the computer-readable storage medium provided in the above embodiments quantifies the criticality of words by the number of times they appear, calculates the frequency gain corresponding to the words based on the number of times they appear, and then retains the key words in the original entity name to generate entity abbreviations based on the frequency gain corresponding to the words, thereby achieving automatic and simple generation of entity abbreviations with high accuracy.
[0199] This invention provides an entity abbreviation generation apparatus for executing the above-described entity abbreviation generation method.
[0200] This invention provides a computer program that can be called by a processor to cause a computer device to execute the entity abbreviation generation method in any of the above method embodiments.
[0201] This invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed on a computer, cause the computer to perform the entity abbreviation generation method in any of the above method embodiments.
[0202] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages is for the purpose of disclosing the best mode of implementation of the invention.
[0203] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0204] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.
[0205] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than that of the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, all the features disclosed in the specification (including the accompanying claims, abstract and drawings) and all the processes or units of any methods or apparatuses so disclosed can be combined with one another in any combination. Unless explicitly stated otherwise, each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features providing the same, equivalent or similar functionality.
[0206] Further, those skilled in the art will appreciate that a combination of features of different embodiments can mean that the scope of the application is not limited to a single embodiment but is instead formed by the different embodiments themselves and the combinations of features thereof. For example, in the following claims, any of the embodiments claimed can be used in any combination.
[0207] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, any reference signs placed between parentheses in the claim do not limit the claim. The use of the word 'at least' followed by a list of one or more items does not exclude additional such items. The use of the words 'first','second' and the like does not imply any order but rather are used for naming purposes only. The word 'one' does not exclude the presence of more than one item. The word 'another' can mean at least a second or a third. The word 'or' is used in the inclusive sense, i.e. it is used to mean 'and / or'. The word 'comprise', 'comprising' or other variants thereof do not exclude the presence of elements other than those listed in a claim. The word 'device' can refer to a piece of equipment or to software. The word 'about' does not exclude 'exactly' or 'precisely'. The use of the term'step' does not exclude the possibility that a device or a system can perform the recited step automatically. The word'step' can refer to anything a person or a device does while performing an action. It can not require human interaction. The word'step' can be interpreted to be a part of a process or method, and / or a part of a device or system.
Claims
1. An entity acronym generation method characterized by, The method comprises: performing word segmentation on the full name of the target entity based on a pre-set corpus to obtain each component word constituting the full name of the target entity; wherein the corpus comprises at least one word and the occurrence frequency of each word; calculating the frequency gain of each component word according to the occurrence frequency of each word according to a preset rule; wherein the occurrence frequency of each component word is determined according to the occurrence frequency of each word; each component word is removed one by one, and the word occurrence frequency of the entity constituted by the remaining component words after removal is calculated respectively according to the occurrence frequency of each component word; the frequency gain of each component word is calculated based on the greedy algorithm according to the word occurrence frequency; determining the short name of the target entity according to the frequency gain of each component word.
2. The method of claim 1, wherein, Before the performing word segmentation on the full name of the target entity based on a pre-set corpus to obtain each component word constituting the full name of the target entity, the method further comprises: combining each two entity full names in the set entity library to obtain at least one entity full name combination; traversing each entity full name combination to obtain a public continuous character subset and a unique continuous character subset in each entity full name combination; wherein the public continuous character subset is a string with a length greater than a preset length threshold, and the unique continuous character subset is the difference set of each entity full name in the entity full name combination and the public continuous character subset; performing relevance analysis on each public continuous character subset to obtain the support degree of each public continuous character subset; establishing a word segmentation library based on the public continuous character subset with a support degree greater than a preset support degree threshold and each unique continuous character subset; segmenting the entity full names in the entity library based on the word segmentation library, and generating a corpus based on the segmentation result; wherein the corpus comprises at least one word and the occurrence frequency of each word.
3. The method of claim 2, wherein, The relevance analysis on each public continuous character subset to obtain the support degree of each public continuous character subset further comprises: performing relevance analysis on each public continuous character subset based on the Apriori algorithm to obtain the support degree of each public continuous character subset.
4. The method of claim 2, wherein, The segmenting the entity full names in the entity library based on the word segmentation library, and generating a corpus based on the segmentation result further comprises: segmenting each entity full name in the entity library based on the word segmentation library to obtain each word constituting each entity full name; establishing a corpus based on each word constituting each entity full name.
5. The method of claim 1, wherein, After the obtaining the occurrence frequency of each component word according to the occurrence frequency of each word, the method further comprises: performing logarithmic processing on the occurrence frequency of each component word.
6. The method according to any one of claims 1 to 5, characterized in that, The determining the short name of the target entity according to the frequency gain of each component word further comprises: sorting each component word in ascending order of frequency gain based on the frequency gain of each component word. combining and removing each of the component words according to the frequency gain from small to large, and calculating the word frequency sum of the entity composed of the remaining component words after the combining and removing according to the corresponding appearance frequency of each of the component words respectively; calculating the frequency gain of each of the component words combination based on the greedy algorithm according to the word frequency sum; constructing the target entity abbreviation based on the component word combination with the frequency gain less than the preset frequency gain threshold.
7. An entity nickname generating apparatus characterized by comprising: The apparatus comprises: a word segmentation module configured to segment the target entity full name based on a pre-set corpus to obtain each component word of the target entity full name; wherein the corpus comprises at least one word and the corresponding appearance frequency of each of the words; a calculation module configured to calculate the frequency gain of each of the component words according to the corresponding appearance frequency of each of the words according to a preset rule; wherein the appearance frequency of each of the component words is determined according to the corresponding appearance frequency of each of the words; each of the component words is removed one by one, and the word frequency sum of the entity composed of the remaining component words after the removing is calculated according to the corresponding appearance frequency of each of the component words respectively; the frequency gain of each of the component words is calculated based on the greedy algorithm according to the word frequency sum; a processing module configured to determine the target entity abbreviation according to the frequency gain of each of the component words.
8. A computer device, comprising: comprises: a processor, a memory, a communication interface and a communication bus, the processor, the memory and the communication interface complete the communication among each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction makes the processor execute the operation of the entity abbreviation generation method in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The storage medium has at least one executable instruction stored therein, and the executable instruction makes the computer device / apparatus execute the operation of the entity abbreviation generation method in any one of claims 1-6 when the computer device / apparatus runs.
Citation Information
Patent Citations
Method and device for abbreviation generation of entity
CN106991085A