Affix Determination Method, Device, Electronic Device, and Storage Medium

Abstract

The present invention relates to the technical field of data mining, and provides a method, an apparatus, an electronic device and a storage medium for determining an affix. The method includes: obtaining a first trie tree, where the first trie tree is generated according to a first string obtained by reversing a second string in a second trie tree, the second string is a string with a word frequency greater than a preset threshold and satisfying a preset complete condition, the second trie tree is generated according to sample words and suffixes of the sample words in a sample set, each sample word or each suffix is a string, and the word frequency of the string is used to represent the number of times the string appears in the sample set; for any target node in the first trie tree, determining whether the target string corresponding to the target node satisfies a preset affix condition, and each node in the first trie tree represents a character in the first string; if the target string satisfies the affix condition, determining the target string as an affix. The embodiments of the present invention greatly improve the efficiency of affix determination.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining, and in particular, to a method, device, electronic device and storage medium for determining affixes. Background Art

[0002] In the technical field of data mining, the mining of affixes has applications in many scenarios. For example, information extraction in text data, or feature analysis in semantic models, etc. The mining of affixes usually aims to find common prefixes and / or suffixes for enriching the thesaurus, intention judgment, etc. However, in scenarios with extremely large amounts of data, how to efficiently determine affixes has become an urgent problem for those skilled in the art.

[0003] In the prior art, since manual annotation is required for affix mining, the efficiency of affix mining is not high. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, device, electronic device and storage medium for determining affixes, which can efficiently determine affixes.

[0005] To achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for determining affixes, the method including: obtaining a first trie, where the first trie is generated from a first string obtained by reversing a second string in a second trie, the second string being a string with a word frequency greater than a preset threshold and satisfying a preset integrity condition, the second trie being generated from sample words in a sample set and suffixes of the sample words, each of the sample words or each of the suffixes being a string, and the word frequency of the string being used to represent the number of times the string appears in the sample set; for any target node in the first trie, determining whether a target string corresponding to the target node satisfies a preset affix condition, where the target string is determined by nodes on a path from the root node of the first trie to the target node, and each node in the first trie represents a character in the first string; if the target string satisfies the affix condition, determining the target string as an affix.

[0007] Optionally, the step of determining whether the target string corresponding to the target node satisfies a preset affix condition includes:

[0008] If the target string does not satisfy the preset word rule, it is determined that the target string does not satisfy the preset affix condition;

[0009] If the target string satisfies the preset word rule and the target node has no child nodes, it is determined that the target string satisfies the preset affix condition;

[0010] If the target string satisfies the preset word rule and the target node has child nodes, it is determined whether the target string satisfies the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold.

[0011] Optionally, the step of determining whether the target string satisfies the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold includes:

[0012] Calculate a reference value according to the preset threshold and the number of child nodes;

[0013] If the word frequency of the target string is not less than the reference value, it is determined that the target string satisfies the preset affix condition;

[0014] If the word frequency of the target string is less than the reference value, it is determined that the target string does not satisfy the preset affix condition.

[0015] Optionally, the method further includes:

[0016] Obtain a sample set, where the sample set includes sample words and the suffixes of the sample words;

[0017] Generate the second trie according to the sample words and the suffixes of the sample words, where each node in the second trie represents a character in the sample word.

[0018] Optionally, the second trie includes multiple nodes, each node corresponds to a string, and the string corresponding to each node is composed of the characters represented by the nodes on the path from the root node in the second trie to each node. The method further includes:

[0019] Use the strings with word frequencies greater than the preset threshold and satisfying the preset complete condition among the multiple strings as the second strings;

[0020] Reverse the second strings to obtain the first strings;

[0021] Generate the first trie according to the first strings, and set the word frequency of the second strings as the word frequency of the first strings.

[0022] Optionally, the affix is a suffix, and the step of obtaining the sample set includes:

[0023] Obtain the original words;

[0024] Generate a suffix of the original word;

[0025] Use the original word as a sample word and the suffix of the original word as the suffix of the sample word to obtain the sample set.

[0026] Optionally, the affix is a prefix, and the step of obtaining the sample set further includes:

[0027] Obtain the original word;

[0028] Reverse the original word to obtain a reversed word;

[0029] Generate a suffix of the reversed word;

[0030] Use the reversed word as a sample word and the suffix of the reversed word as the suffix of the sample word to obtain the sample set.

[0031] In a second aspect, an embodiment of the present invention provides an affix determination device, including: an acquisition module, configured to acquire a first trie tree, where the first trie tree is generated according to a first string obtained by reversing a second string in a second trie tree, the second string is a string with a word frequency greater than a preset threshold and satisfying a preset complete condition, the second trie tree is generated according to sample words in a sample set and suffixes of the sample words, each of the sample words or each of the suffixes is a string, and the word frequency of the string is used to represent the number of times the string appears in the sample set; a judgment module, configured to, for any target node in the first trie tree, judge whether a target string corresponding to the target node satisfies a preset affix condition, where the target string is determined by nodes on a path from the root node of the first trie tree to the target node, and each node in the first trie tree represents a character in the first string; a determination module, configured to, if the target string satisfies the affix condition, determine the target string as an affix.

[0032] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory; the memory is used to store a program; the processor is configured to, when executing the program, implement the affix determination method as described in the first aspect above.

[0033] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the affix determination method as described in the first aspect above is implemented.

[0034] Compared with the prior art, the affix determination method, apparatus, electronic device, and storage medium provided by the embodiments of the present invention first obtain a first trie generated from a first string obtained by reversing a second string in a second trie. The second string is a string whose word frequency is greater than a preset threshold and satisfies a preset integrity condition. The second trie is generated according to the sample words and the suffixes of the sample words in the sample set. Each sample word or each suffix is a string, and the word frequency of the string is used to represent the number of times the string appears in the sample set. Then, for any target node in the first trie, it is determined whether the target string corresponding to the target node satisfies a preset affix condition. The target string is determined by the nodes on the path from the root node of the first trie to the target node. Each node in the first trie represents a character in the first string. If the target string satisfies the affix condition, the target string is determined as an affix. The embodiments of the present invention filter out the second strings with a word frequency greater than the preset threshold through the second trie, and then generate a second trie from the first string obtained by reversing the second string, and finally determine the affixes that satisfy the affix condition. Since no manual annotation is required, the efficiency of affix determination is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1 It is a flowchart example of the affix determination method provided by the embodiments of the present invention.

[0037] Figure 2 It is an example diagram of the second trie provided by the embodiments of the present invention.

[0038] Figure 3 It is another flowchart example of the affix determination method provided by the embodiments of the present invention.

[0039] Figure 4 It is another flowchart example of the affix determination method provided by the embodiments of the present invention.

[0040] Figure 5 It is another flowchart example of the affix determination method provided by the embodiments of the present invention.

[0041] Figure 6 It is another flowchart example of the affix determination method provided by the embodiments of the present invention.

[0042] Figure 7Another flowchart example of the affix determination method provided by the embodiments of the present invention.

[0043] Figure 8 Shows a block diagram of the affix determination device provided by the embodiments of the present invention.

[0044] Figure 9 Shows a block diagram of the electronic device provided by the embodiments of the present invention.

[0045] Icons: 10 - electronic device; 11 - processor; 12 - memory; 13 - bus; 100 - affix determination device; 110 - acquisition module; 120 - judgment module; 130 - determination module; 140 - generation module. Detailed implementation manners

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0048] It should be noted that: like reference numerals and letters denote like items in the following drawings; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0049] In the description of the present invention, it should be noted that if terms such as "upper", "lower", "inner", "outer", etc. are used to indicate the orientation or positional relationship, it is based on the orientation or positional relationship shown in the accompanying drawings or the orientation or positional relationship in which the product of the present invention is usually placed during use. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation of the present invention.

[0050] In addition, if terms such as "first", "second", etc. are used only for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0051] It should be noted that, without conflict, the features in the embodiments of the present invention can be combined with each other.

[0052] In the prior art, in order to determine the affixes existing in a word entry, manual annotation is usually required. Through model training based on sample word entries with manual annotation, the affixes therein are mined. On the one hand, model training requires a large amount of sample data, and on the other hand, a large amount of sample data needs to be manually annotated, resulting in a long training time for the model, thereby reducing the efficiency of using the trained model for affix mining.

[0053] For the application scenario of the live streaming platform, since the number of live streamers is large, usually in the millions, and many live streamer names are named with the same prefix and suffix, therefore, the technical challenges faced in determining affixes (including prefixes and / or suffixes) are greater.

[0054] In view of this, embodiments of the present invention provide an affix determination method, apparatus, electronic device, and storage medium, which efficiently determine affixes through a first trie tree and a second trie tree, and the following will describe it in detail.

[0055] Since the first trie tree involved in the embodiments of the present invention can be pre-generated and directly obtained when affix determination is required, or the first trie tree can be generated first when affix determination is required, and then the affixes are determined according to the first trie tree. And the first trie tree is generated based on the second trie tree. The second trie tree can be pre-generated and directly obtained when generating the second trie tree, or the second trie tree can be generated first when generating the first trie tree, and then the first trie tree is generated based on the second trie tree. To more clearly illustrate the affix determination method, apparatus, electronic device, and storage medium provided by the embodiments of the present invention, the embodiments of the present invention first describe the generation process of the second trie tree, then describe the generation process of the first trie tree, and finally describe the process of determining affixes based on the first trie tree.

[0056] Please refer to Figure 1 , Figure 1 which is a flowchart example of the affix determination method provided by the embodiments of the present invention, and the method includes the following steps:

[0057] Step S100, obtain a sample set, where the sample set includes sample words and the suffixes of the sample words.

[0058] In this embodiment, the sample words in the sample set are not repeated, and each sample word corresponds to a word entry. The word entry can be a character string composed of Chinese characters, or a string composed of letters, numbers, or other characters. For example, the sample word is: "voice live streamer".

[0059] In linguistics, a suffix, also known as an ending, is an affix that is placed after other morphemes. In this embodiment, the suffixes of a sample word include all the suffixes formed by the characters it contains. For example, if the sample word is "voice-over anchor", then its suffixes are "voice-over anchor", "over anchor", and "anchor".

[0060] Step S101: Generate a second trie based on the sample word and its suffixes, where each node in the second trie represents a character in the sample word.

[0061] The second trie is a suffix tree, which includes a root node and at least one other node. The root node does not represent any character, and each of the other nodes represents a character. From the root node to a certain node, the characters passed through on the path are connected to form the string corresponding to that node. In this embodiment, each node in the trie other than the root node corresponds to a character. From the root node to a certain node, the characters passed through on the path are connected to form the character string corresponding to that node. For example, the sample words in the sample set are "Introduction to Natural Language" and "Programming Language", and their corresponding suffixes are {"Natural Language Introduction", "Language Introduction", "Introduction", "Introduction"}, {"Programming Language", "Language"}, respectively. The generated second trie is as Figure 2 shown Figure 2 This is an example diagram of the second trie provided by the embodiment of the present invention.

[0062] It should be noted that the process of generating the second trie is actually the process of adding the sample word and its suffixes to the second trie. During the generation process, the number of times each node is inserted can be recorded at the same time, that is, when adding a sample word or its suffix, the number of times of each node on the corresponding path will increase by 1. Finally, the number of times each node is inserted is obtained. Since the number of times of each character string corresponding to a node consists of two parts: the number of times contributed by itself and the number of times contributed by other sample words including this character string. For example, the sample words are "Proficient in Programming Language" and "Introduction to Natural Language Learning". "Language" is a suffix, and "Language Learning Introduction" is also a suffix. Moreover, "Language" is the first part of "Language Learning Introduction". The number of times of the suffix "Language" includes the number of times contributed by "Language Learning Introduction". Therefore, for the number of times of the suffix "Language" itself, it should be the total number of times minus the number of times contributed by all other sample words or suffixes including this suffix. In the embodiment of the present invention, the number of times of a character string itself is called the word frequency of the character string.

[0063] In this embodiment, since the second trie is a suffix tree, the N-Gram model of all suffixes can be obtained according to the suffix tree. Therefore, the utilization rate of the sample set can be greatly improved.

[0064] In this embodiment, based on the second trie generated by the above method, strings with relatively high word frequencies and being complete words or complete affixes are screened out from the second trie, and then affixes are determined from the screened strings according to the first trie. Therefore, the embodiment of the present invention also provides a method for generating the first trie based on the second trie. Please refer to Figure 3 , Figure 3 which is another flow example diagram of the affix determination method provided by the embodiment of the present invention. The method includes the following steps:

[0065] Step S102: Use strings in multiple strings with word frequencies greater than a preset threshold and meeting a preset complete condition as the second strings.

[0066] In this embodiment, as a specific implementation manner, the second trie can be recursively traversed to obtain the second strings therein. The second strings are strings in all strings in the second trie with word frequencies greater than a preset threshold and meeting a preset complete condition. The preset threshold can be set according to the needs of the actual scenario. The larger the preset threshold, the more reliable the finally determined affix result; the smaller the preset threshold, the richer the finally determined affix result. The preset complete condition is used to represent that the string is a complete word or a complete affix. For example, "voice host" is a complete word, "host" is a complete affix, "sound host" is neither a complete word nor a complete affix. As an implementation manner, the preset complete condition can be represented by a preset regular expression, that is, a string that meets the preset regular expression is considered to meet the preset complete condition. The preset complete condition can also be represented by whether the string exists in a preset database, that is, if the string exists in the preset database, it is considered that the string meets the preset complete condition. A flag bit can also be set for the string in advance, and different values of the flag bit are used to represent whether the string is a complete word or a complete affix. For example, the value of the flag bit is set to -1, indicating that the string is a complete word or a complete affix.

[0067] Step S103: Reverse the second strings to obtain the first strings.

[0068] In this embodiment, the first strings are the strings obtained by reversing the second strings. Reversing the second strings means rearranging them in the reverse order of the positions of each character in the second strings. For example, if the second string is: "voice host", the corresponding first string is: "host voice".

[0069] In this embodiment, by means of string reversal, it is equivalent to considering both the affix itself and the subsequent part of the affix when selecting candidate affixes, and also considering the affix itself and the preceding part of the affix, which is equivalent to considering each affix candidate from a global perspective and ensuring the accuracy of the finally determined affix.

[0070] Step S104: Generate a first trie based on the first string, and set the word frequency of the second string to the word frequency of the first string.

[0071] In this embodiment, the word frequency of the first string is the same as that of the second string. As a specific implementation, the first string and its word frequency can be put into a hash table, and then the first string is read out from the hash table to generate the first trie. The first trie is an ordinary trie, and its generation method is generated according to the existing technology, which will not be elaborated here.

[0072] In this embodiment, the affix can be a prefix or a suffix. Using the first trie and the second trie, both the prefix and the suffix can be determined. Whether determining the prefix or the suffix, the process of generating the first trie based on the second trie is the same, and the process of determining the affix based on the second trie is also the same. The only difference is the sample set used when generating the second trie. The acquisition methods of the sample sets for determining the prefix and the suffix will be described separately below.

[0073] Please refer to Figure 4 , Figure 4 which is another process example diagram of the affix determination method provided by the embodiment of the present invention. This method is for the acquisition method of the sample set for determining the suffix. Step S100 includes the following sub-steps:

[0074] Sub-step S100-10: Obtain the original word.

[0075] In this embodiment, the original word can be the entries crawled on the application platform. For example, for the live broadcast platform, the original word can be the live broadcast names crawled on the live broadcast platform, and each live broadcast name is an original word.

[0076] Sub-step S100-11: Generate the suffix of the original word.

[0077] In this embodiment, the suffix of the original word has been described when describing the suffix of the sample word in the foregoing step S100, and will not be elaborated here.

[0078] Sub-step S100-12: Take the original word as the sample word and the suffix of the original word as the suffix of the sample word to obtain the sample set.

[0079] Please refer to Figure 5 , Figure 5 which is another process example diagram of the affix determination method provided by the embodiment of the present invention. This method is for the acquisition method of the sample set for determining the prefix. Step S100 further includes the following sub-steps:

[0080] Sub-step S100-20: Obtain the original word.

[0081] Sub-step S100-21: Reverse the original word to obtain the reversed word.

[0082] In this embodiment, the process of reversing the original word is the same as the process of reversing the second string in step S103, which will not be elaborated here.

[0083] Sub-step S100-22: Generate the suffix of the reversed word.

[0084] In this embodiment, the suffix of the reversed word has been described when describing the suffix of the sample word in the foregoing step S100, which will not be elaborated here.

[0085] Sub-step S100-23: Use the reversed word as the sample word and the suffix of the reversed word as the suffix of the sample word to obtain the sample set.

[0086] Based on the first trie created in the manner of the foregoing embodiment, in order to determine the affixes therein, the embodiment of the present invention further provides a method for determining affixes. Please refer to Figure 6 , Figure 6 which is another flowchart example of the affix determination method provided by the embodiment of the present invention. The method includes the following steps:

[0087] Step S200: Obtain the first trie, where the first trie is generated from the first string obtained by reversing the second string in the second trie. The second string is a string with a word frequency greater than a preset threshold and satisfying a preset integrity condition. The second trie is generated from the sample words and the suffixes of the sample words in the sample set. Each sample word or each suffix is a string, and the word frequency of the string is used to represent the number of times the string appears in the sample set.

[0088] In this embodiment, the generation processes of the first trie and the second trie have been described in the foregoing embodiment, which will not be elaborated here.

[0089] Step S201: For any target node in the first trie, determine whether the target string corresponding to the target node satisfies a preset affix condition, where the target string is determined by the nodes on the path from the root node of the first trie to the target node, and each node in the first trie represents a character in the first string.

[0090] In this embodiment, as a specific implementation manner, each node in the first trie can be traversed in a layer-by-layer traversal manner, and then it is determined whether the string corresponding to each node satisfies a preset affix condition. The preset affix condition is used to represent the condition for whether the string is an affix, and can be determined according to at least one of the following two dimensions: (1) the string is a word; (2) the word frequency of the string satisfies a preset condition.

[0091] In this embodiment, the judgment process for any target node in the first trie is the same. In this embodiment, any node in the first trie is taken as an example for illustration, and the target node is any node in the first trie.

[0092] Step S202: If the target string meets the affix condition, determine the target string as an affix.

[0093] The above method provided by the embodiment of the present invention filters out the second strings with word frequencies greater than a preset threshold through the second trie, and then generates the second trie based on the first string obtained by reversing the second string, and finally determines the affixes that meet the affix conditions. Since no manual annotation is required, the efficiency of determining affixes is greatly improved. By statistically extracting the repeatedly shared affixes based on the structure of the trie and a large amount of real data online, the effect of determining affixes is better, the efficiency is higher, and the cost is lower. In the application scenario of the anchor platform, the affixes determined based on the above method can be used to construct an anchor index, judge the anchor's intention, etc.

[0094] Based on Figure 6 , the embodiment of the present invention also provides a specific implementation manner for judging whether the target string corresponding to the target node meets the preset affix condition. Please refer to Figure 7 , Figure 7 is another flowchart example of the affix determination method provided by the embodiment of the present invention. Step S201 includes the following sub-steps:

[0095] Sub-step S2010: If the target string does not meet the preset word rule, determine that the target string does not meet the preset affix condition.

[0096] In this embodiment, the preset word rule is used to represent that a string is a word that can express a complete meaning. The preset word rule can be set as needed. If a string does not meet the preset word rule, then this string must not be an affix. Therefore, it is determined that it must not meet the preset affix condition. For example, "programming language" meets the preset word rule, while "programming language entry" does not meet the preset word rule.

[0097] Sub-step S2011: If the target string meets the preset word rule and the target node has no child nodes, determine that the target string meets the preset affix condition.

[0098] In this embodiment, if the target string meets the preset word rule, it only means that the target string is a word that can express a complete meaning, but it does not necessarily meet the preset affix condition. Further judgment needs to be made according to the specific situation of the target node corresponding to the target string. If the target node has no child nodes, that is, the target node is a leaf node, then it is determined that the target string meets the preset affix condition.

[0099] Sub-step S2012: If the target string meets the preset word rule and the target node has child nodes, then determine whether the target string meets the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold.

[0100] In this embodiment, if the target string meets the preset word rule and its corresponding target node has child nodes, it means that the target string may be an affix or just a string including an affix. If the target string appears alone more frequently, the probability that the target string is an affix is greater. If the target string appears as part of other strings more frequently, the probability that the target string is an affix is smaller. Specifically, it can be further determined according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold. The determination method is as follows:

[0101] First, calculate a reference value according to the preset threshold and the number of child nodes.

[0102] In this embodiment, the reference value can be calculated by the following formula:

[0103] average = threshold ÷ (child.size + 1), where average represents the reference value, threshold represents the preset threshold, and child.size represents the number of child nodes.

[0104] Second, if the word frequency of the target string is not less than the reference value, it is determined that the target string meets the preset affix condition.

[0105] Finally, if the word frequency of the target string is less than the reference value, it is determined that the target string does not meet the preset affix condition.

[0106] In the embodiment, for example, the second strings screened out in the second trie are: "anchor" and "audio anchor". The reversed first strings "host" and "host audio" enter the first trie. When traversing to the node "audio", the node "audio" is a leaf node and has no child nodes, which means that "audio anchor" must be a suffix. When traversing to the node "host", since "host" is not a leaf node and it has child nodes, if the word frequency of the node "host" is not less than the reference value, it means that the string "anchor" does not almost always appear attached to the appearance of "audio anchor", so "anchor" is naturally also a suffix, otherwise it cannot be regarded as a suffix.

[0107] It should be noted that the steps of generating the first trie, the steps of generating the second trie, and the steps of determining the affix can be executed on different devices respectively, or on the same device. When executed on different devices, for example, device A executes the steps of generating the second trie to obtain the second trie, device B obtains the second trie, and executes the steps of the first trie based on the second trie to obtain the first trie, and device C obtains the first trie and then executes the steps of determining the affix based on the first trie. At this time, device A executes steps S100 to step S101 and their sub-steps, device B executes steps S102 to S104, and device C executes steps S200 to step S202 and their sub-steps.

[0108] To execute the corresponding steps in the above embodiments and various possible implementation manners, an implementation manner of an affix determination device 100 is given below. Please refer to Figure 8 , Figure 8 FIG. shows a block diagram of the affix determination device 100 provided by an embodiment of the present invention. It should be noted that for the affix determination device 100 provided in this embodiment, its basic principle and the generated technical effects are the same as those in the above embodiments. For the sake of brief description, some parts of this embodiment are not mentioned.

[0109] The affix determination device 100 includes an acquisition module 110, a judgment module 120, a determination module 130, and a generation module 140.

[0110] The acquisition module 110 is configured to acquire a first trie, where the first trie is generated according to a first string obtained by reversing a second string in a second trie, the second string is a string with a word frequency greater than a preset threshold and satisfying a preset integrity condition, the second trie is generated according to sample words in a sample set and suffixes of the sample words, each sample word or each suffix is a string, and the word frequency of the string is used to represent the number of times the string appears in the sample set.

[0111] Optionally, the acquisition module 110 is further configured to: acquire a sample set, where the sample set includes sample words and suffixes of the sample words.

[0112] Optionally, when the affix is a suffix, the acquisition module 110 is specifically configured to: acquire an original word; generate a suffix of the original word; use the original word as a sample word and the suffix of the original word as a suffix of the sample word to obtain the sample set.

[0113] Optionally, when the affix is a prefix, the acquisition module 110 is specifically configured to: acquire an original word; reverse the original word to obtain a reversed word; generate a suffix of the reversed word; use the reversed word as a sample word and the suffix of the reversed word as a suffix of the sample word to obtain the sample set.

[0114] A judgment module 120 is configured to determine, for any target node in the first trie, whether the target string corresponding to the target node meets a preset affix condition, where the target string is determined by the nodes on the path from the root node of the first trie to the target node, and each node in the first trie represents a character in the first string.

[0115] Optionally, the judgment module 120 is specifically configured to: if the target string does not meet the preset word rule, determine that the target string does not meet the preset affix condition; if the target string meets the preset word rule and the target node has no child nodes, determine that the target string meets the preset affix condition; if the target string meets the preset word rule and the target node has child nodes, determine whether the target string meets the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold.

[0116] Optionally, when the judgment module 120 is specifically configured to determine whether the target string meets the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold, it is specifically configured to: calculate a reference value according to the preset threshold and the number of child nodes; if the word frequency of the target string is not less than the reference value, determine that the target string meets the preset affix condition; if the word frequency of the target string is less than the reference value, determine that the target string does not meet the preset affix condition.

[0117] A determination module 130 is configured to, if the target string meets the affix condition, determine the target string as an affix.

[0118] A generation module 140 is configured to: generate the second trie according to the sample word and the suffix of the sample word, where each node in the second trie represents a character in the sample word.

[0119] Optionally, the second trie includes multiple nodes, each node corresponds to a string, and the string corresponding to each node is composed of the characters represented by the nodes on the path from the root node in the second trie to each node. The generation module 140 is further configured to: use the strings with word frequencies greater than the preset threshold and meeting the preset complete condition among the multiple strings as the second strings; reverse the second strings to obtain the first strings; generate the first trie according to the first strings, and set the word frequency of the second strings as the word frequency of the first strings.

[0120] The embodiment of the present invention further provides a block diagram of an electronic device 10, and the method in the above embodiment is applied to the electronic device 10, please refer to Figure 9 , Figure 9 1 is a block diagram of an electronic device 10 provided in an embodiment of the present invention. The electronic device 10 includes a processor 11 , a memory 12 and a bus 13 . The processor 11 and the memory 12 are connected via the bus 13 .

[0121] The processor 11 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the affix determination method can be completed by hardware integrated logic circuits in the processor 11 or by software instructions. The processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0122] The memory 12 is used to store programs, such as Figure 8 The affix determination device in the embodiment includes at least one software function module that can be stored in the memory 12 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 10. After receiving the execution instruction, the processor 11 executes the program to implement the affix determination method disclosed in the above embodiment.

[0123] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the affix determination method described above is implemented.

[0124] In summary, the embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for determining an affix. The method includes: obtaining a first trie tree, where the first trie tree is generated based on a first string obtained by reversing a second string in a second trie tree, the second string being a string with a word frequency greater than a preset threshold and satisfying a preset integrity condition, the second trie tree being generated based on sample words and suffixes of the sample words in a sample set, each of the sample words or each of the suffixes being a string, and the word frequency of the string being used to represent the number of times the string appears in the sample set; for any target node in the first trie tree, determining whether the target string corresponding to the target node satisfies a preset affix condition, where the target string is determined by nodes on the path from the root node of the first trie tree to the target node, and each node in the first trie tree represents a character in the first string; if the target string satisfies the affix condition, determining the target string as an affix. Compared with the prior art, the embodiments of the present invention filter out the second string with a word frequency greater than the preset threshold through the second trie tree, and then generate the first trie tree based on the first string obtained by reversing the second string, and finally determine the affix that satisfies the affix condition. Since no manual annotation is required, the efficiency of affix determination is greatly improved.

[0125] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for determining affixes, characterized in that, The method includes: Obtain a first trie, where the first trie is generated based on a first string obtained by reversing a second string in a second trie. The second string is a string with a word frequency greater than a preset threshold and satisfying a preset complete condition. The second trie is generated based on sample words in a sample set and suffixes of the sample words. Each sample word or each suffix is a string, and the word frequency of the string is used to represent the number of times the string appears in the sample set; For any target node in the first trie, determine whether the target string corresponding to the target node satisfies a preset affix condition, where the target string is determined by nodes on the path from the root node of the first trie to the target node, and each node in the first trie represents a character in the first string; If the target string satisfies the affix condition, determine the target string as an affix; The step of determining whether the target string corresponding to the target node satisfies the preset affix condition includes: If the target string does not satisfy the preset word rule, determine that the target string does not satisfy the preset affix condition; If the target string satisfies the preset word rule and the target node has no child nodes, determine that the target string satisfies the preset affix condition; If the target string satisfies the preset word rule and the target node has child nodes, determine whether the target string satisfies the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold.

2. The affix determination method according to claim 1, characterized in that The step of determining whether the target string satisfies the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold includes: Calculate a reference value according to the preset threshold and the number of child nodes; If the word frequency of the target string is not less than the reference value, determine that the target string satisfies the preset affix condition; If the word frequency of the target string is less than the reference value, determine that the target string does not satisfy the preset affix condition.

3. The affix determination method according to claim 1, wherein The method further includes: Obtain a sample set, where the sample set includes sample words and suffixes of the sample words; Generate the second trie according to the sample words and the suffixes of the sample words, where each node in the second trie represents a character in the sample word.

4. The affix determination method according to claim 3, wherein The second trie includes multiple nodes, and each node corresponds to a string. The string corresponding to each node is composed of characters represented by nodes on the path from the root node in the second trie to each node. The method further includes: Use the strings with a word frequency greater than the preset threshold and satisfying the preset complete condition among the multiple strings as the second string; Reverse the second string to obtain the first string; Generate the first trie according to the first string, and set the word frequency of the second string as the word frequency of the first string.

5. The affix determination method according to claim 3, characterized in that, The affix is a suffix. The step of obtaining the sample set includes: Obtain an original word; Generate a suffix of the original word; Taking the original word as a sample word and the suffix of the original word as the suffix of the sample word to obtain the sample set.

6. The affix determination method according to claim 3, wherein The affix is a prefix, and the step of obtaining the sample set further includes: Obtaining an original word; Reversing the original word to obtain a reversed word; Generating a suffix of the reversed word; Taking the reversed word as a sample word and the suffix of the reversed word as the suffix of the sample word to obtain the sample set.

7. An affix determination device, characterized in that, The device includes: An acquisition module for acquiring a first trie, where the first trie is generated according to a first string obtained by reversing a second string in a second trie, the second string is a string with a word frequency greater than a preset threshold and satisfying a preset integrity condition, the second trie is generated according to sample words in the sample set and suffixes of the sample words, each sample word or each suffix is a string, and the word frequency of the string is used to represent the number of times the string appears in the sample set; A judgment module for judging, for any target node in the first trie, whether the target string corresponding to the target node satisfies a preset affix condition, where the target string is determined by nodes on the path from the root node of the first trie to the target node, and each node in the first trie represents a character in the first string; A determination module for, if the target string satisfies the affix condition, determining the target string as an affix; The judgment module is specifically used for: If the target string does not satisfy a preset word rule, determining that the target string does not satisfy the preset affix condition; If the target string satisfies the preset word rule and the target node has no child nodes, determining that the target string satisfies the preset affix condition; If the target string satisfies the preset word rule and the target node has child nodes, judging whether the target string satisfies the preset affix condition according to the number of child nodes of the target node, the word frequency of the target string, and the preset threshold.

8. An electronic device, characterized in that, Including a processor and a memory; the memory is used to store a program; the processor is used to, when executing the program, implement the affix determination method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the affix determination method according to any one of claims 1-6.

Images