Word segmentation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a word segmentation method, device, electronic device and storage medium, the method comprising: determining multiple preliminary word segmentation results of a character string to be segmented; based on the word segmentation granularity parameter of each preliminary word segmentation result, or based on the word segmentation granularity parameter and the word segmentation universality parameter of each preliminary word segmentation result, selecting the character string segmentation result of the character string to be segmented from all preliminary word segmentation results. The method, device, electronic device and storage medium provided by the embodiment of the present invention segment the character string to be segmented based on the word segmentation granularity parameter of each preliminary word segmentation result of the character string to be segmented, ensuring that the word segmentation granularity in the character string segmentation result is coarse, thereby reducing the probability of incorrectly splitting a complete word, avoiding the problem of word segmentation ambiguity caused by incorrect splitting, and improving the accuracy of the word segmentation result.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a word segmentation method, device, electronic device and storage medium. Background Art

[0002] With the development of computer science, word segmentation technology, as an important basic technology in the field of natural language processing, has been widely used in scenarios such as machine translation, text summarization, and text retrieval.

[0003] The current word segmentation is implemented based on a dictionary, specifically matching the string to be segmented with the entries in the pre-built dictionary. If a certain entry is matched, the characters corresponding to the entry in the string can be divided into a single word. However, when the string to be segmented involves a possible ambiguous word combination, the above method cannot guarantee the correctness of the word segmentation result. Summary of the invention

[0004] The embodiments of the present invention provide a word segmentation method, device, electronic device and storage medium, which are used to solve the defect in the prior art that the word segmentation results obtained by dictionary matching for word segmentation combinations that may have ambiguity are low in accuracy.

[0005] In a first aspect, an embodiment of the present invention provides a word segmentation method, comprising:

[0006] Determine multiple preliminary segmentation results of the character string to be segmented;

[0007] Based on the word segmentation granularity parameter of each preliminary word segmentation result, or based on the word segmentation granularity parameter and the word segmentation universality parameter of each preliminary word segmentation result, the string segmentation result of the string to be segmented is selected from all preliminary word segmentation results.

[0008] Optionally, the word segmentation granularity parameter includes at least one of the total number of characters, the number of words, and the maximum number of characters of a word contained in the corresponding preliminary word segmentation result;

[0009] The word segmentation universality parameter includes the degree of discreteness of the number of characters of the words corresponding to the preliminary word segmentation result, and / or the degree of universality of the words.

[0010] Optionally, the selecting the string segmentation result of the string to be segmented from all preliminary word segmentation results based on the word segmentation granularity parameter of each preliminary word segmentation result, or based on the word segmentation granularity parameter and the word segmentation universality parameter of each preliminary word segmentation result, specifically includes:

[0011] Based on the word segmentation granularity parameter of each preliminary word segmentation result, a number of candidate word segmentation results are selected from all preliminary word segmentation results;

[0012] If the number of candidate word segmentation results is greater than the preset number of word segmentation results, then based on the word segmentation universality parameter of each candidate word segmentation result, the string word segmentation result is selected from all candidate word segmentation results;

[0013] Otherwise, the plurality of candidate word segmentation results are used as the string word segmentation result.

[0014] Optionally, the selecting of a plurality of candidate word segmentation results from all preliminary word segmentation results based on the word segmentation granularity parameter of each preliminary word segmentation result specifically includes:

[0015] If the number of the preliminary word segmentation results with the largest total number of characters is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the largest total number of characters is used as the string segmentation result, otherwise the preliminary word segmentation result with the smallest number of words is selected from the preliminary word segmentation results with the largest total number of characters;

[0016] If the number of the preliminary word segmentation results with the least number of words is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the least number of words is used as the string word segmentation result; otherwise, the preliminary word segmentation result with the largest number of characters in the largest word is selected from the preliminary word segmentation results with the least number of words as the candidate word segmentation result.

[0017] Optionally, the selecting the string word segmentation result from all candidate word segmentation results based on the word segmentation universality parameter of each candidate word segmentation result specifically includes:

[0018] If the number of candidate word segmentation results with the highest degree of discreteness of the number of characters in the word is less than the preset number of word segmentation results, the candidate word segmentation result with the highest degree of discreteness of the number of characters in the word is used as the string word segmentation result;

[0019] Otherwise, the candidate word segmentation result with the highest word commonality is selected from the candidate word segmentation results with the highest word character number discreteness as the string word segmentation result.

[0020] Optionally, the step of selecting the string segmentation result of the string to be segmented from all preliminary segmentation results further includes:

[0021] Based on the user-defined words included in the character string to be segmented, the character string segmentation result is corrected; wherein the user-defined words included in the character string to be segmented are obtained by matching the character string to be segmented with a user-defined dictionary.

[0022] Optionally, the step of selecting the string segmentation result of the string to be segmented from all preliminary segmentation results further includes:

[0023] Display the word segmentation result of the string;

[0024] If a candidate switching request triggered by a user is received, the plurality of preliminary word segmentation results are displayed for selection by the user;

[0025] If a word segmentation update request determined by the user based on the multiple preliminary word segmentation results is received, the string word segmentation result is replaced with the preliminary word segmentation result indicated in the word segmentation update request.

[0026] In a second aspect, an embodiment of the present invention provides a word segmentation device, including:

[0027] A preliminary word segmentation unit, used to determine multiple preliminary word segmentation results of the character string to be segmented;

[0028] The result selection unit is used to select the string segmentation result of the to-be-segmented string from all preliminary segmentation results based on the segmentation granularity parameter of each preliminary segmentation result, or based on the segmentation granularity parameter and the segmentation universality parameter of each preliminary segmentation result.

[0029] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the word segmentation method provided in the first aspect are implemented.

[0030] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the word segmentation method provided in the first aspect.

[0031] The word segmentation method, device, electronic device and storage medium provided by the embodiments of the present invention segment the character string to be segmented based on the word segmentation granularity parameter of each preliminary word segmentation result of the character string to be segmented, thereby ensuring that the word splitting granularity in the character string segmentation result is coarser, thereby reducing the probability of incorrectly splitting complete words, avoiding the problem of word segmentation ambiguity caused by incorrect splitting, and improving the accuracy of the word segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A flowchart of a word segmentation method provided by an embodiment of the present invention;

[0034] Figure 2A flowchart of a method for determining a string segmentation result provided by an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of a process for screening candidate word segmentation results provided by an embodiment of the present invention;

[0036] Figure 4 A flowchart of a method for filtering string segmentation results provided by an embodiment of the present invention;

[0037] Figure 5 A schematic diagram showing the display of a string segmentation result provided by an embodiment of the present invention;

[0038] Figure 6 A schematic diagram of updating a string segmentation result provided by an embodiment of the present invention;

[0039] Figure 7 A schematic diagram of a flow chart of a word segmentation method provided by another embodiment of the present invention;

[0040] Figure 8 A schematic diagram of the structure of a word segmentation device provided by an embodiment of the present invention;

[0041] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention; Description of the drawings:

[0043] 510-function selection key; 520-request trigger key;

[0044] 530-character string segmentation result display component; 540-segmentation switching selection arrow;

[0045] 550-candidate display frame; 560-preliminary word segmentation result display component;

[0046] 570-Slide the scroll wheel for the word segmentation results. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Language itself is ambiguous, so it is difficult for machines to correctly determine how to segment. For example, in the string "ABC" to be segmented, if "AB" and "BC" belong to two different words, "ABC" can be segmented into "A\BC" or "AB\C". How to correctly segment a string that may be ambiguous has always been a problem that needs to be solved in the field of natural language processing technology.

[0049] Figure 1 A flow chart of a word segmentation method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0050] Step 110: determine a plurality of preliminary segmentation results for the character string to be segmented.

[0051] Specifically, the character string to be segmented is a character string that needs to be segmented, and the preliminary segmentation result of the character string to be segmented can be a segmentation result obtained by a conventional dictionary matching method. Considering that there may be some ambiguous word segmentation combinations in the character string to be segmented, multiple preliminary segmentation results can be obtained based on the dictionary matching method. The number of words contained in the multiple preliminary segmentation results here can be the same or different, and the segmentation granularity reflected by the multiple preliminary segmentation results can be the same or different.

[0052] Step 120 , based on the word segmentation granularity parameter of each preliminary word segmentation result, or based on the word segmentation granularity parameter and the word segmentation universality parameter of each preliminary word segmentation result, select the string segmentation result of the character string to be segmented from all preliminary word segmentation results.

[0053] Specifically, considering that in the word segmentation process, the coarser the granularity of the word segmentation, the smaller the probability of a complete word being split incorrectly, and the smaller the probability of ambiguity caused by incorrect splitting, the word segmentation granularity parameter of the preliminary word segmentation result is set to evaluate the granularity of the split of each word in the preliminary word segmentation result.

[0054] In addition, languages ​​usually follow predetermined rules. For example, in the character string "ABC", if the frequency of use of the word "AB" is much higher than that of the word "BC", the probability that the preliminary word segmentation result "AB\C" is the correct word segmentation result is also higher than the preliminary word segmentation result "A\BC". Therefore, the word segmentation universality parameter of the preliminary word segmentation result is set to evaluate the degree of conformity of the preliminary word segmentation result with the general language rules at the word segmentation structure level or the word application level.

[0055] After determining each preliminary word segmentation result of the string to be segmented, the preliminary word segmentation result with the coarsest word segmentation granularity and the least likely to cause ambiguity due to incorrect segmentation can be directly selected from all preliminary word segmentation results according to the word segmentation granularity parameter of each preliminary word segmentation result as the string segmentation result of the string to be segmented. For example, the string to be segmented is "My favorite movie is The Shawshank Redemption", the preliminary word segmentation result A is "My favorite movie is The Shawshank Redemption", and the preliminary word segmentation result B is "My favorite movie is The Shawshank Redemption", where the preliminary word segmentation result A has a coarser word segmentation granularity, and the preliminary word segmentation result A can be used as the string segmentation result.

[0056] Alternatively, the word segmentation granularity parameter and word segmentation universality parameter of each preliminary word segmentation result can be combined to select a preliminary word segmentation result with a coarse word segmentation granularity, which is not easy to cause ambiguity due to incorrect segmentation, and which conforms to the general language rules and the conventional language expression habits from all preliminary word segmentation results as the string segmentation result of the string to be segmented. For example, the string to be segmented is "Nanjing Yangtze River Bridge", the preliminary word segmentation result C is "Nanjing / Yangtze River Bridge", the preliminary word segmentation result D is "Nanjing / Yangtze River / Bridge", and the preliminary word segmentation result E is "Nanjing City / Yangtze River Bridge", where the word segmentation granularity of preliminary word segmentation results C and E are both the coarsest, and the word "Jiang Bridge" itself is used very rarely, so the preliminary word segmentation result C is used as the string segmentation result.

[0057] It should be noted that the string segmentation result of the string to be segmented can be used as the final segmentation result of the string to be segmented. The string segmentation result can be one or more, which is not specifically limited in the embodiment of the present invention.

[0058] The method provided by the embodiment of the present invention segments the character string to be segmented based on the segmentation granularity parameter of each preliminary segmentation result of the character string to be segmented, thereby ensuring that the word splitting granularity in the character string segmentation result is coarser, thereby reducing the probability of incorrectly splitting a complete word, avoiding the problem of segmentation ambiguity caused by incorrect splitting, and improving the accuracy of the segmentation result.

[0059] Based on the above embodiment, step 110 specifically includes:

[0060] Match the character string to be segmented with the general dictionary to obtain a plurality of preliminary segmentation results each containing words less than or equal to the number of segmentation levels;

[0061] The word segmentation level is determined based on the number of characters in the string to be segmented.

[0062] Specifically, the general dictionary is a pre-set dictionary for segmenting a string. Before dictionary matching, the segmentation level can be determined based on the number of characters in the string to be segmented. Here, the segmentation level is used to represent the number of word matching in the segmentation process. Usually, the segmentation level is less than the number of characters in the string to be segmented, and the segmentation level corresponding to the number of characters in the string to be segmented can be obtained by pre-stating the number of words obtained by segmenting strings with different numbers of characters.

[0063] The number of words included in the preliminary word segmentation result obtained by matching the character string to be segmented with the general dictionary should be less than or equal to the number of word segmentation levels. If the number of words included in the preliminary word segmentation result is equal to the number of word segmentation levels, the preliminary word segmentation result may include some or all characters in the character string to be segmented. If the number of words included in the preliminary word segmentation result is less than the number of word segmentation levels, the preliminary word segmentation result includes all characters in the character string to be segmented.

[0064] Based on any of the above embodiments, the word segmentation granularity parameter includes at least one of the total number of characters, the number of words, and the maximum number of word characters contained in the corresponding preliminary word segmentation result.

[0065] Specifically, the preliminary word segmentation result may be obtained by segmenting the word segmentation string according to a preset word segmentation level. After completing the segmentation of the preset word segmentation level, the preliminary word segmentation result may not be able to achieve the segmentation of all characters in the word segmentation string. For example, the initial word segmentation result may be "Nan / Beijing / Mayor" or "Nanjing City / Yangtze River / Bridge". Therefore, the total number of characters contained in each preliminary word segmentation result can be counted to measure the word splitting granularity of the preliminary word segmentation result. Generally, in the case of a preset word segmentation level, the higher the total number of characters contained in the preliminary word segmentation result, the coarser the word splitting granularity.

[0066] Assuming that the number of word segmentation levels is N, the total number of characters contained in the preliminary word segmentation result can be expressed as Sum(M(m), K(k), …, N(n)):

[0067] Sum(M(m),K(k),…,N(n))=

[0068] =strlen(str(m))+strlen(str(k))+…+strlen(str(n))

[0069] In the formula, M(m), K(k), ..., N(n) represent the words with corresponding serial numbers in each level of the preliminary word segmentation results, M(m) is the mth word in the first level of word segmentation, K(k) is the kth word in the second level of word segmentation, and N(n) is the nth word in the last level of word segmentation. str represents the string of the corresponding word, strlen represents the number of characters in the corresponding string, and Sum(M(m), K(k), ..., N(n)) is the sum of the number of characters in the strings of the words at each level in the preliminary word segmentation results.

[0070] In addition, the word segmentation granularity of the preliminary word segmentation result can also be measured by the number of words contained in the preliminary word segmentation result. In the case of a preset word segmentation level, the number of words contained in each preliminary word segmentation result is less than or equal to the word segmentation level. For example, if the preset word segmentation level is 3, the number of words contained in each preliminary word segmentation result is less than 3. For the same character string to be segmented, the fewer the number of words contained in the preliminary word segmentation result, the coarser the word segmentation granularity.

[0071] The number of words contained in the preliminary word segmentation result can be expressed as N(str(m) / str(k) / … / str(n)):

[0072] N(str(m) / str(k) / … / str(n))=((strlen(str(m))!

[0073] =0)? 1:0)+((strlen(str(k))!=0)?1:0)+…+((strlen(str(n))!=0)?1:0)

[0074] In the formula, strlen(str(m))!=0 means that the number of characters in the string of the mth word in the first level of word segmentation is not 0, (strlen(str(m))!=0)?1∶0) means that if the number of characters in the string of the mth word in the first level of word segmentation is not 0, the value is 1, and if the number of characters in the string of the mth word in the first level of word segmentation is 0, the value is 0. If the number of characters in the string of a word in any level of word segmentation is 0, it means that there is no corresponding word under the word segmentation. Based on the above rules, the value of each level of word segmentation is added to count the number of words contained in the preliminary word segmentation result.

[0075] Furthermore, the word splitting granularity of the preliminary word segmentation result can also be measured by the maximum number of word characters contained in the preliminary word segmentation result. Here, the maximum number of word characters specifically refers to the maximum value of the number of word characters in each word of the preliminary word segmentation result. Generally, the larger the number of word characters, the longer the word as a whole, and the smaller the probability of incorrectly splitting the complete word. In the embodiment of the present invention, the larger the maximum number of word characters, the coarser the word splitting granularity.

[0076] The maximum number of characters in a word in the preliminary word segmentation result can be expressed as LN(str(m) / str(k) / … / str(n)):

[0077] LN(str(m) / str(k) / … / str(n))

[0078] =max(strlen(str(m)), strlen(str(k)),…, strlen(str(n)))

[0079] In the formula, strlen(str(m)), strlen(str(k)), …, strlen(str(n)) are the number of characters in the string corresponding to each level of word segmentation in the preliminary word segmentation results. The maximum number of characters in the string corresponding to each level of word segmentation is taken to obtain the maximum number of characters in the word.

[0080] It should be noted that the total number of characters, the number of words and the maximum number of characters in a word contained in the preliminary word segmentation results can all be used to measure the word splitting granularity of the preliminary word segmentation results. The word segmentation granularity parameter can include any one of the total number of characters, the number of words and the maximum number of characters in a word, or a combination of any two of them, or all of the above three.

[0081] Based on any of the above embodiments, the word segmentation universality parameter includes the degree of discreteness of the number of characters of the words corresponding to the preliminary word segmentation result, and / or the degree of universality of the words.

[0082] Specifically, considering that the frequency of use of super-long words in daily language habits is relatively low, the difference between the number of characters of each word in the result of word segmentation of the character string is usually small. Therefore, under the premise of ensuring that the complete word is not split incorrectly, the degree of discreteness of the number of characters in the preliminary word segmentation result can be used to analyze the degree of conformity of the preliminary word segmentation result with the general language rules at the word segmentation structure level. Here, the degree of discreteness of the number of characters in a word refers to the degree of discreteness of the distribution of the number of characters of each word in the preliminary word segmentation result, which can be specifically expressed as the variance or standard deviation of the number of characters of each word. The smaller the degree of discreteness of the number of characters in a word, the smaller the difference between the number of characters of each word in the preliminary word segmentation result, and the closer it is to the general language rules.

[0083] The discrete degree of the number of characters in the word of the preliminary word segmentation result can be expressed as S(str(m) / str(k) / … / str(n)):

[0084]

[0085]

[0086] Wherein, Mean is the mean number of characters in the string of words at each level of segmentation in the preliminary segmentation results, and the degree of dispersion of the number of characters in the word is the standard deviation of the number of characters in the string of each segmented word calculated on the basis of the Mean.

[0087] In addition, in daily language, there are usually some words that are frequently used in daily life, and some words that are less popular words with a lower daily usage rate. The probability that the words obtained by segmenting the word string to be segmented are words with a higher daily usage rate is obviously higher than the probability that they are less popular words. Therefore, the degree of commonality of the words in the preliminary word segmentation results can be used to analyze the degree of conformity of the preliminary word segmentation results with the general language rules at the word application level. Here, the commonality of words is obtained by the frequency of use of each word in the preliminary word segmentation results in daily language, and specifically can be the sum of the scores of each word based on the frequency of daily use. The higher the commonality of words, the higher the frequency of use of each word in the preliminary word segmentation results, and the more the preliminary word segmentation results conform to the general language rules.

[0088] The common degree of words in the preliminary word segmentation results can be expressed as FreSum:

[0089] FreSum=Fre(str(m))+Fre(str(k))+…+Fre(str(n))

[0090] In the formula, Fre represents the frequency of the corresponding word in the general dictionary. FreSum represents the sum of the frequencies of all words in the preliminary word segmentation results in the general dictionary.

[0091] It should be noted that the word character count discreteness and word commonality included in the word segmentation universality parameter can be used to measure the degree of fit between the preliminary word segmentation results and the general language rules. The word segmentation universality parameter can include only the word character count discreteness or the word commonality, or can include both the word character count discreteness and the word commonality.

[0092] Based on any of the above embodiments, Figure 2 A flow chart of a method for determining a string segmentation result provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, step 120 specifically includes:

[0093] Step 121 , based on the word segmentation granularity parameter of each preliminary word segmentation result, a number of candidate word segmentation results are selected from all preliminary word segmentation results.

[0094] Specifically, firstly, the word segmentation granularity parameters of each preliminary word segmentation result are compared and segmented, and preliminary word segmentation results with coarser word segmentation granularity are selected from all preliminary word segmentation results as candidate word segmentation results. Here, there can be one or more candidate word segmentation results.

[0095] Further, the preliminary word segmentation result with the largest total number of characters, or the preliminary word segmentation result with the least number of words, or the preliminary word segmentation result with the largest number of characters in the largest word can be selected from all preliminary word segmentation results as candidate word segmentation results. It is also possible to select the preliminary word segmentation result with the least number of words and / or the largest number of characters in the largest word from among the preliminary word segmentation results with the largest total number of characters, as a candidate word segmentation result, which is not specifically limited in the embodiment of the present invention.

[0096] Step 122, if the number of candidate word segmentation results is greater than the preset number of word segmentation results, then based on the word segmentation universality parameter of each candidate word segmentation result, a string word segmentation result is selected from all candidate word segmentation results;

[0097] Step 123: otherwise, several candidate word segmentation results are used as string word segmentation results.

[0098] Specifically, based on the candidate word segmentation results obtained in step 121, the possibility of ambiguity caused by incorrectly splitting complete words is basically avoided. On this basis, by counting the number of candidate word segmentation results, it is determined whether the candidate word segmentation results need to be further screened. Here, the maximum number of string word segmentation results allowed to exist can be pre-set as the preset number of word segmentation results. For example, if the preset number of word segmentation results is 1, only one string word segmentation result is ultimately generated.

[0099] Thus, the number of candidate word segmentation results can be compared with the preset number of word segmentation results:

[0100] If the number of candidate word segmentation results is greater than the preset number of word segmentation results, step 122 needs to be executed to further screen the candidate word segmentation results based on the word segmentation universality parameters of each candidate word segmentation result, and select the candidate word segmentation results that are more in line with the universal language rules as the string word segmentation results;

[0101] If the number of candidate word segmentation results is less than or equal to the preset number of word segmentation results, step 123 needs to be executed. At this time, there is no need to further screen the candidate word segmentation results, and the candidate word segmentation results can be directly used as the final string word segmentation results.

[0102] The method provided by the embodiment of the present invention screens the preliminary word segmentation results based on the word segmentation granularity parameter to obtain candidate word segmentation results, which first ensures that there is no possibility of ambiguity in the word segmentation results due to incorrect splitting of complete words. On this basis, the number of candidate word segmentation results is used to determine whether further screening based on the word segmentation universality parameter is required, which effectively reduces the complexity of the word segmentation method execution and improves the word segmentation efficiency.

[0103] Based on any of the above embodiments, Figure 3 A schematic diagram of a process flow of a method for screening candidate word segmentation results provided by an embodiment of the present invention, such as Figure 3 As shown, step 121 specifically includes:

[0104] Step 121 - 1a, if the number of preliminary word segmentation results with the largest total number of characters is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the largest total number of characters is used as the string word segmentation result;

[0105] Step 121 - 1b, otherwise, select the preliminary word segmentation result with the least number of words from the preliminary word segmentation results with the largest total number of characters.

[0106] Specifically, in the process of selecting candidate word segmentation results, the preliminary word segmentation result with the largest total number of characters can be first selected from all preliminary word segmentation results. If the number of preliminary word segmentation results obtained by the maximum total number of characters screening is greater than the preset number of word segmentation results, further screening is required, and step 121-1b is executed to select the preliminary word segmentation result with the least number of words. If the number of preliminary word segmentation results obtained by the maximum total number of characters screening is less than or equal to the preset number of word segmentation results, step 121-1a is executed without further screening, and the preliminary word segmentation result obtained by the maximum total number of characters screening can be directly used as the string segmentation result.

[0107] For example, the three-level segmentation of the word string "Nanjing Yangtze River Bridge" can be expressed as the following table:

[0108]

[0109] Among them, the total number of characters contained in the preliminary word segmentation results of sequence numbers 5, 8, 10, 11, 12, 13, and 14 is 7, that is, the preliminary word segmentation results of sequence numbers 5, 10, 11, 12, 13, and 14 are the preliminary word segmentation results with the largest total number of characters, and the number of preliminary word segmentation results with the largest total number of characters is 6. Assuming that the preset number of word segmentation results is 1, and 6 is greater than 1, the number of preliminary word segmentation results currently obtained by filtering with the maximum total number of characters does not meet the preset number of word segmentation results, and further screening is required, and step 121-1b is executed. The number of words contained in the preliminary word segmentation results of sequence numbers 5, 8, 10, 11, and 13 is 3, and the number of words contained in the preliminary word segmentation results of sequence numbers 12 and 14 is 2, then the preliminary word segmentation results of sequence numbers 12 and 14 are used as the preliminary word segmentation results with the least number of words.

[0110] Step 121 - 2a, if the number of the preliminary word segmentation results with the least number of words is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the least number of words is used as a candidate word segmentation result;

[0111] Step 121 - 2b, otherwise, select the preliminary word segmentation result with the largest number of characters in the largest word from the preliminary word segmentation results with the least number of words as the candidate word segmentation result.

[0112] Specifically, after obtaining several preliminary word segmentation results with the least number of words, the number of preliminary word segmentation results with the least number of words can be counted. If the number of preliminary word segmentation results obtained through the two-level screening of the maximum total number of characters and the minimum number of words is still greater than the preset number of word segmentations, further screening is required, and step 121-2b is executed to select the preliminary word segmentation result with the largest number of characters in the largest word as the candidate word segmentation result with a coarser word segmentation granularity. If the number of preliminary word segmentation results obtained through the two-level screening of the maximum total number of characters and the minimum number of words is less than or equal to the preset number of word segmentations, step 121-2a is executed without further screening, and the preliminary word segmentation results obtained through the two-level screening of the maximum total number of characters and the minimum number of words can be directly used as the string segmentation result.

[0113] For example, the preliminary word segmentation results with the least number of words are the preliminary word segmentation results of sequence numbers 12 and 14 shown in the above table. The number of preliminary word segmentation results with the least number of words is 2, which is greater than the preset number of word segmentation results 1, so further screening is required, and step 121-2b is executed. In the preliminary word segmentation result of sequence number 12, the word with the largest number of characters is "Yangtze River Bridge", and the maximum number of characters is 4. In the preliminary word segmentation result of sequence number 14, the word with the largest number of characters is "Nanjing Mayor", and the maximum number of characters is also 4. Therefore, sequence numbers 12 and 14 are both preliminary word segmentation results with the largest number of characters, and are both used as candidate word segmentation results.

[0114] The method provided by the embodiment of the present invention divides the total number of characters, the number of words and the maximum number of characters in a word contained in the word segmentation granularity parameter into three levels of screening conditions for selecting string segmentation results, thereby effectively reducing the complexity of executing the word segmentation method and improving the word segmentation efficiency.

[0115] Based on any of the above embodiments, Figure 4 A flow chart of a method for filtering string segmentation results provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, step 122 specifically includes:

[0116] Step 122-a, if the number of candidate word segmentation results with the highest degree of discreteness of word characters is less than the preset number of word segmentation results, the candidate word segmentation result with the highest degree of discreteness of word characters is used as the string word segmentation result;

[0117] Step 122-b, otherwise, select the candidate word segmentation result with the highest word commonality from the candidate word segmentation results with the highest word character number discreteness as the string segmentation result.

[0118] Specifically, when candidate word segmentation results are obtained and the number of candidate word segmentation results is greater than the preset number of word segmentation results, the candidate word segmentation results need to be screened based on the word segmentation universality parameters of the candidate word segmentation results to obtain the string word segmentation results.

[0119] First, the candidate word segmentation results with the highest degree of discreteness of the number of word characters can be screened out from the candidate word segmentation results. If the number of candidate word segmentation results with the highest degree of discreteness of the number of word characters is greater than the preset number of word segmentation results, further screening is required, and step 122-b is executed to select the candidate word segmentation results with the highest degree of commonness of words as the string word segmentation results with a coarse word segmentation granularity and conforming to the general language rules. If the number of candidate word segmentation results obtained by screening the degree of discreteness of the number of word characters is less than or equal to the preset number of word segmentations, step 122-a is executed without further screening, and the candidate word segmentation results with the highest degree of discreteness of the number of word characters can be directly used as the string word segmentation results.

[0120] For example, the candidate word segmentation results include two preliminary word segmentation results of sequence numbers 12 and 14. The word character counts of the preliminary word segmentation result of sequence number 12 are 3 and 4, and the word character counts of the preliminary word segmentation result of sequence number 14 are 4 and 3, and the word character count discreteness of the two is the same. At this time, further screening is required based on the commonness of words. In the preliminary word segmentation result of sequence number 12, "Nanjing City" and "Yangtze River Bridge" are both common words, and in the preliminary word segmentation result of sequence number 14, "Nanjing Mayor" is a common word, and "Jiang Bridge" is an uncommon word. The commonness of the words in the preliminary word segmentation result of sequence number 12 is higher than that of the preliminary word segmentation result of sequence number 14, so the preliminary word segmentation result of sequence number 12 is used as the final string word segmentation result.

[0121] The method provided by the embodiment of the present invention divides the discrete degree of the number of word characters contained in the word common degree and the word common degree into two levels of screening conditions for selecting string segmentation results, which effectively reduces the complexity of executing the word segmentation method and improves the word segmentation efficiency.

[0122] When applying a pre-set general dictionary for word segmentation, it is usually impossible to take into account personalized words or proprietary words in certain fields. In addition, with the vigorous development of Internet technology, the emergence speed of network hot words closely related to current events has significantly exceeded the update speed of general dictionaries. In order to meet user needs while segmenting words and avoid incorrect segmentation of proprietary words or hot words, based on any of the above embodiments, step 120 also includes:

[0123] Based on the user-defined words included in the character string to be segmented, the character string segmentation result is corrected; wherein the user-defined words included in the character string to be segmented are obtained by matching the character string to be segmented with a user-defined dictionary.

[0124] Here, the custom dictionary can contain personalized words, proper nouns in specific fields, and Internet hot words that are not included in general dictionaries. The words included in the custom dictionary can be pre-set by the user. Among them, personalized words can be personalized words from user feedback or actively set by the user. Personalized words reflect the user's usage habits and preferences; proper nouns can include names of people, places, countries, landscapes, and some abbreviations. Special nouns can be obtained through a pre-set proper noun library; Internet hot words are the latest and hottest Internet words, which can be obtained in real time through web crawlers. After obtaining the string to be segmented, the string to be segmented can be matched with the custom dictionary to obtain the custom words contained in the string to be segmented.

[0125] The custom word itself may not be included in the general dictionary. If the character string to be segmented is segmented only by the general dictionary, the segmentation result obtained may mistakenly segment the custom word contained in the character string to be segmented. Therefore, after obtaining the character string segmentation result of the character string to be segmented through the general dictionary, the character string segmentation result can be corrected based on the custom word contained in the character string to be segmented, and the corrected character string segmentation result is used as the final segmentation result.

[0126] Here, the string segmentation result is corrected for the custom word, specifically, it can be detected whether the string segmentation result contains the complete custom word, if it does, then the processing is not continued; if it does not, then the words that mistakenly separate the custom word in the string segmentation result are re-segmented, and the result after the re-segmentation is used as the final segmentation result. For example, the string to be segmented "I watched Life of Pi yesterday" contains the custom word "Life of Pi", and the string segmentation result "Yesterday / watched / Life of Pi / of / fantasy / drifting" mistakenly separates the custom word, so the string segmentation result is corrected, and the words "young", "pie", "of", "fantasy", and "drifting" are merged to obtain the corrected string segmentation result "Yesterday / watched / Life of Pi".

[0127] The method provided by the embodiment of the present invention corrects the string segmentation result by using the custom words contained in the string to be segmented, thereby retaining the custom words in the segmentation result and avoiding the problem of incorrect segmentation of the custom words, so that the segmentation result can meet user needs and improve the user experience.

[0128] Based on any of the above embodiments, after step 120, the following steps may be further performed:

[0129] Display string segmentation results;

[0130] If a candidate switching request triggered by a user is received, multiple preliminary word segmentation results are displayed for the user to select;

[0131] If a word segmentation update request determined by the user based on the multiple preliminary word segmentation results is received, the string word segmentation result is replaced with the preliminary word segmentation result indicated in the word segmentation update request.

[0132] Specifically, after obtaining the string segmentation result, the string segmentation result can be displayed for the user to confirm whether the segmentation is correct. Considering that the string segmentation result may not meet the user's needs, an embodiment of the present invention provides a method for updating the string segmentation result under human-computer interaction:

[0133] When the user observes the displayed string segmentation result and determines that the string segmentation result does not meet his or her expectations, the candidate switching request can be triggered. Here, the candidate switching request is used to request the terminal to provide and display other segmentation results other than the string segmentation result. The user can trigger the candidate switching request by touching the preset request trigger button on the screen, or can verbally say a preset password such as "provide candidate segmentation results" to trigger the candidate switching request through voice interaction. Figure 5 A schematic diagram of displaying the string segmentation result provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, a function selection key 510 can be set on the left side of the string segmentation result display component 530. The user clicks the function selection key 510 to confirm the currently displayed string segmentation result and translate it, and double-clicks the function selection key 510 to send a candidate switching request. In addition, a request trigger key 520 can be set on the basis of the function selection key 510, and the user double-clicks the request trigger key to send a candidate switching request.

[0134] After receiving the candidate switching request, the terminal displays each preliminary word segmentation result as a candidate word segmentation result, so that the user can select a word segmentation result that meets his or her expectations. Figure 6 A schematic diagram of updating a string segmentation result provided by an embodiment of the present invention, Figure 6 In the embodiment, a word segmentation switching selection arrow 540 is provided in the string word segmentation result display component 530, and the word segmentation switching selection arrow 540 is used to indicate entering the word segmentation result update state. A plurality of preliminary word segmentation result display components 560 are provided in the candidate display frame 550, and a word segmentation result sliding roller 570 is further provided in the candidate display frame 550, which serves as a sliding component of the candidate display frame to facilitate display of each preliminary word segmentation result display component 560.

[0135] After observing the displayed preliminary word segmentation results, the user can select the word segmentation result that meets his or her expectations and trigger the word segmentation update request. Here, the word segmentation update request is used to request the terminal to replace the string word segmentation result with the preliminary word segmentation result specified by the user. The user can trigger the word segmentation update request by touching the selected preliminary word segmentation result display component on the screen, or can verbally say a preset command such as "select the third candidate" to trigger the word segmentation update request through voice interaction.

[0136] After receiving the word segmentation update request, the terminal may replace the string word segmentation result with the preliminary word segmentation result indicated in the word segmentation update request, thereby updating the string word segmentation result.

[0137] The method provided by the embodiment of the present invention realizes the update of the string segmentation result through human-computer interaction, so that the final string segmentation result can meet the user's expectations, which helps to improve the accuracy of subsequent applications such as text translation.

[0138] Based on any of the above embodiments, after displaying multiple preliminary word segmentation results for user selection, if the multiple preliminary word segmentation results do not meet the user's expectations, each word in the current string word segmentation result can be further segmented and the further segmented results can be provided to the user.

[0139] Here, when further segmenting each word in the current string segmentation result, each word can be used as a string to be segmented to execute steps 110 to 120, thereby gradually reducing the granularity of the segmentation.

[0140] Based on any of the above embodiments, Figure 7 A flow chart of a word segmentation method provided by another embodiment of the present invention is shown as follows: Figure 7 As shown, the word segmentation method includes the following steps:

[0141] First, the character string to be segmented is matched with the general dictionary to obtain multiple preliminary segmentation results.

[0142] Secondly, the preliminary word segmentation result with the largest total number of characters is selected from all preliminary word segmentation results. If the number of preliminary word segmentation results obtained by the maximum total number of characters screening is greater than the preset number of word segmentation results, further screening is required to select the preliminary word segmentation result with the least number of words, otherwise the preliminary word segmentation result with the largest total number of characters is used as the string segmentation result.

[0143] After further screening to obtain the preliminary word segmentation result with the least number of words, if the number of preliminary word segmentation results with the least number of words is greater than the preset number of word segmentation results, further screening is required to select the preliminary word segmentation result with the largest number of characters in the largest word, otherwise the preliminary word segmentation result with the least number of words is used as the string segmentation result.

[0144] After further screening to obtain the preliminary word segmentation result with the largest number of maximum word characters, if the number of preliminary word segmentation results with the largest number of maximum word characters is greater than the preset number of word segmentation results, further screening is required to select the preliminary word segmentation result with the highest degree of discreteness of word character numbers, otherwise the preliminary word segmentation result with the largest number of maximum word characters is used as the string segmentation result.

[0145] After further screening to obtain the preliminary word segmentation results with the highest degree of discreteness of word character counts, if the number of preliminary word segmentation results with the highest degree of discreteness of word character counts is greater than the preset number of word segmentation results, further screening is required to select the candidate word segmentation results with the highest degree of word commonality as the string word segmentation results, otherwise the preliminary word segmentation results with the highest degree of discreteness of word character counts are used as the string word segmentation results.

[0146] After obtaining the string segmentation result, the string segmentation result can be modified in combination with the user-defined words contained in the string to be segmented. After the modification is completed, it can be determined whether it is necessary to perform secondary segmentation based on the string segmentation result according to user needs. If necessary, each word in the string segmentation result is used as the string to be segmented for segmentation. If not, the segmentation is terminated.

[0147] Based on any of the above embodiments, Figure 8 A schematic diagram of the structure of a word segmentation device provided by an embodiment of the present invention is shown in FIG. Figure 8 As shown, the word segmentation device includes a preliminary word segmentation unit 810 and a result selection unit 820;

[0148] The preliminary word segmentation unit 810 is used to determine a plurality of preliminary word segmentation results of the character string to be segmented;

[0149] The result selection unit 820 is used to select the string segmentation result of the to-be-segmented string from all preliminary segmentation results based on the segmentation granularity parameter of each preliminary segmentation result, or based on the segmentation granularity parameter and the segmentation universality parameter of each preliminary segmentation result.

[0150] The device provided by the embodiment of the present invention segments the character string to be segmented based on the segmentation granularity parameter of each preliminary segmentation result of the character string to be segmented, thereby ensuring that the word splitting granularity in the character string segmentation result is coarser, thereby reducing the probability of incorrectly splitting a complete word, avoiding the problem of segmentation ambiguity caused by incorrect splitting, and improving the accuracy of the segmentation result.

[0151] Based on any of the above embodiments, the word segmentation granularity parameter includes at least one of the total number of characters, the number of words, and the maximum number of characters of a word contained in the corresponding preliminary word segmentation result;

[0152] The word segmentation universality parameter includes the degree of discreteness of the number of characters of the words corresponding to the preliminary word segmentation result, and / or the degree of universality of the words.

[0153] Based on any of the above embodiments, the result selection unit 820 includes:

[0154] A candidate selection subunit is used to select a number of candidate word segmentation results from all preliminary word segmentation results based on the word segmentation granularity parameter of each preliminary word segmentation result;

[0155] The final selection subunit is used to select the string segmentation result from all candidate segmentation results based on the segmentation universality parameter of each candidate segmentation result if the number of candidate segmentation results is greater than the preset number of segmentation results; otherwise, the plurality of candidate segmentation results are used as the string segmentation result.

[0156] Based on any of the above embodiments, the candidate selection subunit is specifically used for:

[0157] If the number of the preliminary word segmentation results with the largest total number of characters is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the largest total number of characters is used as the string segmentation result, otherwise the preliminary word segmentation result with the smallest number of words is selected from the preliminary word segmentation results with the largest total number of characters;

[0158] If the number of the preliminary word segmentation results with the least number of words is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the least number of words is used as the string word segmentation result; otherwise, the preliminary word segmentation result with the largest number of characters in the largest word is selected from the preliminary word segmentation results with the least number of words as the candidate word segmentation result.

[0159] Based on any of the above embodiments, the final selection subunit is specifically used for:

[0160] If the number of candidate word segmentation results with the highest degree of discreteness of the number of characters in the word is less than the preset number of word segmentation results, the candidate word segmentation result with the highest degree of discreteness of the number of characters in the word is used as the string word segmentation result;

[0161] Otherwise, the candidate word segmentation result with the highest word commonality is selected from the candidate word segmentation results with the highest word character number discreteness as the string word segmentation result.

[0162] Based on any of the above embodiments, the device further includes a correction unit, wherein the correction unit is used to:

[0163] Based on the user-defined words included in the character string to be segmented, the character string segmentation result is corrected; wherein the user-defined words included in the character string to be segmented are obtained by matching the character string to be segmented with a user-defined dictionary.

[0164] Based on any of the above embodiments, the device further includes an updating unit, wherein the updating unit is configured to:

[0165] Display the word segmentation result of the string;

[0166] If a candidate switching request triggered by a user is received, the plurality of preliminary word segmentation results are displayed for selection by the user;

[0167] If a word segmentation update request determined by the user based on the multiple preliminary word segmentation results is received, the string word segmentation result is replaced with the preliminary word segmentation result indicated in the word segmentation update request.

[0168] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic command in the memory 930 to execute the following method:

[0169] Determine multiple preliminary segmentation results of the character string to be segmented;

[0170] Based on the word segmentation granularity parameter of each preliminary word segmentation result, or based on the word segmentation granularity parameter and the word segmentation universality parameter of each preliminary word segmentation result, the string segmentation result of the string to be segmented is selected from all preliminary word segmentation results.

[0171] In addition, the logic commands in the above-mentioned memory 930 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several commands to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0172] An embodiment of the present invention further provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method provided in each of the above embodiments is implemented, for example, including:

[0173] Determine multiple preliminary segmentation results of the character string to be segmented;

[0174] Based on the word segmentation granularity parameter of each preliminary word segmentation result, or based on the word segmentation granularity parameter and the word segmentation universality parameter of each preliminary word segmentation result, the string segmentation result of the string to be segmented is selected from all preliminary word segmentation results.

[0175] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0176] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of commands to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A word segmentation method, characterized in that: include: Determine multiple preliminary segmentation results of the character string to be segmented; Based on the word segmentation granularity parameter and the word segmentation universality parameter of each preliminary word segmentation result, selecting the string segmentation result of the string to be segmented from all preliminary word segmentation results; The word segmentation universality parameter includes the universality of the word, which is obtained by the usage frequency of each word in the preliminary word segmentation result in daily language. The higher the universality of the word, the higher the usage frequency of each word in the preliminary word segmentation result, and the more the preliminary word segmentation result conforms to the general language rules; the word segmentation granularity parameter includes at least one of the total number of characters, the number of words, and the maximum number of characters of a word contained in the corresponding preliminary word segmentation result; The selecting the string segmentation result of the string to be segmented from all the preliminary segmentation results based on the segmentation granularity parameter and the segmentation universality parameter of each preliminary segmentation result specifically includes: Based on the word segmentation granularity parameter of each preliminary word segmentation result, a number of candidate word segmentation results are selected from all preliminary word segmentation results; If the number of candidate word segmentation results is greater than the preset number of word segmentation results, then based on the word segmentation universality parameter of each candidate word segmentation result, the string word segmentation result is selected from all candidate word segmentation results; Otherwise, the plurality of candidate word segmentation results are used as the string word segmentation result.

2. The word segmentation method according to claim 1, characterized in that: The word segmentation universality parameter also includes the degree of discreteness of the number of characters of the words corresponding to the preliminary word segmentation result.

3. The word segmentation method according to claim 1, characterized in that: The word segmentation granularity parameter of each preliminary word segmentation result is used to select a number of candidate word segmentation results from all preliminary word segmentation results, specifically including: If the number of the preliminary word segmentation results with the largest total number of characters is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the largest total number of characters is used as the string segmentation result, otherwise the preliminary word segmentation result with the smallest number of words is selected from the preliminary word segmentation results with the largest total number of characters; If the number of the preliminary word segmentation results with the least number of words is less than or equal to the preset number of word segmentation results, the preliminary word segmentation result with the least number of words is used as the string word segmentation result; otherwise, the preliminary word segmentation result with the largest number of characters in the largest word is selected from the preliminary word segmentation results with the least number of words as the candidate word segmentation result.

4. The word segmentation method according to claim 2, characterized in that: The selecting the string word segmentation result from all candidate word segmentation results based on the word segmentation universality parameter of each candidate word segmentation result specifically includes: If the number of candidate word segmentation results with the highest degree of discreteness of the number of characters in the word is less than the preset number of word segmentation results, the candidate word segmentation result with the highest degree of discreteness of the number of characters in the word is used as the string word segmentation result; Otherwise, the candidate word segmentation result with the highest word commonality is selected from the candidate word segmentation results with the highest word character number discreteness as the string word segmentation result.

5. The word segmentation method according to any one of claims 1 to 4, characterized in that: The step of selecting the string segmentation result of the string to be segmented from all the preliminary segmentation results further includes: Based on the user-defined words included in the character string to be segmented, the character string segmentation result is corrected; wherein the user-defined words included in the character string to be segmented are obtained by matching the character string to be segmented with a user-defined dictionary.

6. The word segmentation method according to any one of claims 1 to 4, characterized in that: The step of selecting the string segmentation result of the string to be segmented from all the preliminary segmentation results further includes: Display the word segmentation result of the string; If a candidate switching request triggered by a user is received, the plurality of preliminary word segmentation results are displayed for selection by the user; If a word segmentation update request determined by the user based on the multiple preliminary word segmentation results is received, the string word segmentation result is replaced with the preliminary word segmentation result indicated in the word segmentation update request.

7. A word segmentation device, characterized in that: include: A preliminary word segmentation unit, used to determine multiple preliminary word segmentation results of the character string to be segmented; A result selection unit, configured to select a string segmentation result of the string to be segmented from all preliminary segmentation results based on a segmentation granularity parameter and a segmentation universality parameter of each preliminary segmentation result; The word segmentation universality parameter includes the universality of the word, which is obtained by the usage frequency of each word in the preliminary word segmentation result in daily language. The higher the universality of the word, the higher the usage frequency of each word in the preliminary word segmentation result, and the more the preliminary word segmentation result conforms to the general language rules; the word segmentation granularity parameter includes at least one of the total number of characters, the number of words, and the maximum number of characters of a word contained in the corresponding preliminary word segmentation result; The result selection unit is specifically used for: Based on the word segmentation granularity parameter of each preliminary word segmentation result, a number of candidate word segmentation results are selected from all preliminary word segmentation results; If the number of candidate word segmentation results is greater than the preset number of word segmentation results, then based on the word segmentation universality parameter of each candidate word segmentation result, the string word segmentation result is selected from all candidate word segmentation results; Otherwise, the plurality of candidate word segmentation results are used as the string word segmentation result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the word segmentation method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the word segmentation method according to any one of claims 1 to 6 are implemented.

Images