Word cloud generation method and device

CN116805146BActive Publication Date: 2026-10-23BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210267845.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-17
Publication Date
2026-10-23
Estimated Expiration
2042-03-17

AI Technical Summary

Technical Problem

[0004]然而,上述过程生成的词云不能准确的反应特定场景中的关键信息

Benefits of technology

[0017]本公开实施例提供了一种词云生成方法及设备,该方法包括:通过词典中的词语对第一文本进行分词得到多个第一词语,词典中包括置信度大于或等于第一置信度阈值的至少一种长度的第二词语,第二词语是从预设时间周期内生成的第二文本中预先提取得到的,置信度与第二词语在第二文本中的出现频次正相关,置信度与第二词语中的各个字分别在第二文本中的出现频次负相关;根据上述第一词语生成第一文本的词云,该词云用于表示第一文本中的关键信息。本公开实施例可以准确的表示更粗粒度的第二词语的置信度,这些置信度较高的第二词语是整体出现较多的词语。该更粗粒度的第二词语是指长度比词典中最小粒度的第三词语更长的词语,其可以更准确的表示关键信息。从而,将置信度较高的更粗粒度的第二词语添加到词典中,可以使生成词云所使用的词语包括这些更粗粒度的词语,这些更粗粒度的词语有助于提高词云反应关键信息的准确度。此外,由于添加到词典中的词语是置信度较高的第二词语,从而可以避免向词典中添加的词语过多,进而有助于避免词典过大而导致的分词过程较长的问题。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116805146B_ABST
    Figure CN116805146B_ABST
Patent Text Reader

Abstract

The present disclosure provides a word cloud generation method and device, and relates to the technical field of computer processing. The method comprises the following steps: performing word segmentation on a first text by using words in a dictionary to obtain first words, the dictionary comprising second words with a confidence greater than or equal to a first confidence threshold, the second words being extracted from a second text generated in a preset time period, the confidence being positively correlated with the frequency of the second words in the second text and negatively correlated with the frequency of each character in the second words in the second text; and generating a word cloud of the first text according to the first words. The present disclosure can add second words with high confidence to the dictionary, and the second words are composed of two third words with the smallest granularity, so that the words used to generate the word cloud can include these second words with a larger granularity. Since the second words with a larger granularity can more accurately represent key information, the present disclosure can help improve the accuracy of the word cloud in reflecting key information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of computer processing technology, and in particular to a word cloud generation method and apparatus. Background Technology

[0002] A word cloud is a method of visually representing text. It is a cloud-like graphic composed of words in a text, used to display key information. Typically, the most prominent word in the word cloud is the key information in the text.

[0003] In existing technologies, the process of generating a word cloud may include: first, segmenting each text according to a word segmentation algorithm to obtain multiple words in each text; then, deleting stop words from these words, where stop words refer to function words and non-search words in the retrieval process, which can also be understood as non-keywords; then, counting the frequency of each word in these texts; finally, selecting the words with the highest frequency to generate a word cloud.

[0004] However, the word cloud generated by the above process cannot accurately reflect the key information in a specific scenario. Summary of the Invention

[0005] This disclosure provides a word cloud generation method and device that can enable word clouds to accurately reflect key information in a specific scenario.

[0006] In a first aspect, embodiments of this disclosure provide a word cloud generation method, the method comprising:

[0007] The first text is segmented using words from a dictionary to obtain multiple first words. The dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence level threshold. The second word is pre-extracted from the second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and the confidence level is negatively correlated with the frequency of occurrence of each character in the second word in the second text.

[0008] A word cloud is generated from the first word to represent the key information in the first text.

[0009] Secondly, embodiments of this disclosure provide a word cloud generation apparatus, the apparatus comprising:

[0010] The word segmentation module is used to segment a first text into multiple first words using words in a dictionary. The dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence level threshold. The second word is pre-extracted from a second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and the confidence level is negatively correlated with the frequency of occurrence of each character in the second word in the second text.

[0011] The word cloud generation module is used to generate a word cloud of the first text based on the first word, and the word cloud is used to represent key information in the first text.

[0012] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor and a memory;

[0013] The memory stores computer-executed instructions;

[0014] The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method as described in the first aspect.

[0015] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, cause a computing device to implement the method described in the first aspect.

[0016] Fifthly, embodiments of this disclosure provide a computer program for implementing the method as described in the first aspect.

[0017] This disclosure provides a word cloud generation method and apparatus. The method includes: segmenting a first text using words from a dictionary to obtain multiple first words; the dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence threshold; the second words are pre-extracted from the second text generated within a preset time period; the confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and negatively correlated with the frequency of occurrence of each character in the second word in the second text; and generating a word cloud of the first text based on the first words, which is used to represent key information in the first text. This disclosure can accurately represent the confidence level of coarser-grained second words, which are words that appear more frequently overall. The coarser-grained second words refer to words longer than the smallest-grained third word in the dictionary, and can more accurately represent key information. Therefore, adding coarser-grained second words with higher confidence levels to the dictionary allows the words used to generate the word cloud to include these coarser-grained words, which helps improve the accuracy of the word cloud in reflecting key information. Furthermore, since the words added to the dictionary are second words with high confidence, it is possible to avoid adding too many words to the dictionary, which in turn helps to avoid the problem of a long word segmentation process caused by an overly large dictionary. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a word cloud diagram mentioned in the embodiments of this disclosure;

[0020] Figure 2 This is a schematic diagram of the word cloud generation process provided in an embodiment of this disclosure;

[0021] Figure 3 This is a flowchart of the steps of a word cloud generation method provided in an embodiment of this disclosure;

[0022] Figure 4 This is a schematic diagram illustrating the specific process of extracting a second word from a second text according to an embodiment of this disclosure;

[0023] Figure 5 This is an example diagram illustrating a process for extracting a second word according to an embodiment of this disclosure;

[0024] Figure 6 This is a structural block diagram of a word cloud generation device provided in an embodiment of this disclosure;

[0025] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure;

[0026] Figure 8 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0028] Figure 1 This is a word cloud diagram mentioned in the embodiments of this disclosure. (Refer to...) Figure 1 As shown, a word cloud contains many WDs (words). The display position, size, and direction of each word can be different. Generally, words that appear more frequently in the text are displayed more prominently or are larger in size in the word cloud.

[0029] As described in the background section, the above-mentioned technology generates... Figure 1 The word cloud shown cannot accurately reflect the key information in a specific scenario.

[0030] After researching existing technologies, the inventors discovered that the problem stems from the overly fine-grained word segmentation algorithm used in the aforementioned process. This results in the segmented words failing to express the specific meaning within a given context, thus the generated word cloud cannot accurately reflect the key information of that scenario. For example, using existing word segmentation algorithms, the phrase "comments in the comment section cannot be posted" is segmented into "comments," "section," "in," "comments," "post," "not," and "post." If these words are displayed in a word cloud, they do not convey the intended meaning of the text. Consequently, word clouds generated from a large number of such texts also fail to accurately reflect the key information of that scenario. For instance, a word cloud containing the words "post," "post," "comments," and "not" completely obscures the intended meaning of the word cloud.

[0031] To address the aforementioned issues, this embodiment of the disclosure considers modifying the dictionary required for the word segmentation algorithm to include more coarse-grained words. Thus, after calling the dictionary for word segmentation, coarser-grained words can be obtained, such as "cannot be sent," "comment section," or "comments cannot be sent." It can be seen that these words better express a specific meaning, and the word cloud generated based on such coarser-grained words can more accurately reflect key information in a specific scenario. For example, if the word cloud contains "comments cannot be sent," we know that comment sending is abnormal, thus requiring testing and analysis of the comment sending function to resolve the problem.

[0032] Furthermore, by adding coarser-grained words to the dictionary, the aforementioned problems can be solved without modifying the overall process of the existing word cloud generation algorithm. This not only minimizes the impact on the existing word cloud generation algorithm but also reduces development costs.

[0033] However, the key to solving the problem lies in how to include more coarse-grained words in the dictionary. To improve the accuracy of word segmentation, embodiments of this disclosure consider adding coarser-grained words to the dictionary according to a preset time period. That is, at each time period, coarser-grained words from the text within that time period are added to the dictionary. This not only ensures that the dictionary always includes the latest coarser-grained words, thus improving the accuracy of word segmentation, but also avoids classifying the most recent coarser-grained words already present in the text as finer-grained words.

[0034] Of course, not all coarse-grained words within a given time period can be added to the dictionary. This would result in a very large dictionary, leading to too many words to match during segmentation, thus increasing the segmentation time and reducing the segmentation speed. To improve segmentation speed, this embodiment only adds coarse-grained words with higher confidence levels to the dictionary.

[0035] The confidence level of this disclosure is positively correlated not only with the frequency of common occurrences but also negatively correlated with the frequency of occurrence of individual characters within a coarser-grained word. In other words, for a coarser-grained word, if the overall frequency of the word is higher and the frequency of occurrence of each individual character is lower, then the coarser-grained word represents a word with higher confidence. It can be seen that the confidence level is used to represent the probability that the characters within the word typically appear as a whole. This allows coarser-grained words that typically appear as a whole in a specific scenario to be added to the dictionary. This addition process not only reduces the number of coarser-grained words added to the dictionary and improves word segmentation speed but also identifies words that typically appear as a whole as much as possible for display in the word cloud, helping the word cloud accurately reflect the key information in the text.

[0036] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present disclosure will now be described with reference to the accompanying drawings.

[0037] Figure 2 This is a schematic diagram of the word cloud generation process provided in an embodiment of this disclosure. (Refer to...) Figure 2 As shown, Figure 2 The text includes m1 first texts T1_1 to T1_m1 and m2 second texts T2_1 to T2_m2. The generated word cloud is the word cloud of these first texts T1_1 to T1_m1. Specifically, the process of generating the word cloud relies on two algorithms: a word segmentation algorithm and a word cloud generation algorithm.

[0038] The input to the word segmentation algorithm includes the first text T1_1 to T1_m1 and each word in the dictionary. The word segmentation algorithm is used to segment each first text according to each word in the dictionary to obtain the words in the first text, which are called the first words.

[0039] The above word cloud generation algorithm takes as input the first word in the first text and outputs a word cloud of the first text T1_1 to T1_m1, which can represent the key information in the first text T1_1 to T1_m1.

[0040] As can be seen, the above word segmentation algorithm requires the use of a dictionary, which is generated offline in advance. In this embodiment, the dictionary includes not only words from the prior art (hereinafter referred to as third words), but also second words whose confidence level is greater than or equal to the first confidence threshold as determined in this embodiment. The third words are the finer-grained words mentioned in this embodiment, while the second words are longer words composed of two or more third words, i.e., the coarser-grained words mentioned earlier.

[0041] The aforementioned confidence level is determined according to the confidence level determination algorithm provided in this embodiment. The input to the confidence level algorithm includes: the second words in m2 second texts T2_1 to T2_m2 generated within a preset time period, and the output is the confidence level of each second word. For example, the preset time period can be daily, meaning that second words are added to the dictionary every day. The specific process of adding second words every day may include: first, obtaining the second texts generated within that day at a fixed time each day; then, calculating the confidence level of second words of different lengths in each second text; and finally, adding all second words in the second texts with a confidence level greater than or equal to a first preset confidence level to the dictionary.

[0042] Of course, as words are continuously added to the dictionary, the dictionary grows larger over time, resulting in a larger number of words to match during subsequent word segmentation, which slows down the segmentation speed. To solve the above problem, this embodiment of the disclosure can also delete second words from the dictionary through another time period. For example, if a second word is added to the dictionary every day, then second words added before the current week or month can be deleted from the dictionary weekly or monthly.

[0043] Based on the above Figure 2 The word cloud generation process shown is as follows: Figure 3 This is a flowchart of the steps of a word cloud generation method provided in an embodiment of this disclosure. Figure 3 The method shown can be applied to electronic devices, which can be any device with data processing capabilities; this disclosure does not limit such applications. (Refer to...) Figure 3 As shown, the word cloud generation method includes steps S101 to S102.

[0044] S101: The first text is segmented using words from the dictionary to obtain multiple first words. The dictionary includes at least one second word of length with a confidence level greater than or equal to the first confidence level threshold. The second word is pre-extracted from the second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and negatively correlated with the frequency of occurrence of each character in the second word in the second text.

[0045] Among them, the first text is segmented using words from the dictionary to obtain multiple first words, which are... Figure 2 The process involves inputting the first text into a word segmentation algorithm to obtain the first word. For each first text, the word segmentation algorithm works by matching each word in the dictionary against the first text. If the first text contains a target word from the dictionary, that target word is used as the first word obtained after word segmentation. Following this process, all the first words for each first text can be obtained.

[0046] The first text here can be multiple texts, or a single text formed by concatenating multiple texts. Similarly, the second text can also be multiple texts, or a single text formed by concatenating multiple texts. The following explanation will use the example where both the first and second texts are multiple.

[0047] It should be noted that there are still some third words with the minimum granularity in the above dictionary, so the above target word may be the third word in the dictionary or the above second word. The second word is a longer-length word composed of a plurality of the above third words, and the third word is a word that cannot be further split into words. For example, "happy" can be split into two characters "hap" and "py", but these two characters cannot form a word, so "happy" is the third word with the minimum granularity. For another example, "happy and joyful" can be split into two third words "happy" and "joyful", so "happy and joyful" can be the second word.

[0048] The above second words are pre-extracted from the second text generated within a preset time period, which can be understood as language materials (corpus for short). These language materials are generally used for training neural network models, extracting text features, etc., and the trained neural network model can be used to process other texts. Each second word may include one or more second words with different lengths. In order to extract the second words of various lengths as much as possible, when extracting the above second words, we need to constantly change the target length of the word, and the target length is used to indicate the number of characters included in the word. Figure 4 is a schematic diagram of a specific process for extracting the second word from a second text provided by the embodiments of the present disclosure, with reference to Figure 4 as shown, the process may include the following steps:

[0049] S1011: Set the target length to the minimum length.

[0050] Wherein, the minimum length can be set arbitrarily.

[0051] S1012: Set the window of the target length at the start position of the second text.

[0052] That is, in the current state, the start position of the window corresponds to the start position of the second text.

[0053] S1013: Determine the second word corresponding to the window in the second text.

[0054] Wherein, the second word is the word between the start position and the end position corresponding to the window in the second text.

[0055] S1014: Determine whether the window has reached the end position in the second text.

[0056] Specifically, if no, proceed to S1015; if yes, proceed to S1016.

[0057] S1015: Slide the window backward by a first length in the second text, and proceed to S1013.

[0058] The first length can be represented by the number of characters, and the first length can be any length. However, in order to avoid missing some words in the second text due to excessive sliding, this embodiment of the disclosure can set the first length to 1, so that it moves one character forward each time.

[0059] For example, if the first length is set to 3, then when the window before sliding corresponds to the second word "what" in the second text "Why can't I post a comment", sliding the window 3 characters backward will result in the window corresponding to the second word "not" in the second text "Why can't I post a comment". It can be seen that this sliding will miss the word "post a comment".

[0060] However, if the first length is set to 1, then when the window before sliding corresponds to the second word "what" in the second text "Why can't I post a comment", after sliding the window one character backward, the window corresponds to the second word "what comment" in the second text "Why can't I post a comment", so it can be seen that this sliding will not miss any words.

[0061] S1016: Determine whether the target length has reached the maximum length.

[0062] Specifically, if no, proceed to S1017; if yes, proceed to S1018.

[0063] S1017: Increase the target length to the second length and proceed to S1012.

[0064] The second length can be represented by the number of characters, and can be any length. However, to avoid omitting second words of any length, this embodiment sets the second length to 1, increasing the window length by 1 each time. This allows for the acquisition of second words of any length from the minimum length 1 to the maximum length Lmax, meaning the target length is continuous and no second words of any length between the minimum and maximum length are omitted.

[0065] S1018: End the generation of the second word for the second text.

[0066] Figure 5 This is an example diagram illustrating a process for extracting a second word according to an embodiment of this disclosure. Figure 5 The process shown is the process of extracting the second word from the second text when the target length is the minimum length of 2. The second text includes 5 words: CH1 to CH5.

[0067] Reference Figure 5 As shown, when the second word is determined for the first time through a window of length 2, the window is set at the beginning of the second text via S1012, as follows. Figure 5At LC1, as shown, the second word is determined to be a word composed of CH1 and CH2 by S1013.

[0068] Next, S1014 determines that the window has not reached the end position of the second text, thus proceeding to S1015. Then, S1015 slides the window backward by one character; at this point, the window is located... Figure 5 At LC2, proceed to S1013 to determine that the second word is a word composed of CH2 and CH3.

[0069] Next, S1014 determines that the window has not reached the end position of the second text, thus proceeding to S1015. Then, S1015 slides the window backward by one character; at this point, the window is located... Figure 5 At LC3, the process proceeds to S1013 to determine that the second word is a word composed of CH3 and CH4.

[0070] Next, S1014 determines that the window has not reached the end position of the second text, thus proceeding to S1015. Then, S1015 slides the window backward by one character; at this point, the window is located... Figure 5 At LC4, the process proceeds to S1013 to determine that the second word is composed of CH4 and CH5.

[0071] Next, S1014 determines that the window has reached the end position in the second text, and proceeds to S1016. S1016 determines whether the target length has reached the maximum length. If not, proceeds to S1017; if so, proceeds to S1018.

[0072] It is understandable that each second word in each of the aforementioned second texts corresponds to a confidence level. The confidence level of a second word is positively correlated with its frequency of occurrence in the corresponding second text, and negatively correlated with the frequency of occurrence of each character within that second word in each of the various second texts. For example, the confidence level of the j-th second word WD_i_j in the i-th second text T2_i is positively correlated with the frequency of occurrence of WD_i_j in T2_i, and negatively correlated with the frequency of occurrence of each character within WD_i_j in each of T2_1 to T2_m2.

[0073] In other words, when the frequency of each character in WD_i_j in each of T2_1 to T2_m2 is fixed, the confidence level increases as the frequency of WD_i_j in T2_i increases, and decreases as the frequency of WD_i_j in T2_i decreases. Conversely, when the frequency of WD_i_j in T2_i is fixed, the confidence level decreases as the frequency of each character in WD_i_j in each of T2_1 to T2_m2 increases, and increases as the frequency of each character in WD_i_j in each of T2_1 to T2_m2 decreases.

[0074] It should be noted that each character in WD_i_j above corresponds to a frequency of occurrence in T2_1 to T2_m2, which is the total frequency of occurrence of that character in all the second texts.

[0075] Where i is greater than or equal to 1 and less than or equal to m2, and the range of j is uncertain and is related to the number of words included in the second text.

[0076] In summary, the confidence score of each second word is a function of the frequency of occurrence of the second word in the corresponding second text and the frequency of occurrence of each character in the second word in each second text. This function can be any function that satisfies the above relationship.

[0077] Optionally, the confidence level is determined based on the ratio of the frequency of the second word in each second text to the target value. This confidence level can be the ratio itself, or it can be obtained by linearly transforming the ratio. The calculation process for the target value differs under the following two conditions.

[0078] The first condition is when the second word contains two characters.

[0079] Under the first condition mentioned above, the target value is the product of the frequencies of occurrence of the two characters in the second word in each second text.

[0080] Specifically, the target value TV can be determined using the following formula:

[0081] TV=P(CH1)·P(CH2) (1)

[0082] Wherein, P(CH1) is the frequency of occurrence of the first character CH1 in the second word WD in each second text, and P(CH2) is the frequency of occurrence of the second character CH2 in the second word WD in each second text.

[0083] Based on the above formula (1), the confidence level CON of the second word WD, which includes two characters, is calculated as follows:

[0084]

[0085] Wherein, P(WD) represents the frequency of the second word WD in each second text.

[0086] Understandably, for a second word containing only two characters, since it contains fewer characters, the frequency of each character in the second text is only two. Combining this with the frequency of the second word in each text, calculating the confidence score only requires determining three frequencies, resulting in low computational complexity. However, because these three values ​​represent all dimensions of information about the second word, the accuracy of the confidence score can be guaranteed to be high.

[0087] The second condition is that the second word must contain at least three characters.

[0088] Under the second condition mentioned above, the target value is the maximum of the first and second values. The first value is the product of the frequency of the first character in the second word in each second text and the frequency of the segment composed of the remaining characters in the second word in each second text. The second value is the product of the frequency of the last character in the second word in each second text and the frequency of the segment composed of the remaining characters in the second word in each second text.

[0089] Specifically, the target value TV can be determined using the following formula:

[0090] TV=max[P(CH1)·P(FG1), P(FG2)·P(CHM)] (3)

[0091] Wherein, P(CH1) is the frequency of occurrence of the first character CH1 in the second word WD in each second text, P(FG1) is the product of the frequencies of occurrence of the segment FG1 composed of characters other than CH1 in WD in each second text, P(FG2) is the frequency of occurrence of the segment FG2 composed of characters other than the last character CHM in WD in each second text, and P(CHM) is the frequency of occurrence of the last character CHM in WD in each second text.

[0092] Based on the above formula (3), the confidence level CON of the second word WD, which includes at least three words, is calculated as follows:

[0093]

[0094] Wherein, P(WD) represents the frequency of the second word WD in each second text.

[0095] It can be seen that formulas (2) and (4) above both use the ratio directly as the confidence level. However, in practical applications, the ratio can be linearly transformed, but the positive correlation between the ratio and the confidence level can be maintained.

[0096] Understandably, for a second word that includes at least three words, since it contains a large number of characters, if the product of the frequency of each character in the second word in each second text is taken as the target value, then the frequency of each character in the second text needs to be determined, which results in high computational complexity.

[0097] To reduce computational complexity, calculations are performed using formulas (3) and (4) above. It can be seen that regardless of the length of the second word, formulas (3) and (4) only require determining five frequencies of occurrence.

[0098] Furthermore, it can be seen from formulas (3) and (4) that the present invention believes that when CH1 and FG1 appear more often, that is, when P(CH1) is larger and / or P(FG1) is larger, the confidence level of the second word can be determined to be low. It is not necessary to determine that the frequency of each word is large before determining that the confidence level of the second word is low.

[0099] Similarly, when CHM and FG2 appear frequently on their own, that is, when P(CHM) is large and / or P(FG2) is large, it can be determined that the confidence level of the second word is also low. It is not necessary to determine that the frequency of each word is large before determining that the confidence level of the second word is low.

[0100] Therefore, if CH1 and FG1 appear frequently on their own, or if CHM and FG2 appear frequently on their own, then the confidence level can be determined to be low. and Determining the minimum value in the range as the confidence level helps ensure the accuracy of the confidence level, making it a most conservative value.

[0101] After determining the confidence level of each second word using the method described above, second words with a confidence level greater than or equal to a first confidence threshold need to be added to the dictionary. This first confidence threshold can be a preset fixed value; for example, if the confidence level is a value between 0 and 1, then the first confidence threshold can be fixed at 0.9.

[0102] However, this fixed value cannot be dynamically adjusted according to the confidence level, which may result in the first confidence threshold being too large or too small relative to the confidence of all second words.

[0103] If the first confidence threshold is too high, there may be no second word with a confidence level greater than or equal to the first confidence threshold, thus failing to achieve the purpose of coarser-grained word segmentation in this embodiment of the present disclosure.

[0104] If the first confidence threshold is too small, there may be too many second words with confidence levels greater than or equal to the first confidence threshold. This could lead to two problems: firstly, making the dictionary too large, which would hinder word segmentation speed; secondly, it might add the entire second text to the dictionary, failing to achieve the desired word segmentation effect.

[0105] To address the aforementioned issues, this disclosure provides a method for dynamically determining a first confidence threshold. This method determines the first confidence threshold based on the confidence levels of the second words in all second texts, allowing the first confidence threshold to dynamically change as the confidence levels of the second words in all second texts change. This avoids the aforementioned problems caused by the first confidence threshold being too large or too small.

[0106] Specifically, firstly, the confidence level of each third word in the dictionary is determined based on the confidence levels of all second words in all second texts. That is, the confidence levels of each third word in the dictionary are selected from the confidence levels of all second words in all second texts.

[0107] Then, the third words are sorted in descending order according to their confidence level to obtain the first sequence.

[0108] Finally, the confidence level of the third word located at the first target position in the first sequence is determined as the first confidence threshold, and the ratio of the number of third words before the first target position to the total number of third words in the first sequence corresponds to the first preset proportion.

[0109] The first preset percentage is a value greater than 0 and less than 1. For example, the first preset percentage could be 25%. The first target position can be obtained by multiplying the number of third words in the first sequence by this first preset percentage and rounding down. For example, if the number of third words is 100, then the first target position could be 25, and the confidence level of the 25th word in the first sequence could be the first confidence threshold.

[0110] Understandably, since the third word preceding the first target position is the third word with the highest confidence level in the dictionary, representing the first preset percentage, using the confidence level of the first target position as the first confidence threshold ensures that the number of added second words is positively correlated with this first preset percentage. Ideally, the proportion of added second words in all words across all second texts should be close to this first preset percentage. This avoids situations where there are no second words to add to the dictionary, or an excessive number of second words are added.

[0111] S102: Generate a word cloud of the first text based on the first word mentioned above. This word cloud is used to represent key information in the first text.

[0112] The process of generating a word cloud here refers to inputting the first word from all the first texts into... Figure 2 The word cloud generation algorithm in this paper generates word clouds from these first texts. The principle behind this algorithm is roughly to generate word clouds based on the frequency of occurrence of the first word, ensuring that the prominence of the first word displayed in the generated word cloud is positively correlated with its frequency. In other words, the most prominent first word in the word cloud is the one with the highest frequency, and the least prominent first word is the one with the lowest frequency. This way, when a user first sees the word cloud, they can obtain key information represented by the most frequently occurring words in each text.

[0113] As described in the preceding S101, this embodiment of the present disclosure makes the first word in the word cloud more accurately reflect the key information of the first text by adding a coarser-grained second word to the dictionary. In this embodiment, besides directly generating the word cloud from the first word, at least one processing step can be performed on the first word before generating the word cloud from the processed word. Different processing steps are used to achieve different purposes, and the purpose of each processing step will be described in detail later.

[0114] It can be seen that one or more processing steps can be performed on the first word. These processing steps include one or more of the following: a first processing step of connecting adjacent words into a new word; a second processing step of merging similar words into a target word; a third processing step of filtering keywords; and a fourth processing step of filtering words with preset parts of speech.

[0115] The following describes the process of generating a word cloud when the first word is processed.

[0116] If the first process is to connect adjacent words into a new word, then the words required to generate the word cloud include: the fourth word obtained by connecting the adjacent first words and the unconnected first words. Therefore, the process of generating the word cloud includes: generating the word cloud based on the fourth word obtained by connecting and the unconnected first words.

[0117] If the first process is a second process that merges similar words into a target word, then the words required to generate the word cloud include: the first words that have not been merged and the target words obtained by merging them. Thus, the process of generating the word cloud includes: generating a word cloud based on the first words that have not been merged and the target words obtained by merging them.

[0118] If this process is the third process for filtering keywords, then the words needed to generate the word cloud include: keywords filtered from the first word. The process of generating the word cloud includes: generating a word cloud based on the keywords, where the keywords are the first words with high importance parameters.

[0119] If this process is the fourth process of filtering words with preset parts of speech, then the words needed to generate the word cloud include: words with preset parts of speech filtered from the first words, and the process of generating the word cloud includes: generating the word cloud based on the first words with preset parts of speech.

[0120] The following describes the process of generating a word cloud when the first word undergoes at least two processing steps.

[0121] Based on the first to fourth processing methods described above, the at least two processing methods applied to the first word can be a flexible combination of the first to fourth processing methods, and the order of combination between different processing methods is not limited.

[0122] When combining different processing steps, the next processing step applies to all words obtained after the previous processing step, not just the first word. It should be noted that after at least two processing steps, the words used to generate the word cloud are those obtained after the last processing step.

[0123] For example, when the first and fourth processes are combined, the first process is performed before the fourth process. Thus, the fourth process targets not only the unconnected first words, but also the fourth words obtained by connecting adjacent first words. Therefore, the words required to generate the word cloud after the first and fourth processes include: unconnected first words with preset parts of speech and fourth words with preset parts of speech.

[0124] For example, when the second and third processes are combined, the second process precedes the third process. Thus, the words targeted by the third process include: the target words obtained by merging similar words in each of the first words and the unmerged first words. Therefore, the words used to generate the word cloud after the second and third processes include: the target words belonging to the preset part of speech obtained by merging similar words in each of the first words and the unmerged first words with the preset part of speech.

[0125] The following details the word cloud generation process corresponding to the four processing methods described above, using the specific process of each processing method on the first word as an example. When multiple processing methods are combined, the process of the first processing method after the combination is the same as the specific process of processing the first word. The difference between the process of the processing method after the first processing method and the specific process of processing the first word is that the word being processed is no longer the first word, but the word obtained from the previous processing method, but the processing principle is the same.

[0126] In the first process described above, more coarse-grained words are provided by connecting the first words in the first text, thereby further improving the accuracy of the word cloud in reflecting key information.

[0127] Specifically, if the confidence level of the fourth word formed by two adjacent first words in the first text is greater than or equal to the second confidence threshold, then a word cloud of the first text is generated based on the fourth word and the words other than the first words that constitute the fourth word, and the second confidence threshold is less than the aforementioned first confidence threshold.

[0128] The fourth word is one of the second words in the preceding second text, and its confidence level is generated when the confidence levels of each second word in the preceding second text are generated.

[0129] Understandably, when the confidence level of a fourth word formed by two adjacent first words in a first text is greater than or equal to the second confidence threshold, this fourth word can be considered a frequently occurring word. Therefore, these two adjacent first words can be connected to form the fourth word, and a word cloud can be generated based on this connected fourth word and all the remaining first words. For example, if all the first texts include 100 first words from WD1 to WD100, and two adjacent first words WD2 and WD3 in one of the first texts form the fourth word WD101, and its confidence level is greater than or equal to the second confidence threshold, then a word cloud can be generated based on WD101, WD1, WD4 to WD100.

[0130] Of course, the process of connecting two adjacent first words to form a fourth word can be performed once or multiple times. For example, after performing the above connection, the remaining first words in the first text can be connected to form fourth words, or the fourth word can be connected to the words before and after it.

[0131] The above-mentioned connection between the fourth word and the words before and after the fourth word may include at least one of the following: connecting the fourth word with the first word after the fourth word, or connecting the fourth word with the first word before the fourth word.

[0132] In the first continuation connection method, the fourth word and the first word following the fourth word are continued to be connected. Specifically, if the confidence level of the fifth word formed by the last first word constituting the fourth word and the next first word is greater than or equal to the second confidence level threshold, then a word cloud of the first text is generated based on the fifth word and the first words other than those constituting the fifth word.

[0133] For example, in the aforementioned example, after generating a word cloud based on WD101, WD1, WD4 to WD100, if the last word of the fourth word WD101, WD3, and the first word after WD3 in the current first text, WD4, constitute the fifth word WD102, and its confidence level is greater than or equal to the second confidence threshold, then WD101 and WD4 can be connected to form WD102, and a word cloud can be generated based on WD102 and WD1, WD5 to WD100.

[0134] As can be seen, in this embodiment of the present disclosure, after connecting the first word to obtain the fourth word, it is also possible to obtain a coarser-grained and frequently occurring fifth word by connecting the fourth word and the next first word, which helps to further improve the accuracy of word cloud in reflecting key information.

[0135] In the second continuation connection method, the fourth word is continued to be connected with the first word preceding the fourth word. Specifically, if the confidence level of the sixth word formed by the first word constituting the fourth word and the preceding first word is greater than or equal to the aforementioned second confidence threshold, then a word cloud of the first text is generated based on the sixth word and the first words other than those constituting the sixth word.

[0136] For example, in the aforementioned example, after generating a word cloud based on WD101, WD1, WD4 to WD100, if the first word of the fourth word WD101, WD2, and the first word WD1 before the current first text constitute the sixth word WD103, and its confidence is greater than or equal to the second confidence threshold, then WD1 and WD101 can be connected to form WD103, and a word cloud can be generated based on WD103 and WD4 to WD100.

[0137] As can be seen, in this embodiment of the present disclosure, after connecting the first word to obtain the fourth word, it is also possible to obtain a coarser-grained and frequently occurring sixth word by connecting the previous first word and the fourth word, which helps to further improve the accuracy of word cloud in reflecting key information.

[0138] Of course, in practical applications, the fourth word can be connected to the preceding first word, and also to the following first word. For example, in the previous example, after generating a word cloud from WD101, WD1, WD4 to WD100, the preceding word WD1, the fourth word WD101, and the following word WD4 can all be merged to obtain word WD104. Thus, a word cloud can be generated from WD104, WD5 to WD100. This allows for a coarser-grained word cloud, helping to further improve the accuracy of the word cloud in reflecting key information.

[0139] Corresponding to the aforementioned first confidence threshold, the second confidence threshold here can also be a preset fixed value. For example, if the confidence level is a value between 0 and 1, then the second confidence threshold can be fixedly set to 0.7.

[0140] However, this fixed value cannot be dynamically adjusted according to the confidence level, which may result in the second confidence threshold being too large or too small relative to the confidence level of all second words.

[0141] When the second confidence threshold is too high, it may result in the absence of a fourth, fifth, and sixth word with a confidence level greater than or equal to the second confidence threshold, thus making it impossible to connect the first word and consequently failing to achieve the purpose of obtaining coarser-grained words in this embodiment of the present disclosure.

[0142] If the second confidence threshold is too low, there may be too many second words with confidence levels greater than or equal to the second confidence threshold. Consequently, it is possible to connect all the first words in the first text, which is equivalent to generating a word cloud based on the first text. This would result in a poorly readable word cloud, including some unimportant information.

[0143] To address the aforementioned issues, this disclosure provides a method for dynamically determining a second confidence threshold. This method determines the second confidence threshold based on the confidence of the second words in all second texts, allowing the second confidence threshold to dynamically change as the confidence of the second words in all second texts changes. This avoids the aforementioned problems caused by the second confidence threshold being too large or too small.

[0144] Specifically, first, the confidence of each third word in the dictionary is determined according to the confidence of all second words in all second texts. That is, the confidence of each third word in the dictionary is selected from the confidence of all second words in all second texts.

[0145] Then, the third words are sorted in descending order according to the confidence of the third words to obtain a first sequence.

[0146] Finally, the confidence of the third word located at the second target position in the first sequence is determined as a second confidence threshold. The ratio of the number of third words before the second target position to the total number of third words in the first sequence corresponds to a second preset proportion, and the second preset proportion is greater than the first preset proportion.

[0147] The second preset proportion is a value greater than 0 and less than 1, and greater than the first preset proportion. For example, the first preset proportion may be 25%, then the second preset proportion may be 35%. Thus, the number of third words in the first sequence can be multiplied by the second preset proportion, and an integer is taken to obtain the second target position. For example, if there are 100 third words, the second target position may be 35, so that the confidence of the 35th word in the first sequence can be the second confidence threshold.

[0148] It can be understood that, since the third words before the second target position are the top second preset proportion of third words with higher confidence in the dictionary, by taking the confidence at the second target position as the second confidence threshold to connect the first words with confidence greater than or equal to the second confidence threshold, it can be ensured that the number of connected first words is positively correlated with the second preset proportion. Ideally, the proportion of the number of connected first words in all first words of all first texts is close to the second preset proportion. Thereby, it can be avoided that there are no adjacent first words that can be connected, or all first words are connected.

[0149] In the above second processing, similar first words are merged into a target word.

[0150] Among the words required for generating the word cloud, there may be some different words that express the same key information. For example, "hot" and "very hot" express the same key information. These words can be merged into one word. The merging here is different from the aforementioned connection. The aforementioned connection refers to connecting two words into a new word, for example, connecting "send not" and "out" into a word "cannot send out". However, merging refers to merging two words with similar meanings into one word, for example, merging "hot" and "very hot" into one word "very hot", which is actually selecting one word from the two words as the merged word.

[0151] Specifically, the merging process of the first word may include: first, identifying at least two similar words among multiple first words, wherein the at least two similar words satisfy at least one of the following conditions: semantic similarity greater than or equal to a first preset threshold, and Lewinstein distance less than or equal to a second preset threshold; then, merging the at least two similar words among the first words into a target word, wherein the target word is one of the most frequently occurring similar words; and then, generating a word cloud of the first text based on the target word and the remaining unmerged first words.

[0152] In this context, similar words are defined as at least two first words whose semantic similarity is greater than or equal to a first preset threshold and / or whose Lewinstein distance is less than or equal to a second preset threshold. In practical applications, to improve the accuracy of merging, first words with semantic similarity greater than or equal to the first preset threshold and a Lewinstein distance less than or equal to the second preset threshold can be merged.

[0153] The semantic similarity between the two words can be the similarity between their semantic vectors. The similarity between semantic vectors can be determined using existing methods such as cosine similarity, Euclidean distance-based similarity, or Manhattan distance-based similarity. This disclosure does not limit the scope of the methods used.

[0154] The Lewinstein distance between two first words is the number of edits required to change one of the first words to the second word.

[0155] The above-described merging process based on similar words can be performed multiple times. In the first merge, two similar words from the first word are merged into the target word. The target word and the remaining words (excluding the similar words) constitute the words after the first merge. In the second merge, the words from the first round are merged again, and this process is repeated until no similar words remain after the first merge. For example, if there are first words: WD1, WD2, WD3, and WD4, then in the first merge, the similar words WD1 and WD2 are merged into a single target word WD2, resulting in the words WD2, WD3, and WD4 after the first merge. In the second merge, the similar words WD3 and WD4 are merged into the target word WD4, resulting in the words WD2 and WD4 after the second merge. Since WD2 and WD4 are not similar words, the merging process is stopped.

[0156] Through the above merging process of the embodiments of the present disclosure, the display space of the word cloud can be saved. For example, for "hot" and "very hot", if they are not merged, "hot" and "very hot" will be displayed separately in the word cloud, but such a display method is not appropriate, because the two represent the same key information. After merging the two into "very hot", only one piece of the same key information is displayed in the generated word cloud, which can avoid displaying duplicate key information in the word cloud and save the display space of the word cloud.

[0157] In addition, through the above merging process of the embodiments of the present disclosure, the accuracy of the word cloud in reflecting key information can be further improved. For example, for "hot" and "very hot", if they are not merged, the occurrence frequency of "hot" in the first text is FQ1, and its display size in the word cloud is SZ1, and the occurrence frequency of "very hot" in the first text is FQ2, and its display size in the word cloud is SZ2. Since the two represent the same key information, after merging the two into "very hot", its display size in the word cloud is SZ3 corresponding to the occurrence frequency FQ3 obtained by adding FQ1 and FQ2. Since FQ3 is greater than FQ1 and FQ2, SZ3 is greater than SZ1 and SZ2. Since "hot" and "very hot" represent the same key information, and the same key information not only includes "hot" with the occurrence frequency of FQ1, but also includes "very hot" with the occurrence frequency of FQ2, compared with displaying "hot" of SZ1 and "very hot" of SZ2 separately in the word cloud, displaying "very hot" with the size of SZ3 in the word cloud can represent the same key information more accurately.

[0158] In the above third processing, keywords are acquired from the first words in the first text, so as to generate a word cloud according to the keywords. In this way, the word cloud only includes keywords, thereby reducing the display size of the word cloud.

[0159] Specifically, before generating the word cloud, for each first word of each first text, an importance parameter of the first word is determined according to the number of occurrences of the first word in the corresponding first text and the number of first texts in which the first word occurs, and the importance parameter is used to indicate the importance degree of the first word in the first text; then, when generating the word cloud, the word cloud of the first text is generated according to the first words whose importance parameters meet a preset condition, and the preset condition includes at least one of the following: being greater than or equal to a preset importance threshold, or being located at a front position in a second sequence, where the second sequence is obtained by sorting the first words in descending order of the importance parameters.

[0160] The importance parameter IMT is positively correlated with the frequency (TMS) of the first word in the corresponding first text, and negatively correlated with the number of first texts containing the first word (QTT). Therefore, IMT can be understood as a function of TMS and QTT, and this function can be any function that satisfies the aforementioned relationships between IMT, TMS, and QTT.

[0161] Specifically, for a first word in a first text, the aforementioned importance parameter can be calculated through the following process: First, determine the ratio of the number of times the first word appears in the first text to the total number of times the first word appears in the first text, thus obtaining the frequency of occurrence of the first word in the first text; then, determine the ratio between the total number of first texts and the number of first texts containing the first word, thus obtaining the inverse document frequency of the first word; finally, determine the importance parameter of the first word based on the aforementioned frequency of occurrence and the aforementioned inverse document frequency.

[0162] The process of determining the importance parameters described above can be expressed by the following formula:

[0163] First, the frequency F1_i_j of the j-th first word WD_i_j in the i-th first text T1_i can be calculated using the following formula:

[0164]

[0165] Where TMS_i_j is the number of times WD_i_j appears in the first text T1_i, and m1 is the total number of occurrences in the first text.

[0166] Then, the inverse document frequency F2_i_j of the first word WD_i_j can be calculated according to the following formula:

[0167]

[0168] Where WD_i_j∈T1_i means that WD_i_j appears in the i-th first text T1_i, and thus |{i:WD_i_j∈T1_i)}| represents the number of first texts in which WD_i_j appears.

[0169] Finally, the importance parameter is determined based on the calculation results of formulas (5) and (6). The importance parameter is positively correlated with the calculation results of formulas (5) and (6). Therefore, the importance parameter can be considered as a function of the calculation results of formulas (5) and (6), and this function can be any function that satisfies the positive correlation between input and output.

[0170] Optionally, when determining the importance parameter based on the calculation results of formulas (5) and (6), firstly, the target logarithm is obtained by taking the logarithm of the inverse document frequency obtained by formula (6); then, the product of the occurrence frequency obtained by formula (5) and the target logarithm is determined as the importance parameter of the first word.

[0171] Specifically, the above process can be calculated using the following formula:

[0172]

[0173] In the fourth process described above, before generating the word cloud, it is necessary to select some words with preset parts of speech from the first words so as to generate the word cloud based on these first words with preset parts of speech.

[0174] Specifically, the process of generating a word cloud from the first word may include: generating a word cloud of the first text based on the first word with a preset part of speech, wherein the preset part of speech includes at least one of the following: noun, verb, adjective, idiom, or proverb.

[0175] It can be seen that the preset parts of speech are the more important parts of speech in the first text. Therefore, when generating the word cloud, only these more important parts of speech are considered, which helps to avoid unnecessary words occupying the display space of the word cloud and helps to save the display space of the word cloud.

[0176] Corresponding to the word cloud generation method in the above embodiments, Figure 6 This is a structural block diagram of a word cloud generation apparatus provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 6 The word cloud generation device 200 mentioned above includes: a word segmentation module 201 and a word cloud generation module 202.

[0177] The word segmentation module 201 is used to segment the first text using words in a dictionary to obtain multiple first words. The dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence level threshold. The second word is pre-extracted from the second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and the confidence level is negatively correlated with the frequency of occurrence of each character in the second word in the second text.

[0178] The word cloud generation module 202 is used to generate a word cloud of the first text based on the first word, wherein the word cloud is used to represent key information in the first text.

[0179] Optionally, the confidence level is determined based on the ratio of the frequency of the second word in the second text to the target value.

[0180] Wherein, when the second word includes two characters, the target value is the product of the frequencies of occurrence of the two characters in the second word in the second text.

[0181] When the second word includes at least three characters, the target value is the maximum of the first value and the second value. The first value is the product of the frequency of the first character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text. The second value is the product of the frequency of the last character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text.

[0182] Optionally, the dictionary also includes a third word, where at least two of the third words constitute a second word, and the first confidence threshold is pre-generated by the following module:

[0183] The confidence level determination module is used to determine the confidence level of the third word based on the confidence level of the second word.

[0184] The first sequence generation module is used to sort the third word in descending order according to the confidence level of the third word to obtain the first sequence.

[0185] The first threshold determination module is used to determine the confidence level of the third word located at the first target position in the first sequence as the first confidence threshold, and the ratio of the number of third words before the first target position to the total number of third words in the first sequence corresponds to the first preset proportion.

[0186] Optionally, the word cloud generation module 202 is further configured to:

[0187] If the confidence level of the fourth word formed by two adjacent first words in the first text is greater than or equal to the second confidence threshold, then a word cloud of the first text is generated based on the fourth word and the words other than the first words that constitute the fourth word, and the second confidence threshold is less than the first confidence threshold.

[0188] Optionally, the word cloud generation module 202 is further configured to: if the confidence level of the fifth word formed by the last first word constituting the fourth word and the next first word is greater than or equal to the second confidence level threshold, then generate a word cloud of the first text based on the fifth word and the first words other than the words constituting the fifth word.

[0189] Optionally, the word cloud generation module 202 is further configured to:

[0190] If the confidence level of the sixth word formed by the first word constituting the fourth word and the preceding first word is greater than or equal to the second confidence level threshold, then a word cloud of the first text is generated based on the sixth word and the first words other than the words constituting the sixth word.

[0191] Optionally, the second confidence threshold is pre-generated using the following module:

[0192] The second threshold determination module is used to determine the confidence level of the third word located at the second target position in the first sequence as the second confidence threshold. The ratio of the number of third words before the second target position to the total number of third words in the first sequence corresponds to a second preset proportion, and the second preset proportion is greater than the first preset proportion.

[0193] Optionally, the word cloud generation module 202 is further configured to:

[0194] Identify at least two similar words among the multiple first words, wherein the similar words satisfy at least one of the following conditions: semantic similarity greater than or equal to a first preset threshold, and Lewinstein distance less than or equal to a second preset threshold; merge at least two similar words among the first words into a target word, wherein the target word is one of the most frequently occurring similar words; generate a word cloud of the first text based on the target word and the remaining unmerged first words.

[0195] Optionally, the above-mentioned device further includes:

[0196] An importance determination module is used to determine the importance parameter of each first word for each first text before generating a word cloud of the first text based on the first word. If there are multiple first texts, the importance parameter is used to represent the degree of importance of the first word in the first text.

[0197] Based on the aforementioned importance determination module, the word cloud generation module 202 is further configured to:

[0198] A word cloud of the first text is generated based on the first words that satisfy the preset conditions according to the importance parameter. The preset conditions include at least one of the following: greater than or equal to a preset importance threshold, or located in a prominent position in a second sequence. The second sequence is obtained by arranging each first word in descending order according to the importance parameter.

[0199] Optionally, the importance determination module is further configured to:

[0200] The frequency of occurrence of the first word in the first text is obtained by determining the ratio of the number of times the first word appears in the first text to the total number of times the first word appears in the first text; the inverse document frequency of the first word is obtained by determining the ratio between the total number of first texts and the number of first texts containing the first word; and the importance parameter of the first word is determined based on the frequency of occurrence and the inverse document frequency.

[0201] Optionally, the importance determination module is further configured to:

[0202] The target logarithm is obtained by taking the logarithm of the inverse document frequency; the product of the occurrence frequency and the target logarithm is determined as the importance parameter of the first word.

[0203] Optionally, the word cloud generation module 202 is further configured to:

[0204] The word cloud of the first text is generated based on the first word with a preset part of speech, wherein the preset part of speech includes at least one of the following: noun, verb, adjective, idiom, and proverb.

[0205] The word cloud generation device provided in this embodiment can be used to perform the above-mentioned... Figure 3 The technical solutions of the method embodiments shown are similar in implementation principle and technical effect, and will not be described again here.

[0206] Figure 7 This is a structural block diagram of an electronic device 600 provided in an embodiment of the present disclosure. The electronic device 600 includes a memory 602 and at least one processor 601.

[0207] Among them, memory 602 stores computer-executed instructions.

[0208] At least one processor 601 executes computer execution instructions stored in memory 602, causing electronic device 601 to perform the aforementioned functions. Figure 3 The method in the middle.

[0209] In addition, the electronic device may also include a receiver 603 and a transmitter 604, wherein the receiver 603 is used to receive information from other devices or equipment and forward it to the processor 601, and the transmitter 604 is used to send information to other devices or equipment.

[0210] Further, refer to Figure 8The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. This electronic device 900 can be a terminal device. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0211] like Figure 8 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0212] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0213] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0214] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0215] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0216] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0217] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0218] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0219] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0220] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0221] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0222] In a first example of the first aspect, this disclosure provides a word cloud generation method, the method comprising:

[0223] The first text is segmented using words from a dictionary to obtain multiple first words. The dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence level threshold. The second word is pre-extracted from the second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and the confidence level is negatively correlated with the frequency of occurrence of each character in the second word in the second text.

[0224] A word cloud is generated from the first word to represent the key information in the first text.

[0225] Based on the first example of the first aspect, in the second example of the first aspect, the confidence level is determined based on the ratio of the frequency of the second word in the second text to the target value;

[0226] When the second word includes two characters, the target value is the product of the frequencies of occurrence of the two characters in the second word in the second text.

[0227] When the second word includes at least three characters, the target value is the maximum of the first value and the second value. The first value is the product of the frequency of the first character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text. The second value is the product of the frequency of the last character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text.

[0228] Based on the first example of the first aspect, in the third example of the first aspect, the dictionary further includes a third word, at least two of the third words constitute a second word, and the first confidence threshold is pre-generated through the following steps:

[0229] The confidence level of the third word is determined based on the confidence level of the second word;

[0230] The third words are sorted in descending order according to their confidence level to obtain the first sequence;

[0231] The confidence level of the third word located at the first target position in the first sequence is determined as the first confidence threshold, and the ratio of the number of third words before the first target position to the total number of third words in the first sequence corresponds to the first preset proportion.

[0232] Based on the third example of the first aspect, in the fourth example of the first aspect, generating the word cloud of the first text based on the first word includes:

[0233] If the confidence level of the fourth word formed by two adjacent first words in the first text is greater than or equal to the second confidence threshold, then a word cloud of the first text is generated based on the fourth word and the words other than the first words that constitute the fourth word, and the second confidence threshold is less than the first confidence threshold.

[0234] Based on the fourth example of the first aspect, in the fifth example of the first aspect, generating a word cloud of the first text based on the fourth word and words other than the first word constituting the fourth word includes:

[0235] If the confidence level of the fifth word formed by the last first word constituting the fourth word and the next first word is greater than or equal to the second confidence level threshold, then a word cloud of the first text is generated based on the fifth word and the first words other than the words constituting the fifth word.

[0236] Based on the fourth example of the first aspect, in the sixth example of the first aspect, generating a word cloud of the first text based on the fourth word and words other than the first word constituting the fourth word includes:

[0237] If the confidence level of the sixth word formed by the first word constituting the fourth word and the preceding first word is greater than or equal to the second confidence level threshold, then a word cloud of the first text is generated based on the sixth word and the first words other than the words constituting the sixth word.

[0238] Based on examples four through six of the first aspect, in example seven of the first aspect, the second confidence threshold is pre-generated through the following steps:

[0239] The confidence level of the third word located at the second target position in the first sequence is determined as the second confidence threshold. The ratio of the number of third words before the second target position to the total number of third words in the first sequence corresponds to the second preset proportion, which is greater than the first preset proportion.

[0240] Based on the first to sixth examples of the first aspect, in the eighth example of the first aspect, generating a word cloud of the first text based on the first word includes:

[0241] Determine at least two similar words from a plurality of the first words, wherein the similar words satisfy at least one of the following conditions: semantic similarity is greater than or equal to a first preset threshold, and Lewinstein distance is less than or equal to a second preset threshold;

[0242] At least two similar words in the first word are merged into a target word, wherein the target word is one of the similar words that appears most frequently.

[0243] Generate a word cloud of the first text based on the target word and the remaining unmerged first words.

[0244] Based on the first to sixth examples of the first aspect, in the ninth example of the first aspect, before generating the word cloud of the first text based on the first word, the method further includes:

[0245] If there are multiple first texts, then for each first text, the importance parameter of each first word is determined based on the number of times the first word appears in the first text and the number of first texts containing the first word. The importance parameter is used to represent the degree of importance of the first word in the first text.

[0246] The step of generating a word cloud of the first text based on the first word includes:

[0247] A word cloud of the first text is generated based on the first words that satisfy the preset conditions according to the importance parameter. The preset conditions include at least one of the following: greater than or equal to a preset importance threshold, or located in a prominent position in a second sequence. The second sequence is obtained by arranging each first word in descending order according to the importance parameter.

[0248] Based on the ninth example of the first aspect, in the tenth example of the first aspect, the determination of the importance parameter of each first word for each first text, according to the number of times the first word appears in the first text and the number of first texts containing the first word, includes:

[0249] The frequency of the first word in the first text is obtained by determining the ratio of the number of times the first word appears in the first text to the total number of times the first word appears in the first text.

[0250] The ratio between the total number of the first texts and the number of first texts containing the first word is determined to obtain the inverse document frequency of the first word;

[0251] The importance parameter of the first word is determined based on the occurrence frequency and the inverse document frequency.

[0252] Based on the tenth example of the first aspect, in the eleventh example of the first aspect, determining the importance parameter of the first word according to the occurrence frequency and the inverse document frequency includes:

[0253] The target logarithm is obtained by taking the logarithm of the inverse document frequency;

[0254] The product of the frequency of occurrence and the target logarithm is determined as the importance parameter of the first word.

[0255] Based on the first to sixth examples of the first aspect, in the twelfth example of the first aspect, generating a word cloud of the first text based on the first word includes:

[0256] The word cloud of the first text is generated based on the first word with a preset part of speech, wherein the preset part of speech includes at least one of the following: noun, verb, adjective, idiom, and proverb.

[0257] In a first example of the second aspect, a word cloud generation apparatus is provided, the apparatus comprising:

[0258] The word segmentation module is used to segment a first text into multiple first words using words in a dictionary. The dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence level threshold. The second word is pre-extracted from a second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and the confidence level is negatively correlated with the frequency of occurrence of each character in the second word in the second text.

[0259] The word cloud generation module is used to generate a word cloud of the first text based on the first word, and the word cloud is used to represent key information in the first text.

[0260] Based on the first example of the second aspect, in the second example of the second aspect, the confidence level is determined based on the ratio of the frequency of the second word in the second text to the target value;

[0261] When the second word includes two characters, the target value is the product of the frequencies of occurrence of the two characters in the second word in the second text.

[0262] When the second word includes at least three characters, the target value is the maximum of the first value and the second value. The first value is the product of the frequency of the first character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text. The second value is the product of the frequency of the last character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text.

[0263] Based on the first example of the second aspect, in the third example of the second aspect, the dictionary also includes a third word, at least two of the third words constituting a second word, and the first confidence threshold is pre-generated by the following module:

[0264] A confidence determination module is used to determine the confidence of each third word in the dictionary based on the confidence of the second word;

[0265] The first sequence generation module is used to sort the third word in descending order according to the confidence level of the third word to obtain a first sequence;

[0266] The first threshold determination module is used to determine the confidence level of the third word located at the first target position in the first sequence as the first confidence threshold, and the ratio of the number of third words before the first target position to the total number of third words in the first sequence corresponds to the first preset proportion.

[0267] Based on the third example of the second aspect, in the fourth example of the second aspect, the word cloud generation module is further used for:

[0268] If the confidence level of the fourth word formed by two adjacent first words in the first text is greater than or equal to the second confidence threshold, then a word cloud of the first text is generated based on the fourth word and the words other than the first words that constitute the fourth word, and the second confidence threshold is less than the first confidence threshold.

[0269] Based on the fourth example of the second aspect, in the fifth example of the second aspect, the word cloud generation module is further configured to: if the confidence level of the fifth word formed by the last first word constituting the fourth word and the next first word is greater than or equal to the second confidence level threshold, then generate a word cloud of the first text based on the fifth word and the first words other than the words constituting the fifth word.

[0270] Based on the fourth example of the second aspect, in the sixth example of the second aspect, the word cloud generation module is further used for:

[0271] If the confidence level of the sixth word formed by the first word constituting the fourth word and the preceding first word is greater than or equal to the second confidence level threshold, then a word cloud of the first text is generated based on the sixth word and the first words other than the words constituting the sixth word.

[0272] Based on examples four through six of the second aspect, in example seven of the second aspect, the second confidence threshold is pre-generated using the following module:

[0273] The second threshold determination module is used to determine the confidence level of the third word located at the second target position in the first sequence as the second confidence threshold. The ratio of the number of third words before the second target position to the total number of third words in the first sequence corresponds to a second preset proportion, and the second preset proportion is greater than the first preset proportion.

[0274] Based on the first to sixth examples of the second aspect, in the eighth example of the second aspect, the word cloud generation module is further used for:

[0275] Determine at least two similar words from a plurality of the first words, wherein the similar words satisfy at least one of the following conditions: semantic similarity is greater than or equal to a first preset threshold, and Lewinstein distance is less than or equal to a second preset threshold;

[0276] At least two similar words in the first word are merged into a target word, wherein the target word is one of the similar words that appears most frequently.

[0277] Generate a word cloud of the first text based on the target word and the remaining unmerged first words.

[0278] Based on the first to sixth examples of the second aspect, in the ninth example of the second aspect, the above-mentioned device further includes:

[0279] An importance determination module is used to determine the importance parameter of each first word for each first text if there are multiple first texts before generating a word cloud of the first text based on the first word. The importance parameter is used to represent the degree of importance of the first word in the first text.

[0280] The word cloud generation module is also used for:

[0281] A word cloud of the first text is generated based on the first words that satisfy the preset conditions according to the importance parameter. The preset conditions include at least one of the following: greater than or equal to a preset importance threshold, or located in a prominent position in a second sequence. The second sequence is obtained by arranging each first word in descending order according to the importance parameter.

[0282] Based on the ninth example of the second aspect, in the tenth example of the second aspect, the importance determination module is further configured to:

[0283] The frequency of the first word in the first text is obtained by determining the ratio of the number of times the first word appears in the first text to the total number of times the first word appears in the first text.

[0284] The ratio between the total number of the first texts and the number of first texts containing the first word is determined to obtain the inverse document frequency of the first word;

[0285] The importance parameter of the first word is determined based on the occurrence frequency and the inverse document frequency.

[0286] Based on the tenth example of the second aspect, in the eleventh example of the second aspect, the importance determination module is further configured to:

[0287] The target logarithm is obtained by taking the logarithm of the inverse document frequency;

[0288] The product of the frequency of occurrence and the target logarithm is determined as the importance parameter of the first word.

[0289] Based on the first to sixth examples of the second aspect, in the twelfth example of the second aspect, the word cloud generation module is further used for:

[0290] The word cloud of the first text is generated based on the first word with a preset part of speech, wherein the preset part of speech includes at least one of the following: noun, verb, adjective, idiom, and proverb.

[0291] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0292] The memory stores computer-executed instructions;

[0293] The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to implement the method described in any of the first aspects.

[0294] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, the computer-readable storage medium storing computer-executable instructions that, when executed by a processor, cause a computing device to implement the method described in any one of the first aspects.

[0295] Fifthly, according to one or more embodiments of the present disclosure, a computer program is provided for implementing the method described in any of the first aspects.

[0296] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0297] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0298] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A word cloud generation method, characterized in that, The method includes: The first text is segmented using words from a dictionary to obtain multiple first words. The dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence level threshold. The second word is pre-extracted from the second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and the confidence level is negatively correlated with the frequency of occurrence of each character in the second word in the second text. A word cloud is generated from the first word to represent the key information in the first text; The dictionary also includes a third word, and at least two of the third words constitute a second word. The first confidence threshold is generated in advance through the following steps: determining the confidence of the third word based on the confidence of the second word; The third words are sorted in descending order according to their confidence level to obtain the first sequence; The confidence level of the third word located at the first target position in the first sequence is determined as the first confidence threshold, and the ratio of the number of third words before the first target position to the total number of third words in the first sequence corresponds to the first preset proportion.

2. The method according to claim 1, characterized in that, The confidence level of the second word is determined based on the ratio of the frequency of the second word in the second text to the target value; When the second word includes two characters, the target value is the product of the frequencies of occurrence of the two characters in the second word in the second text. When the second word includes at least three characters, the target value is the maximum of the first value and the second value. The first value is the product of the frequency of the first character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text. The second value is the product of the frequency of the last character in the second word in the second text and the frequency of the segment formed by the remaining characters in the second word in the second text.

3. The method according to claim 1, characterized in that, The step of generating a word cloud of the first text based on the first word includes: If the confidence level of the fourth word formed by two adjacent first words in the first text is greater than or equal to the second confidence threshold, then a word cloud of the first text is generated based on the fourth word and the words other than the first words that constitute the fourth word, and the second confidence threshold is less than the first confidence threshold.

4. The method according to claim 3, characterized in that, The step of generating a word cloud of the first text based on the fourth word and words other than the first word constituting the fourth word includes: If the confidence level of the fifth word formed by the last first word constituting the fourth word and the next first word is greater than or equal to the second confidence level threshold, then a word cloud of the first text is generated based on the fifth word and the first words other than the words constituting the fifth word.

5. The method according to claim 3, characterized in that, The step of generating a word cloud of the first text based on the fourth word and words other than the first word constituting the fourth word includes: If the confidence level of the sixth word formed by the first word constituting the fourth word and the preceding first word is greater than or equal to the second confidence level threshold, then a word cloud of the first text is generated based on the sixth word and the first words other than the words constituting the sixth word.

6. The method according to any one of claims 3 to 5, characterized in that, The second confidence threshold is generated in advance through the following steps: The confidence level of the third word located at the second target position in the first sequence is determined as the second confidence threshold. The ratio of the number of third words before the second target position to the total number of third words in the first sequence corresponds to the second preset proportion, which is greater than the first preset proportion.

7. The method according to any one of claims 1 to 5, characterized in that, The step of generating a word cloud of the first text based on the first word includes: Determine at least two similar words from a plurality of the first words, wherein the similar words satisfy at least one of the following conditions: semantic similarity is greater than or equal to a first preset threshold, and Lewinstein distance is less than or equal to a second preset threshold; At least two similar words in the first word are merged into a target word, wherein the target word is one of the similar words that appears most frequently. Generate a word cloud of the first text based on the target word and the remaining unmerged first words.

8. The method according to any one of claims 1 to 5, characterized in that, Before generating the word cloud of the first text based on the first word, the method further includes: If there are multiple first texts, then for each first text, the importance parameter of each first word is determined based on the number of times the first word appears in the first text and the number of first texts containing the first word. The importance parameter is used to represent the degree of importance of the first word in the first text. The step of generating a word cloud of the first text based on the first word includes: A word cloud of the first text is generated based on the first words that satisfy the preset conditions according to the importance parameter. The preset conditions include at least one of the following: greater than or equal to a preset importance threshold, or located in a prominent position in a second sequence. The second sequence is obtained by arranging each first word in descending order according to the importance parameter.

9. The method according to claim 8, characterized in that, For each of the first texts, the importance parameter of each first word is determined based on the number of times the first word appears in the first text and the number of first texts containing the first word, including: The frequency of the first word in the first text is obtained by determining the ratio of the number of times the first word appears in the first text to the total number of times the first word appears in all the first texts. The ratio between the total number of the first texts and the number of first texts containing the first word is determined to obtain the inverse document frequency of the first word; The importance parameter of the first word is determined based on the occurrence frequency and the inverse document frequency.

10. The method according to claim 9, characterized in that, The step of determining the importance parameter of the first word based on the occurrence frequency and the inverse document frequency includes: The target logarithm is obtained by taking the logarithm of the inverse document frequency; The product of the frequency of occurrence and the target logarithm is determined as the importance parameter of the first word.

11. The method according to any one of claims 1 to 5, characterized in that, The step of generating a word cloud of the first text based on the first word includes: The word cloud of the first text is generated based on the first word with a preset part of speech, wherein the preset part of speech includes at least one of the following: noun, verb, adjective, idiom, or proverb.

12. A word cloud generation device, characterized in that, The device includes: The word segmentation module is used to segment a first text into multiple first words using words in a dictionary. The dictionary includes at least one second word of length with a confidence level greater than or equal to a first confidence level threshold. The second word is pre-extracted from a second text generated within a preset time period. The confidence level is positively correlated with the frequency of occurrence of the second word in the second text, and the confidence level is negatively correlated with the frequency of occurrence of each character in the second word in the second text. A word cloud generation module is used to generate a word cloud of the first text based on the first word, wherein the word cloud is used to represent key information in the first text; The dictionary also includes a third word, and at least two of the third words constitute a second word. The first confidence threshold is generated in advance through the following steps: determining the confidence of the third word based on the confidence of the second word; The third words are sorted in descending order according to their confidence level to obtain the first sequence; The confidence level of the third word located at the first target position in the first sequence is determined as the first confidence threshold, and the ratio of the number of third words before the first target position to the total number of third words in the first sequence corresponds to the first preset proportion.

13. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, cause a computing device to implement the method as described in any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, The computer program is used to implement the method as described in any one of claims 1 to 11.

Citation Information

Patent Citations

  • Full-text retrieval system based on improved IF-IDF algorithm

    CN110619036A

  • Text data processing method and device

    CN113779973A

  • Systems, methods and apparatus for relative frequency based phrase mining

    US20110035211A1