Intelligent word cloud display method and system based on machine learning and ES word cloud statistics

By introducing machine learning and ES word cloud statistics methods into traditional word cloud display technology, the shortcomings of deep mining of text semantics and themes in traditional technology are solved, and large-scale text data analysis and visualization with high expressiveness and real-time performance are achieved.

CN119884362AInactive Publication Date: 2025-04-25ZHEJIANG SHUXIN NETWORK CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202510368751.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-27
Publication Date
2025-04-25
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional word cloud display technology lacks deep exploration of text semantics and themes, resulting in weak expressiveness and interpretability of word cloud graphs, and it is difficult to meet real-time and lightweight application scenarios, and the threshold for use is also high.

Method used

An intelligent word cloud display method based on machine learning and ES word cloud statistics is adopted to generate word clouds with higher expressiveness by preprocessing, feature extraction, word frequency statistics and context analysis of text data.

Benefits of technology

It realizes comprehensive analysis and visual presentation of text content, improves the expressiveness and interpretability of word cloud display, lowers the threshold for using algorithms, and meets real-time and lightweight processing needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119884362A_ABST
    Figure CN119884362A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent word cloud display method based on machine learning and ES word cloud statistics, and belongs to the technical field of word cloud generation. The method comprises the steps of obtaining to-be-processed text data; preprocessing to obtain preprocessed text data; carrying out feature extraction to obtain keywords; word frequencies are obtained through statistics; performing statistics to obtain inverse document frequency; obtaining a basic weight according to the word frequency and the inverse document frequency; obtaining an adjustment factor according to the context of the keyword; obtaining a weight according to the basic weight and the adjustment factor; and generating a word cloud. According to the method, the powerful text retrieval and analysis capability of the ES is utilized, processing such as word frequency statistics, sentiment analysis and theme classification is rapidly carried out on mass text data, and the text is rapidly analyzed and processed. Therefore, comprehensive analysis and visual presentation of the text content are realized, expressive force and interpretability of word cloud display are improved, and a more intelligent, efficient and simple text visualization tool is provided for a user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of word cloud generation, and in particular to an intelligent word cloud display method and system based on machine learning and ES word cloud statistics. Background Art

[0002] With the continuous evolution of the big data era, people need to extract valuable information from massive amounts of text and display it visually. As a commonly used text visualization technology, word cloud forms an expressive image by displaying words that appear frequently in the text, which is used to provide an intuitive display of the text theme or focus. However, traditional word cloud display technology often only considers the frequency of words, lacks in-depth mining of text semantics and themes, resulting in word cloud images lacking expressiveness and interpretability. Therefore, how to use advanced technologies such as machine learning to achieve intelligent word cloud display has become a research hotspot.

[0003] Currently, there is a method for automatic text summarization and word cloud generation based on word vectors and convolutional neural networks on the market. This method extracts keywords and text summaries by training a convolutional neural network model, and visualizes the keywords in a word cloud according to the model's prediction results. However, this method requires a large amount of data training, is not suitable for real-time and lightweight application scenarios, and has a high threshold for use.

[0004] In the existing technology, word cloud display is mainly based on word frequency statistics, lacking in-depth mining of text meanings and themes, and has weak expressiveness in complex application scenarios. Traditional word cloud display is usually offline data processing, which cannot meet the requirements of real-time and lightweight. In addition, the traditional word cloud display technology has a high learning cost and a high threshold for use.

[0005] For example, if we need to conduct sentiment analysis and topic classification based on topics related to the XX event on Weibo, the traditional word frequency statistics method cannot fully mine the deep information and themes of the text content, which may lead to the omission or misjudgment of keywords. In addition, the traditional offline data processing method cannot update the data in time, and it is difficult to meet the requirements of real-time and lightweight. Summary of the invention

[0006] The purpose of the present invention is to provide an intelligent word cloud display method and system based on machine learning and ES word cloud statistics, which utilizes the text-word cloud algorithm of machine learning and the powerful text retrieval and analysis capabilities of ES to achieve an efficient, intelligent and easy-to-use text-word cloud visualization solution.

[0007] In order to solve the above technical problems, the present invention provides an intelligent word cloud display method based on machine learning and ES word cloud statistics, comprising the following steps: Get the text data to be processed; Preprocessing the text data to be processed to obtain preprocessed text data; Perform feature extraction on the preprocessed text data to obtain keywords; Count the keywords in the preprocessed text data to get the word frequency; Get a collection of documents; Count the documents containing keywords in the document collection to get the inverse document frequency; According to the word frequency and inverse document frequency, the basic weight is obtained; According to the context of the keyword, the adjustment factor is obtained; According to the basic weight and the adjustment factor, the weight is obtained; Generate a word cloud based on keywords, weights and word frequencies.

[0008] Preferably, preprocessing the text data to be processed to obtain preprocessed text data specifically includes the following steps: Perform data cleaning, word segmentation and stop word removal on the text data to be processed.

[0009] Preferably, data cleaning, word segmentation and stop word removal are performed on the text data to be processed, specifically including the following steps: Remove special characters, numbers, punctuation marks, and unnecessary blank characters from the text data to be processed, and perform capitalization unification to obtain cleaned text data; Use natural language processing technology to perform word segmentation on the cleaned text data to obtain the word segmentation results; Stop words that are common in the word segmentation results and contribute little to the meaning of the text are removed to obtain preprocessed text data.

[0010] Preferably, the calculation formula of the word frequency is: TF = (number of times the keyword appears in the preprocessed text data / total number of words in the preprocessed text data); The calculation formula of the inverse document frequency is: IDF = log (total number of documents in the document collection / number of documents containing the keyword) The calculation formula of the basic weight is: Yi = TF * IDF In the formula: TF is the term frequency; IDF is the inverse document frequency; Yi is the basic weight; The calculation formula of the weight is: For Wi = Yi * K Where: Wi is the weight; K is the adjustment factor.

[0011] Preferably, obtaining the adjustment factor according to the context of the keyword specifically includes the following steps: Count the degree words and negative words in the context of each keyword in the pre-processed text data; Determine the adjustment factor based on all degree words and negative words.

[0012] Preferably, the degree words include very, extremely, slightly and slightly, and each degree word corresponds to a coefficient; the coefficient of very is 1.5, the coefficient of extremely is 2, the coefficient of slightly is 0.8, and the coefficient of slightly is 0.7; The coefficient of the negative word is 0.1; The adjustment factor is the product of the coefficients of all degree words and negative words of the keyword in the pre-processed text data.

[0013] Preferably, feature extraction uses machine learning algorithms, including TF-IDF, Word2Vec, and BERT.

[0014] Preferably, generating a word cloud according to keywords, weights and word frequencies specifically includes the following steps: Based on a word cloud generation algorithm, a word cloud is generated according to keywords, weights and frequency information; the word cloud generation algorithm includes a WordCloud library.

[0015] Preferably, the method further comprises the following steps: Send the word cloud to the user interface for interaction.

[0016] The present invention also provides an intelligent word cloud display system based on machine learning and ES word cloud statistics, comprising: A first acquisition module, used to acquire text data to be processed; A preprocessing module, used for preprocessing the text data to be processed to obtain preprocessed text data; The feature extraction module is used to extract features from the preprocessed text data to obtain keywords; The keyword statistics module is used to count the keywords in the preprocessed text data to obtain the word frequency; The second acquisition module is used to acquire a document collection; The document statistics module is used to count the documents containing keywords in the document collection and obtain the inverse document frequency; The basic weight calculation module is used to obtain the basic weight according to the word frequency and inverse document frequency; An adjustment factor calculation module, used to obtain an adjustment factor according to the context of the keyword; The weight calculation module is used to obtain the weight according to the basic weight and the adjustment factor; The word cloud generation module is used to generate word clouds based on keywords, weights and word frequencies.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention utilizes the powerful text retrieval and analysis capabilities of ES to quickly perform word frequency statistics, sentiment analysis, topic classification and other processing on massive text data, and quickly analyze and process the text. In addition, the machine learning algorithm automatically extracts key information from the text and generates vocabulary that reflects the relevance of the text content, avoiding the process of manual processing of annotation and screening in traditional word cloud technology, making the algorithm more real-time and lightweight. This enables comprehensive analysis and visual presentation of text content, improves the expressiveness and interpretability of word cloud display, and provides users with a more intelligent, efficient and simple text visualization tool. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0019] Figure 1 It is a flow chart of an intelligent word cloud display method based on machine learning and ES word cloud statistics of the present invention. DETAILED DESCRIPTION

[0020] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0021] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0022] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0023] The present invention is further described in detail below in conjunction with the accompanying drawings: like Figure 1 As shown, the present invention provides an intelligent word cloud display method based on machine learning and ES word cloud statistics, comprising the following steps: Get the text data to be processed; Preprocessing the text data to be processed to obtain preprocessed text data; Perform feature extraction on the preprocessed text data to obtain keywords; Count the keywords in the preprocessed text data to get the word frequency; Get a collection of documents; Count the documents containing keywords in the document collection to get the inverse document frequency; According to the word frequency and inverse document frequency, the basic weight is obtained; According to the context of the keyword, the adjustment factor is obtained; According to the basic weight and the adjustment factor, the weight is obtained; Generate a word cloud based on keywords, weights and word frequencies.

[0024] Preferably, preprocessing the text data to be processed to obtain preprocessed text data specifically includes the following steps: Perform data cleaning, word segmentation and stop word removal on the text data to be processed.

[0025] Preferably, data cleaning, word segmentation and stop word removal are performed on the text data to be processed, specifically including the following steps: Remove special characters, numbers, punctuation marks, and unnecessary blank characters from the text data to be processed, and perform capitalization unification to obtain cleaned text data; Use natural language processing technology to perform word segmentation on the cleaned text data to obtain the word segmentation results; Stop words that are common in the word segmentation results and contribute little to the meaning of the text are removed to obtain preprocessed text data.

[0026] Preferably, the calculation formula of the word frequency is: TF = (number of times the keyword appears in the preprocessed text data / total number of words in the preprocessed text data); The calculation formula of the inverse document frequency is: IDF = log (total number of documents in the document collection / number of documents containing the keyword) The calculation formula of the basic weight is: Yi = TF * IDF In the formula: TF is the term frequency; IDF is the inverse document frequency; Yi is the basic weight; The calculation formula of the weight is: For Wi = Yi * K Where: Wi is the weight; K is the adjustment factor.

[0027] Preferably, obtaining the adjustment factor according to the context of the keyword specifically includes the following steps: Count the degree words and negative words in the context of each keyword in the pre-processed text data; Determine the adjustment factor based on all degree words and negative words.

[0028] Preferably, the degree words include very, extremely, slightly and slightly, and each degree word corresponds to a coefficient; the coefficient of very is 1.5, the coefficient of extremely is 2, the coefficient of slightly is 0.8, and the coefficient of slightly is 0.7; The coefficient of the negative word is 0.1; The adjustment factor is the product of the coefficients of all degree words and negative words of the keyword in the pre-processed text data.

[0029] Preferably, feature extraction uses machine learning algorithms, including TF-IDF, Word2Vec, and BERT.

[0030] Preferably, generating a word cloud according to keywords, weights and word frequencies specifically includes the following steps: Based on a word cloud generation algorithm, a word cloud is generated according to keywords, weights and frequency information; the word cloud generation algorithm includes a WordCloud library.

[0031] Preferably, the method further comprises the following steps: Send the word cloud to the user interface for interaction.

[0032] The present invention also provides an intelligent word cloud display system based on machine learning and ES word cloud statistics, comprising: A first acquisition module, used to acquire text data to be processed; A preprocessing module, used for preprocessing the text data to be processed to obtain preprocessed text data; The feature extraction module is used to extract features from the preprocessed text data to obtain keywords; The keyword statistics module is used to count the keywords in the preprocessed text data to obtain the word frequency; The second acquisition module is used to acquire a document collection; The document statistics module is used to count the documents containing keywords in the document collection and obtain the inverse document frequency; The basic weight calculation module is used to obtain the basic weight according to the word frequency and inverse document frequency; An adjustment factor calculation module, used to obtain an adjustment factor according to the context of the keyword; A weight calculation module, configured to obtain a weight according to a base weight and an adjustment factor; A word cloud generation module, configured to generate a word cloud according to keywords, weights, and word frequencies.

[0033] Aiming at the shortcomings in the prior art, the present invention adopts an intelligent text analysis technology combining ES and machine learning algorithms, which can automatically perform sentiment analysis and topic classification on a large amount of review data, so as to quickly identify the sentiment tendency and main topics of the reviews. Specifically, a sentiment classification model can be automatically trained by means of machine learning algorithms to perform automated sentiment analysis on the reviews. At the same time, topic models such as machine learning - LDA are used to perform topic modeling on relevant reviews, so as to discover the main topics hidden in the reviews. The extracted data is stored in ES. When a user queries a relevant topic, ES can be connected through the Flask or Django framework of Python to retrieve data in real time and perform visual presentation, so as to achieve accurate word cloud display.

[0034] To better illustrate the technical effects of the present invention, the following specific embodiments are provided to illustrate the above technical processes: Embodiment 1. The present invention proposes an intelligent word cloud display method based on machine learning and ES word cloud statistics, which specifically includes the following steps: I. Text data preprocessing 1. Clean the original text data (text data to be processed): This step includes removing special characters, numbers, punctuation marks, and unnecessary blank characters in the text to ensure the cleanliness of the text data. At the same time, the text is unified in case to facilitate the consistency of subsequent processing.

[0035] 2. Word segmentation operation: Adopt advanced natural language processing technology to perform accurate word segmentation on the cleaned text, and decompose the text into meaningful word units.

[0036] 3. Remove stop words: In the word segmentation results, remove common stop words that contribute less to the meaning of the text, such as "de", "he", "shi", etc., to reduce noise and focus on key information.

[0037] II. Feature extraction 1. Utilize machine learning algorithms: The present invention adopts advanced machine learning algorithms such as TF-IDF (term frequency - inverse document frequency), Word2Vec, or BERT to perform in-depth feature extraction on the preprocessed text data and identify the keywords in the text.

[0038] 2. Keywords and their weights: The algorithm will output the importance weights of each keyword, and these weights will be used in the subsequent word cloud statistics and generation process.

[0039] Wi=Yi*K Where: Wi is the weight; Yi is the basic weight and K is the context adjustment factor. According to each degree word in the context of each keyword, the K value is adjusted according to its strength: The base weight (Yi) is the initial importance weight of the keyword in the text: Formula: Yi = TF * IDF Term frequency (TF): The frequency with which a keyword appears in the current document: TF = (number of times the keyword appears in the document) / (total number of words in the document).

[0040] Inverse Document Frequency (IDF): The rarity of a keyword in the entire document collection: IDF = log(total number of documents / number of documents containing the keyword); The importance of keywords is calculated based on their statistical properties in the corpus (such as mutual information, chi-square test, etc.).

[0041] The degree words include very, extremely, slightly and slightly, and each level of the degree word corresponds to a coefficient; the coefficient of very is 1.5, the coefficient of extremely is 2, the coefficient of slightly is 0.8, and the coefficient of slightly is 0.7; "Very": K = K * 1.5 "Extreme": K = K * 2.0 “Slightly”: K = K * 0.8 “Slightly”: K = K * 0.7 If negative words appear in the following text of a keyword, the K value will decrease, indicating that the importance of the keyword is reduced; negative words include "not", and the coefficient is 0.1.

[0042] For example: "Very": This dress is totally unsuitable for me; the color is too bright.

[0043] "extremely": She was extremely reluctant to attend the party because there were too many people and it was too noisy.

[0044] "a little": He was a little dissatisfied with the result, but he could accept it.

[0045] “Slightly”: Her tone was slightly dissatisfied, and she was obviously not very satisfied with the discussion just now.

[0046] “No”: K = K * 0.1 The adjustment factor is the product of the coefficients of all degree words and negative words of the keywords in the preprocessed text data.

[0047] Examples: Suppose there is a document collection with a total of 1000 documents, of which 100 contain the keyword "good". In the current document, "good" appears 5 times, "extremely" appears once before "good", "slightly" appears once, and the negative word "not" appears once after "slightly", so the total number of "good" words in the document is 200.

[0048] Calculate TF: TF = 5 / 200 = 0.025 Calculate IDF: IDF = log(1000 / 100) = log(10) ≈ 2.302 Calculate Yi: Yi = TF * IDF = 0.025 * 2.302 ≈ 0.0575 Therefore, the basic weight Yi of the keyword "good" ≈ 0.0575. "Extremely": K = K * 2.0, "Slightly": K = K * 0.8; "Not": K = K * 0.1 Adjustment factor K = 2.0 * 0.8 * 0.1 = 0.16 The final weight Wi is calculated as follows: Wi = Yi * K = 0.0575 * 0.16 ≈ 0.0092 3. ES word cloud statistics 1. Import into ES search engine: Import the keywords and weights obtained from feature extraction into the Elasticsearch search engine, and use its data processing capabilities to perform word cloud statistics.

[0049] 2. Obtain frequency information: Through the aggregation analysis function of ES, the frequency of occurrence of keywords in text data is counted to provide data support for word cloud generation; the frequency is the sum of the weights of all keywords.

[0050] 4. Word Cloud Generation 1. Use word cloud generation algorithm: According to the weight and frequency information of keywords, use a specific word cloud generation algorithm, such as the WordCloud library, to generate a word cloud.

[0051] 2. Personalized and intelligent features: The generated word cloud can not only reflect the key topics of the text data, but also be personalized according to the user's preferences, such as font size, color, shape, etc.

[0052] 5. User Interaction 1. Providing a user interface: The present invention provides an intuitive user interface that allows users to perform interactive operations.

[0053] 2. Custom word cloud style: Users can customize the visual style of the word cloud, including color scheme, font type, layout mode, etc.

[0054] 3. Filter keywords: Users can filter specific keywords as needed to highlight or hide certain information and achieve personalized word cloud display.

[0055] The intelligent word cloud display method of the present invention can achieve the following effects: 1. Efficient text data preprocessing technology: including operations such as cleaning, word segmentation and removal of stop words of raw text data. These steps are the basis for ensuring the accuracy of subsequent processing.

[0056] 2. Advanced feature extraction algorithms: Use machine learning algorithms such as TF-IDF, Word2Vec or BERT to extract features from text data and identify keywords and their weights.

[0057] 3. Word cloud statistics of ES search engine: Use Elasticsearch search engine to perform keyword frequency statistics, which improves the speed and efficiency of processing large amounts of text data.

[0058] 4. Personalized word cloud generation algorithm: Generate a word cloud with intelligent and personalized characteristics based on the weight and frequency information of keywords.

[0059] 5. User-friendly interactive interface: Provides a user interface that allows users to customize word cloud styles and filter keywords to enhance user experience.

[0060] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules, modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0061] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0062] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0063] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above.

[0064] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0065] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An intelligent word cloud display method based on machine learning and ES word cloud statistics, characterized in that: The following steps are involved: Get the text data to be processed; Preprocessing the text data to be processed to obtain preprocessed text data; Perform feature extraction on the preprocessed text data to obtain keywords; Count the keywords in the preprocessed text data to get the word frequency; Get a collection of documents; Count the documents containing keywords in the document collection to get the inverse document frequency; According to the word frequency and inverse document frequency, the basic weight is obtained; According to the context of the keyword, the adjustment factor is obtained; According to the basic weight and the adjustment factor, the weight is obtained; Generate a word cloud based on keywords, weights and word frequencies.

2. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 1 is characterized in that: Preprocessing the text data to be processed to obtain preprocessed text data specifically includes the following steps: Perform data cleaning, word segmentation and stop word removal on the text data to be processed.

3. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 2 is characterized in that: The text data to be processed is cleaned, segmented, and stop words are removed, including the following steps: Remove special characters, numbers, punctuation marks, and unnecessary blank characters from the text data to be processed, and perform capitalization unification to obtain cleaned text data; Use natural language processing technology to perform word segmentation on the cleaned text data to obtain the word segmentation results; Stop words that are common in the word segmentation results and contribute little to the meaning of the text are removed to obtain preprocessed text data.

4. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 3 is characterized in that: The calculation formula of the word frequency is: TF = (number of times the keyword appears in the preprocessed text data / total number of words in the preprocessed text data); The calculation formula of the inverse document frequency is: IDF = log (total number of documents in the document collection / number of documents containing the keyword) The calculation formula of the basic weight is: Yi = TF * IDF In the formula: TF is the term frequency; IDF is the inverse document frequency; Yi is the basic weight; The calculation formula of the weight is: For Wi = Yi * K Where: Wi is the weight; K is the adjustment factor.

5. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 4 is characterized in that: According to the context of the keyword, the adjustment factor is obtained, which specifically includes the following steps: Count the degree words and negative words in the context of each keyword in the pre-processed text data; Determine the adjustment factor based on all degree words and negative words.

6. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 5 is characterized in that: The degree words include very, extremely, slightly and slightly, and each degree word corresponds to a coefficient; the coefficient of very is 1.5, the coefficient of extremely is 2, the coefficient of slightly is 0.8, and the coefficient of slightly is 0.7; The coefficient of the negative word is 0.1; The adjustment factor is the product of the coefficients of all degree words and negative words of the keyword in the pre-processed text data.

7. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 6 is characterized in that: Feature extraction uses machine learning algorithms, including TF-IDF, Word2Vec, and BERT.

8. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 7 is characterized in that: Generate a word cloud based on keywords, weights and word frequencies, which includes the following steps: Based on a word cloud generation algorithm, a word cloud is generated according to keywords, weights and frequency information; the word cloud generation algorithm includes a WordCloud library.

9. The intelligent word cloud display method based on machine learning and ES word cloud statistics according to claim 8 is characterized in that: The following steps are also included: Send the word cloud to the user interface for interaction.

10. An intelligent word cloud display system based on machine learning and ES word cloud statistics, used to implement the intelligent word cloud display method based on machine learning and ES word cloud statistics as described in any one of claims 1 to 9, characterized in that: include: A first acquisition module, used to acquire text data to be processed; A preprocessing module, used for preprocessing the text data to be processed to obtain preprocessed text data; The feature extraction module is used to extract features from the preprocessed text data to obtain keywords; The keyword statistics module is used to count the keywords in the preprocessed text data to obtain the word frequency; The second acquisition module is used to acquire a document collection; The document statistics module is used to count the documents containing keywords in the document collection and obtain the inverse document frequency; The basic weight calculation module is used to obtain the basic weight according to the word frequency and inverse document frequency; An adjustment factor calculation module, used to obtain an adjustment factor according to the context of the keyword; The weight calculation module is used to obtain the weight according to the basic weight and the adjustment factor; The word cloud generation module is used to generate word clouds based on keywords, weights and word frequencies.

Citation Information

Patent Citations

  • Data analysis method and device based on NLP technology

    CN109977399A

  • Video word cloud generation method and device, storage medium and electronic equipment

    CN113850178A

  • Method for creating structured document based on deep learning model

    CN119558307A