A micro-blog negative comment data analysis method based on cosine similarity-C4.5 decision tree

Abstract

This invention provides a method for analyzing negative Weibo comment data based on a cosine similarity-C4.5 decision tree, comprising: Step S1, preprocessing the original Weibo comment data, identifying whether the comments contain keywords to form a complete descriptive attribute set using the AC automaton algorithm, and labeling the influence category based on the number of comment interactions; Step S2, using the AC automaton algorithm to perform multi-keyword fast matching on the comment text; Step S3, based on the improved C4.5 algorithm, first merging attribute values ​​with similar information entropy using cosine similarity measurement, then calculating the information gain rate of each attribute and recursively constructing a decision tree; Step S4, using a pessimistic error pruning algorithm based on confidence interval estimation to perform post-pruning on the decision tree and extract classification rules. This invention improves retrieval efficiency through the AC automaton, reduces model redundancy through cosine similarity merging, and enhances generalization ability through pessimistic pruning. The three synergistically achieve a simultaneous improvement in classification accuracy, model simplicity, and processing efficiency.

Description

Technical Field

[0001] This invention relates to the technical field of big data analysis, and more specifically, to a method for analyzing negative comments on Weibo based on a cosine similarity-C4.5 decision tree. Background Technology

[0002] As a mainstream social media platform, Weibo generates a massive amount of user comments daily. The timely detection and impact assessment of negative comments are crucial for public opinion monitoring, government decision-making, and corporate crisis management. Existing Weibo negative comment analysis technologies are mainly divided into three categories: Keyword-based rule filtering: Comments are quickly filtered by a pre-set sensitive word library. This method is fast, but it is difficult to distinguish negative comments with different influences. In addition, traditional string matching (such as KMP, regular expressions) has a performance bottleneck when the number of keywords is large, with a time complexity of O(m×n), which cannot meet the needs of real-time processing. Machine learning-based classification methods include Naive Bayes, Support Vector Machines (SVM), and decision trees (C4.5, etc.). Among them, C4.5 decision trees are widely used due to their strong interpretability and ability to handle discrete attributes, but their direct application to Weibo data has the following drawbacks: Redundant attribute values: Weibo comments cover a wide range of fields, and overly detailed attribute value division leads to redundant decision tree branches and complex rules. For example, comments in the categories of "entertainment" and "lifestyle" may have highly similar influence distributions, but the C4.5 algorithm does not consider attribute value merging, resulting in a bloated model. Overfitting problem: There are a large number of extreme negative comments on Weibo (such as malicious insults). These isolated data points can easily lead to over-segmentation of decision trees, reducing generalization ability. Traditional pruning methods (such as pre-pruning and validation set-based pruning) are not sensitive to extreme data. Deep learning-based text classification, such as BERT and TextCNN, has high classification accuracy, but the models are complex, consume a lot of computing resources, and lack interpretability, making them unsuitable for real-time processing and public opinion analysis scenarios.

[0003] In recent years, some researchers have attempted to use cosine similarity for feature selection or clustering, but there have been no public reports of combining it with C4.5 decision trees to solve the attribute value merging problem. As an efficient multi-pattern matching algorithm, AC automata has been widely used in keyword filtering, but its integrated application with decision tree classification is still a blank.

[0004] In summary, existing technologies lack a method for analyzing negative comments on Weibo that can simplify the model structure, improve processing efficiency, and effectively suppress extreme data interference while ensuring classification accuracy. Summary of the Invention

[0005] The technical problem this invention aims to solve is how to simplify the model structure, improve processing efficiency, and effectively suppress extreme data interference while ensuring classification accuracy. To overcome the shortcomings of the existing technology (or related technology), this invention provides a method for analyzing negative comments on Weibo based on a cosine similarity-C4.5 decision tree.

[0006] This invention provides a method for analyzing negative comments on Weibo based on a cosine similarity-C4.5 decision tree, comprising: Step S1: Clean and filter the original Weibo comment data, extract user type, comment field, and number of negative comments to form a descriptive attribute set; at the same time, use the AC automaton algorithm to quickly match multiple keywords to obtain the attribute value of whether the keyword is contained and add it to the descriptive attribute set; according to whether the sum of the number of likes, reposts and comments of the comment text exceeds the preset influence threshold, label each comment text with a category tag; Step S2: For each descriptive attribute, map the data subset corresponding to each attribute value into a vector with the number of samples in each category as the component, calculate the information entropy of each data subset, and when the difference between the information entropy of two attribute values ​​is less than the first preset threshold and the cosine similarity of the corresponding vector is greater than the second preset threshold, merge the two attribute values ​​into a new attribute value. Step S3: Calculate the information gain ratio of each descriptive attribute after merging, select the descriptive attribute with the largest information gain ratio as the internal node of the decision tree, and recursively divide the descriptive attribute set to generate the decision tree. Step S4: Traverse the internal nodes of the decision tree from bottom to top. Based on the number of samples in the node, the actual error rate, and the standard deviation corresponding to the preset confidence level, calculate the pessimistic error rate estimate of each internal node. If the pessimistic error rate estimate is not greater than the preset error rate threshold, then prune the internal node and its subtree and convert it into a leaf node. Set the category label to the category with the most samples in the internal node. After pruning, extract the classification rules in IF-THEN form from the decision tree.

[0007] The microblogging negative comment data analysis method based on cosine similarity-C4.5 decision tree proposed in this invention has the following advantages compared with existing technologies: This invention enables rapid multi-keyword matching through the AC automaton algorithm, significantly improving retrieval efficiency and reducing time complexity. It introduces cosine similarity measurement into the decision tree construction, automatically merging attribute values ​​with similar information entropy, reducing redundant branches, and simplifying the model structure. A pessimistic error pruning algorithm is employed to eliminate overfitting caused by extreme negative comment data, enhancing the model's generalization ability and stability. Overall, it addresses the shortcomings of traditional methods in training set construction, data redundancy handling, extreme data interference, and keyword retrieval, achieving efficient, accurate, and stable Weibo negative comment classification. This simplifies the model structure, improves processing efficiency, and effectively suppresses extreme data interference while maintaining classification accuracy.

[0008] In one possible implementation, in step S1, the user type is determined based on the number of followers, including ordinary users and users with a fan base; the field of the comments is automatically identified through a text classification model, including military and political, entertainment, and lifestyle; the number of negative comments is determined by comparing the number of negative comments posted by the user in the past with a preset threshold.

[0009] Compared with existing technologies, the above technical solution can classify users based on the number of followers, distinguishing between ordinary users and users with a follower base, making the descriptive attributes more distinctive; it can automatically identify the fields involved in comments using a text classification model, improving the automation and accuracy of descriptive attribute extraction; and it can construct a comprehensive and quantifiable set of descriptive attributes by combining the number of historical negative reviews and keyword matching results, laying a good foundation for subsequent classification.

[0010] In one possible implementation, step S1, the AC automaton algorithm includes: constructing a Trie tree with preset sensitive keywords and abusive words, where all keywords share a common prefix; constructing a mismatch pointer for each node using breadth-first traversal, pointing to the node with the longest common prefix and suffix; matching each comment text character by character, jumping along the mismatch pointer when a match fails, until a match is successful or the root node is returned; and recording the matched keyword when a marker node is reached.

[0011] Compared with existing technologies, the above technical solution can use a Trie tree structure to share common keyword prefixes, saving storage space and supporting large-scale keyword sets; it can quickly jump when a match fails through mismatch pointers, avoiding backtracking, so that all keywords can be found in a single scan; the time complexity is reduced from O(m×n) of the traditional method to O(m+n) (where m is the text length and n is the total length of keywords), which significantly improves retrieval efficiency and meets the needs of real-time processing of microblog data.

[0012] In one possible implementation, in step S2, the first preset threshold value ranges from 0.05 to 0.10, and the second preset threshold value ranges from 0.85 to 0.95.

[0013] In one possible implementation, in step S3, the information gain ratio is calculated as follows: first, the category information entropy describing the entire attribute set is calculated, then the information entropy of each data subset is calculated according to the attribute and the conditional entropy is obtained by weighted summation. The information gain is the difference between the two, the split information measure is the entropy of the attribute value, and the information gain ratio is the ratio of information gain to split information measure.

[0014] Compared with existing technologies, the above technical solution can overcome the shortcomings of information gain bias towards multi-valued attributes by calculating the information gain ratio, ensuring that the attribute with the best classification ability is selected as the split node; it provides the basis for the calculation of information entropy for subsequent cosine similarity merging attribute values, so that the merging decision is based on the objective judgment of similar information content; this calculation process is the core of the C4.5 algorithm, ensuring the accuracy and rationality of decision tree classification.

[0015] In one possible implementation, the formula for calculating the pessimistic error rate estimate in step S4 is: in, This represents a pessimistic error rate estimate; This represents the actual error rate of the node; Represents a node The number of samples included; This represents the standard normal distribution quantile corresponding to the confidence level c, which ranges from 0.25 to 0.5. The corresponding z-value is determined by statistical lookup tables. The preset error rate threshold C ranges from 0.4 to 0.6, and the value is set according to the tolerance for error rate in actual application.

[0016] In one possible implementation, after extracting the fractional rules in step S4, the method further includes applying the fractional rules to automatically classify new comment data, outputting the classification results and corresponding rule paths for the public opinion monitoring system to perform subsequent processing and early warning.

[0017] Compared with existing technologies, the above technical solution can extract classification rules in the form of IF-THEN with high interpretability, which makes it easier for public opinion analysts to understand and verify the classification basis. Applying the classification rules to new comment data can realize automatic classification and real-time early warning, improving the response speed and processing capacity of the public opinion monitoring system. Outputting rule paths helps to trace the reasons for classification and provides a basis for subsequent manual intervention or strategy adjustment.

[0018] In one possible implementation, the labeling method of category tags in step S1 also includes multi-classification setting, which divides the comment text into multiple levels such as high influence, medium influence, and low influence according to the degree of influence. At this time, the vector dimension corresponding to each attribute value is expanded to the number of categories, and the calculation of information entropy and information gain rate is adjusted to a multi-class form accordingly.

[0019] Compared with existing technologies, the above-mentioned technical solution can achieve more granular public opinion classification, meeting the requirements of refined management in different application scenarios. For example, high-impact comments can be dealt with immediately, medium-impact comments can be continuously observed, and low-impact comments can be ignored. The multi-classification setting provides richer training targets for subsequent machine learning models, which can further improve the practicality and flexibility of classification. Attached Figure Description

[0020] Figure 1 This is a flowchart of the steps of the present invention. Detailed Implementation

[0021] First, those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0022] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0023] See Figure 1 This invention discloses a method for analyzing negative comments on Weibo based on a cosine similarity-C4.5 decision tree, including: Step S1: Clean and filter the original Weibo comment data, extract user type, comment field, and number of negative comments to form a descriptive attribute set; at the same time, use the AC automaton algorithm to quickly match multiple keywords to obtain the attribute value of whether the keyword is contained and add it to the descriptive attribute set; according to whether the sum of the number of likes, reposts and comments of the comment text exceeds the preset influence threshold, label each comment text with a category tag; Step S2: For each descriptive attribute, map the data subset corresponding to each attribute value into a vector with the number of samples in each category as the component, calculate the information entropy of each data subset, and when the difference between the information entropy of two attribute values ​​is less than the first preset threshold and the cosine similarity of the corresponding vector is greater than the second preset threshold, merge the two attribute values ​​into a new attribute value. Step S3: Calculate the information gain ratio of each descriptive attribute after merging, select the descriptive attribute with the largest information gain ratio as the internal node of the decision tree, and recursively divide the descriptive attribute set to generate the decision tree. Step S4: Traverse the internal nodes of the decision tree from bottom to top. Based on the number of samples in the node, the actual error rate, and the standard deviation corresponding to the preset confidence level, calculate the pessimistic error rate estimate of each internal node. If the pessimistic error rate estimate is not greater than the preset error rate threshold, then prune the internal node and its subtree and convert it into a leaf node. Set the category label to the category with the most samples in the internal node. After pruning, extract the classification rules in IF-THEN form from the decision tree.

[0024] In this embodiment of the invention, step S1 specifically includes: Data filtering: Filter comments from the raw Weibo comment data that were posted within a preset validity period (e.g., the last 30 days) and whose users have passed real-name authentication to ensure data authenticity and timeliness; Attribute extraction: User type (A1): Determined by the number of followers. Users with more than 1,000 followers are considered "users with a fan base", otherwise they are considered "ordinary users". The comment covers the following areas (A2): It is identified as "military and political", "entertainment" and "lifestyle" by a pre-trained lightweight text classification model (such as Naive Bayes based on TF-IDF); Number of negative comments posted (A3): This counts the number of negative comments posted by the user in historical data (which can be combined with keyword matching or sentiment analysis), and compares them with a preset threshold d (usually 5~10), categorized as ">d" or "≤d". AC automaton keyword recognition: A Trie tree is constructed using preset sensitive keywords and abusive words (such as 500-1000 words). Each node represents a character, the root node is empty, and all keywords share a common prefix. A breadth-first traversal is used to construct a fail pointer for each node, so that when a match fails, the node can quickly jump to the next possible matching position. Scan each comment text character by character, starting from the root node. If a match is found, move down; otherwise, jump along the fail pointer. When a marker node is reached, record the keyword. All keywords can be identified in one scan, with a time complexity of O(m+n). The matching result is used as attribute A4: "whether it contains the keyword", with a value of "contains" or "does not contain". In this case, the complete description attribute set is {A1, A2, A3, A4}. Category labeling: Calculate the sum of likes, shares, and comments for the comment text. If it exceeds the preset influence threshold n (usually 50~100), it is labeled as a high-influence category (C1); otherwise, it is labeled as a low-influence category (C2).

[0025] In this embodiment of the invention, step S1 involves using the AC automaton algorithm to construct a Trie tree and mismatch pointers, performing multi-keyword fast matching on each comment text, and determining whether it contains the attribute value (A4) of the keyword. The specific implementation includes: Trie tree construction: A Trie tree is constructed using a preset set of sensitive keywords and abusive words (e.g., 500-1000 words). Each node represents a character, the root node is empty, all keywords share a common prefix, and the last node of each keyword is marked as the output node, storing the keyword information. Mismatch pointer construction: A breadth-first traversal algorithm is used to construct a mismatch pointer (fail pointer) for each node. The root node's fail pointer points to itself. For other nodes, if the parent node reaches the current node through the character 'c', the current node's fail pointer points to the longest suffix node that the parent node's fail pointer can reach through the character 'c'. If it does not exist, it points to the root node. The mismatch pointer allows for a quick jump to the next possible matching position when a match fails. Matching process: Scan each comment text character by character, starting from the root node. If the current character matches successfully, move down the tree edge; if the match fails, jump along the mismatch pointer until a match is successful or return to the root node. When a marker node is reached, record the matched keyword and its position. All keywords can be identified in one scan, with a time complexity of O(m+n) (where m is the text length and n is the total length of the keywords), which is a significant improvement in efficiency compared to the traditional string matching O(m×n).

[0026] In this embodiment of the invention, step S3 involves calculating the information gain ratio of each descriptive attribute based on the improved C4.5 algorithm, selecting the optimal attribute as the decision node, and introducing a cosine similarity measurement mechanism during the construction process to identify and merge attribute value pairs with similar information entropy to simplify the decision tree structure. Specifically, this includes the following sub-steps: Calculate the category information entropy: Suppose a dataset T contains N comments, where N1 are samples of class C1 and N2 are samples of class C2, then the category information entropy is... ; Calculate the conditional entropy of each attribute: For each descriptive attribute A k Based on its attribute values, T is divided into multiple data subsets. Calculate the information entropy of each data subset. Then, the conditional entropy is obtained by weighted summation. ; Calculate information gain: ; Calculate the split information metric: It is used to measure the distribution of values ​​of the attribute itself; Calculate the information gain ratio: The descriptive attribute with the highest information gain ratio is selected as the splitting attribute of the current node.

[0027] In this embodiment of the invention, step S3 introduces cosine similarity to merge attribute values. Before dividing attribute values, the similarity of the attribute values ​​of each attribute is determined and merged to reduce the number of branches. The specific process is as follows: Map the data subset corresponding to each attribute value to a vector. The components of the vector are the number of samples of each category in the data subset. For example, if the "military and political" subset describing attribute A2 contains 60 C1 samples and 40 C2 samples, then the vector is [60,40]. Calculate the information entropy of the data subset corresponding to each attribute value. If the difference between the information entropy of two attribute values ​​is less than the preset threshold ε (usually taken as 0.05~0.10), they are considered to carry similar amounts of information, and proceed to the next step of similarity calculation. Calculate the cosine of the angle between two vectors: If the cosine value is greater than the preset similarity threshold θ (usually taken as 0.85~0.95), the two attribute values ​​will be merged into a new attribute value. The merged new attribute value vector is the sum of the components of the original vector, that is... The value space of the original attributes is simplified accordingly, and the number of branches in the subsequent decision tree is reduced. Repeat the above steps to recursively build the decision tree until all leaf nodes meet the purity requirement (e.g., all samples belong to the same category) or can no longer split.

[0028] In this embodiment of the invention, in step S4, a pessimistic error pruning algorithm is used to perform post-pruning on the generated decision tree to eliminate overfitting caused by extreme data and extract classification rules. The specific implementation is as follows: Pessimistic error rate estimation: Traverse each internal node of the decision tree from bottom to top. For node v, assume it contains... There are samples, of which the number of misclassified samples is . (i.e., the number of samples in this node that do not belong to the majority class), the actual error rate f = / Based on the statistical confidence interval theory, the pessimistic error rate estimate of the node is calculated as follows: Where z is the standard normal distribution quantile corresponding to the pre-set confidence level c, this formula gives an upper bound estimate of the actual error rate under the pre-set confidence level c; Pruning decision: Set an error rate threshold C (usually 0.4~0.6). If q ≤ C, it means that the node and its subtree have limited contribution to classification or may even introduce overfitting. Then, prune the node and its subtree, convert the node into a leaf node, and set its class label to the class with the most samples in the node. If q > C, then keep the node and its subtree and continue traversing upwards. Pruning strategy: Starting from the internal nodes at the bottom of the decision tree, pruning decisions are made layer by layer from bottom to top to ensure that end nodes that are susceptible to noise are processed first. Rule extraction: After pruning, classification rules in the form of IF-THEN are extracted from the optimized decision tree. Each rule corresponds to a path from the root node to the leaf node. The splitting conditions on the path are connected by logical AND. The category of the leaf node is the rule conclusion. For example: "IF (whether it contains keywords = contains) AND (user type = ordinary user) AND (number of negative comments > 5) THEN category = C1". The extracted rules can be used for automatic classification of new data, and the rule paths are output for the public opinion monitoring system to perform subsequent processing and early warning.

[0029] In this embodiment of the invention, step S1 may also include multi-classification settings: the labeling method of category tags can be expanded to multi-classification, and the comment text can be divided into multiple levels such as high influence (C1), medium influence (C2), and low influence (C3) according to the degree of influence, so as to support more granular public opinion analysis. At this time, the calculation of information entropy, information gain, etc. need to be adjusted to multi-class form accordingly.

[0030] In this embodiment of the invention, to more clearly illustrate the invention, the complete implementation process is described below with reference to a specific example: Scenario Setting: The Cyberspace Administration of a certain city needs to analyze negative comments on the topic of "urban traffic management" on Weibo. There are 1,000 comments from January 2024. The parameters are set as follows: influence threshold n=50 (more than 50 likes + reposts + comments are considered high influence), negative comment count threshold d=5, information entropy threshold ε=0.05, cosine similarity threshold θ=0.90, confidence level c=0.25, and error rate threshold C=0.50. Implementation steps: Data preprocessing: 800 comments that meet the criteria were selected, of which 350 were high-impact (C1) and 450 were low-impact (C2). Attribute values, such as user type and comment field, were extracted. Calculate the information entropy: H(T) = 0.9887, and calculate the information gain ratio of each attribute: GainRatio(A1) = 0.1205, GainRatio(A2) = 0.0987, GainRatio(A3) = 0.1356, GainRatio(A4) = 0.1634; Attribute value merging: For attribute A2, the information entropy difference between "entertainment" and "lifestyle" is 0.0203 < ε, and the cosine similarity is 0.9978 > θ. Therefore, they are merged into "non-military and political". The value of A2 is simplified to {military and political, non-military and political}. Constructing a decision tree: Select A4, which has the highest information gain ratio, as the root node, and recursively construct the tree to obtain a decision tree with 9 nodes. Pruning and rule extraction: The pessimistic error rate of each node is calculated from bottom to top. Since none of them exceed the threshold C, no pruning is required. Four classification rules are extracted, such as "IF (whether it contains keywords = contains) AND (user type = users with a fan base) THEN category = C1". Performance validation: Evaluated using a test set of 200 comments, the accuracy rate was 86.5%, which is better than traditional K-means (73.2%) and traditional C4.5 (79.8%). The AC automaton took 0.15 seconds to retrieve 200 comments, which is 8 times more efficient than traditional matching (1.2 seconds). As can be seen from this embodiment, the method of the present invention has achieved significant improvements in classification accuracy, model simplicity, and processing efficiency, providing an efficient and stable technical solution for the analysis of negative comments on Weibo.

[0031] In the description of this invention, the references to "one embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0032] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A micro-blog negative review data analysis method based on cosine similarity-C4.5 decision tree, characterized in that, include: Step S1: Clean and filter the original Weibo comment data, extract user type, comment field, and number of negative comments to form a descriptive attribute set; at the same time, use the AC automaton algorithm to quickly match multiple keywords to obtain the attribute value of whether the keyword is contained and add it to the descriptive attribute set; according to whether the sum of the number of likes, reposts and comments of the comment text exceeds the preset influence threshold, label each comment text with a category tag; Step S2: For each descriptive attribute, map the data subset corresponding to each attribute value into a vector with the number of samples in each category as the component, calculate the information entropy of each data subset, and when the difference between the information entropy of two attribute values ​​is less than the first preset threshold and the cosine similarity of the corresponding vector is greater than the second preset threshold, merge the two attribute values ​​into a new attribute value. Step S3: Calculate the information gain ratio of each descriptive attribute after merging, select the descriptive attribute with the largest information gain ratio as the internal node of the decision tree, and recursively divide the descriptive attribute set to generate the decision tree. Step S4: Traverse the internal nodes of the decision tree from bottom to top. Based on the number of samples in the node, the actual error rate, and the standard deviation corresponding to the preset confidence level, calculate the pessimistic error rate estimate of each internal node. If the pessimistic error rate estimate is not greater than the preset error rate threshold, then prune the internal node and its subtree and convert it into a leaf node. Set the category label to the category with the most samples in the internal node. After pruning, extract the classification rules in IF-THEN form from the decision tree. 2.The micro-blog negative comment data analysis method of claim 1, wherein, In step S1, the user type is determined based on the number of followers, including ordinary users and users with a fan base; the field of the comments is automatically identified through a text classification model, including military and political, entertainment, and lifestyle. The number of negative reviews published is determined by comparing the number of negative reviews a user has published in the past with a preset threshold. 3.The micro-blog negative review data analysis method of claim 1, wherein, In step S1, the AC automaton algorithm includes: constructing a Trie tree with preset sensitive keywords and abusive words, where all keywords share a common prefix; constructing a mismatch pointer for each node using breadth-first traversal, pointing to the node with the longest common prefix and suffix; matching each comment text character by character, jumping along the mismatch pointer when a match fails, until a match is successful or the root node is returned; and recording the matched keyword when a marker node is reached. 4.The micro-blog negative review data analysis method of claim 1, wherein, In step S2, the first preset threshold ranges from 0.05 to 0.10, and the second preset threshold ranges from 0.85 to 0.

95. 5.The micro-blog negative review data analysis method of claim 1, wherein, In step S3, the information gain ratio is calculated as follows: first, the category information entropy describing the entire attribute set is calculated, then the information entropy of each data subset is calculated according to the attribute and the conditional entropy is obtained by weighted summation. The information gain is the difference between the two, the split information measure is the entropy of the attribute value, and the information gain ratio is the ratio of information gain to split information measure. 6.The micro-blog negative review data analysis method of claim 1, wherein, In step S4, the formula for calculating the pessimistic error rate estimate is as follows: wherein, represents a pessimistic error rate estimate value; represents an actual error rate of a node; represents a node a number of samples contained; represents a standard normal distribution quantile corresponding to a confidence level c, the confidence level c has a value range of 0.25-0.5, and a corresponding z value is determined by a statistical table; a preset error rate threshold C has a value range of 0.4-0.6, and a value is set according to a tolerance of an error rate in actual application.

7. The Weibo negative comment data analysis method according to claim 1, characterized in that, After extracting the fractional rules in step S4, the method also includes applying the fractional rules to automatically classify new comment data, outputting the classification results and corresponding rule paths for the public opinion monitoring system to perform subsequent processing and early warning.

8. The Weibo negative comment data analysis method according to claim 1, characterized in that, The labeling method for category tags in step S1 also includes multi-category settings, which divide the comment text into multiple levels such as high influence, medium influence, and low influence according to the degree of influence. At this time, the vector dimension corresponding to each attribute value is expanded to the number of categories, and the calculation of information entropy and information gain rate is adjusted to a multi-category form accordingly.

Images