Internet hotspot data mining system and method based on artificial intelligence

Abstract

The invention relates to the technical field of Internet, and discloses an Internet hotspot data mining system and method based on artificial intelligence, and the system comprises a data collection module, a semantic understanding module, a dynamic clustering module, a popularity evaluation module, a visual output module and a feedback optimization module. By setting a multi-source calibration unit, when cross-platform hotspot data acquisition is carried out, standardized processing of multi-source heterogeneous data acquisition is ensured by establishing a dynamic semantic feature library and configuring adaptive weight parameters for different data platforms; meanwhile, by monitoring the semantic offset of the collected data in real time, characterization deviation generated during cross-platform data collection can be detected and eliminated, the accuracy of hotspot clustering similarity calculation is guaranteed, hotspot recognition errors are reduced, and by deploying a semantic tracking engine, when hotspot event propagation analysis is carried out, the analysis accuracy is improved. The deviation degree of event core elements is calculated by constructing spatio-temporal feature vectors, and whether topic semantics are migrated or not is judged in real time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, in particular to an Internet hot spot data mining system and method based on artificial intelligence. BACKGROUND

[0002] Internet, also known as international network, is a global network infrastructure formed by interconnection of numerous computer networks, connected by a set of common protocols to form a single network in logic. With the rapid development of information network, the Internet, as a highly open "fourth media", has brought almost revolutionary changes to traditional media, and has become a new information medium with increasing social influence. More and more social people express their attitudes, opinions and requirements through the publication and comments of posts and articles on the network, so that the Internet can more quickly, directly, sharply and truly reflect public opinion. Network public opinion involves a very wide range, but focuses on social hot issues. Generally, these events that attract concentrated attention are called topics.

[0003] At present, in the process of Internet hot spot data mining, there are various complexities of data sources. When real-time monitoring of multi-platform hot spots is carried out, the heterogeneous data stream collected by the distributed crawler system cannot identify the semantic representation deviation generated in the cross-platform data collection in real time. When the same event data from different sources appears feature drift, it will cause the similarity calculation error of hot spot clustering to increase, and cannot guarantee the accuracy of hot spot identification. At the same time, when tracking hot events, it is impossible to detect in real time whether the core semantic elements in the event transmission process have deviated, which will cause the hot evolution path identification to deviate from the actual transmission track, and the analysis model cannot be dynamically updated when the topic features migrate. When verifying the authenticity of hot spots, due to the differences in the transmission form of hot spots on multiple platforms, it is impossible to realize the collaborative verification of cross-platform transmission characteristics when evaluating the value of hot spots, which may cause false events to be misjudged and high-value events to be missed in the hot spot screening process, affecting the detection accuracy and coverage completeness of the hot spot mining system.

[0004] Therefore, the present application provides an Internet hot spot data mining system and method based on artificial intelligence to solve the above problems. SUMMARY

[0005] (I) Technical problems solved

[0006] In view of the deficiencies of the prior art, the present application provides an Internet hot spot data mining system and method based on artificial intelligence to solve the problems raised in the background art.

[0007] (II) Technical solutions

[0008] In order to achieve the above object, the present application provides the following technical scheme to achieve: the internet hot spot data mining system and method based on artificial intelligence, the method comprises the following steps:

[0009] S1, collecting multi-source heterogeneous data in real time through a distributed crawler, including data news titles, social topics and forum post texts, adding a timestamp label;

[0010] S2, performing word segmentation, stop word filtering and symbol standardization on the multi-source heterogeneous data to generate a structured text set;

[0011] S3, generating a 128-dimensional semantic vector of the text using a pre-trained BERT model, and performing L2 normalization processing;

[0012] S4, calculating the similarity between texts and generating an association label, when the similarity is higher than a dynamic threshold, the association label is generated;

[0013] S5, constructing a dynamic relationship graph with texts as nodes and association labels as edges, and assigning nodes with a similarity higher than a threshold to the same initial clustering cluster;

[0014] S6, using a priority queue merging algorithm to iteratively merge the initial clustering cluster: taking the two clusters with the highest similarity as the merging target, updating the semantic center vector of the merged cluster, until the similarity between all clusters is lower than the merging threshold;

[0015] S7, calculating the heat index of each merged cluster, the heat index is dynamically calculated and generated by the number of texts in the cluster, the time decay factor and the propagation platform weight coefficient;

[0016] S8, extracting the core keywords and time evolution characteristics of the hot spot data;

[0017] S9, arranging the hot spot data in descending order of heat index, generating a visual hot spot ranking list and outputting a multi-dimensional analysis report;

[0018] S10, continuously optimizing the similarity calculation model and heat index weight parameter according to user feedback data.

[0019] Preferably, the step S1 specifically comprises:

[0020] S11, configuring an API interface cluster, respectively connecting a news aggregation platform API, a social media open interface and a forum RSS subscription source;

[0021] S12, using an asynchronous IO mechanism to concurrently collect multi-platform data streams, adding source platform ID and timestamp metadata to the collected data;

[0022] S13, establishing a data cleaning pipeline to filter advertising texts, repeated contents and noise data with a non-language symbol proportion of more than 30%.

[0023] Preferably, the text preprocessing in step S2 includes:

[0024] S21. Use a multi-granularity word segmentation model to perform semantic segmentation on the text, wherein the word segmentation model integrates dictionary matching and BiLSTM sequence tagging algorithm;

[0025] S22. Load a multi-domain stop word library to perform stop word filtering, wherein the stop word library includes general stop words, domain-specific stop words, and dynamically generated high-frequency low-value words;

[0026] S23. Perform standardized conversions on special symbols, including full-width to half-width conversions, currency symbol unification, and semantic encoding conversions of emoticons.

[0027] Preferably, the deep learning model processing in step S3 includes:

[0028] S31. Load the pre-trained BERT model and add a domain adaptation layer on top of the model. The adaptation layer includes a bidirectional LSTM and an attention mechanism.

[0029] S32, using contrastive learning strategy to fine-tune model parameters, with the training goal of maximizing the similarity of similar text vectors and minimizing the similarity of heterogeneous text vectors;

[0030] S33. Extract the last hidden state of the model as a 128-dimensional semantic vector and perform L2 normalization to generate a unit vector.

[0031] Preferably, the similarity calculation in step S4 adopts:

[0032] S41. Calculate the base value based on cosine similarity:

[0033]

[0034] in is the semantic vector of text i, j, is the L2 norm of the vector, sim base is the basic similarity value;

[0035] S41, superposition edit distance compensation factor:

[0036]

[0037] Where α is the adjustable mixing coefficient, L i , L j are text length, sim final is the final similarity value, sim base is the basic similarity value, edit distance is the Levenshtein edit distance function, ti is the original content of text i, t j is the original content of text j.

[0038] Preferably, the cluster merging algorithm of step S6 includes:

[0039] S61, construct a maximum heap to store cluster pair similarities, with the top element of the heap being the cluster pair with the highest current similarity;

[0040] S62: When the top similarity is greater than the merging threshold, perform cluster merging and generate a new cluster.

[0041] S63. Calculate the new cluster center vector as the weighted average of the original cluster vectors:

[0042]

[0043] where n i is the number of texts in cluster i, n j is the number of texts in cluster j, is the semantic center vector of the new cluster, is the center vector of cluster i, is the center vector of cluster j;

[0044] S64, updating the similarity value between the new cluster and the remaining clusters, and maintaining the maximum heap structure;

[0045] S65. Set time decay constraints: clusters older than 48 hours will not participate in the merge operation.

[0046] Preferably, the heat index in step S7 is calculated as:

[0047]

[0048] Where HotIndex is the hot index value, N is the total number of valid texts in the cluster, T is the time difference of the latest text, β is the time decay coefficient, and w p is the propagation weight of platform p, c p is the number of texts from platform p, and P is the platform set.

[0049] Preferably, the hotspot data extraction in step S8 includes:

[0050] S81, use TextRank algorithm to extract high-frequency keywords and key phrases within the cluster;

[0051] S82. Construct a time slice distribution map to identify the characteristics of the outbreak, duration, and subsidence periods of hot events;

[0052] S83. Generate a cross-platform propagation path map, marking the core propagation nodes and secondary propagation links.

[0053] Preferably, the system comprises:

[0054] Data collection module: Configure a multi-threaded crawler framework and API adapter to achieve real-time collection and timestamp marking of multi-source heterogeneous data;

[0055] Semantic Understanding Module: This module includes a preprocessing unit and a deep learning vectorization unit, which performs text cleaning and semantic vector generation.

[0056] Dynamic clustering module: realizes similarity calculation, relationship graph construction and iterative cluster merging functions;

[0057] Hotness evaluation module: calculates multi-dimensional hotness index and performs hotspot screening;

[0058] Visual output module: generates hotspot rankings, keyword cloud maps and dynamic demonstrations of communication paths;

[0059] Feedback optimization module: collects user behavior data and adjusts model parameters.

[0060] Preferably, it also includes:

[0061] The dynamic clustering module includes a priority queue processor to maintain the similarity ranking between clusters;

[0062] The heat evaluation module integrates a time decay calculator and a cross-platform weight configurator;

[0063] The visual output module supports interactive data drilling, response time range screening and platform source filtering operations;

[0064] The feedback optimization module establishes a parameter adjustment closed loop, and automatically triggers the similarity model retraining process when the deviation between the user's click behavior and the system's recommended hot spots exceeds a threshold.

[0065] (3) Beneficial effects

[0066] Compared with the existing technology, the present invention provides an Internet hotspot data mining system and method based on artificial intelligence, which has the following beneficial effects:

[0067] 1. In the present invention, by setting up a multi-source calibration unit, when collecting cross-platform hotspot data, by establishing a dynamic semantic feature library and configuring adaptive weight parameters for different data platforms, the standardized processing of multi-source heterogeneous data collection is guaranteed; at the same time, by real-time monitoring of the semantic offset of the collected data, the representation deviation generated during cross-platform data collection can be detected and eliminated, the accuracy of hotspot clustering similarity calculation is guaranteed, and the hotspot identification error is reduced.

[0068] 2. In the application, by deploying a semantic tracking engine, when performing hot event propagation analysis, the deviation degree of the event core elements is calculated by constructing a space-time feature vector, and it is judged in real time whether the topic semantics has migrated; when the evolution path deviates from the preset threshold, the analysis model is corrected in real time through the semantic gravity relocation algorithm, so that the system can avoid the deviation of hot tracking from the actual propagation trajectory, and dynamically update the identification parameters when the topic characteristics deviate, improve the recognition accuracy of the hot evolution path.

[0069] 3. In the application, by constructing a confidence fusion module, when performing hot spot authenticity verification, a multi-dimensional evaluation matrix is created to automatically divide the platform evaluation level, and a regional confidence score is generated according to the platform characteristics; based on the collaborative analysis of the cross-platform confidence score, a hot spot comprehensive value index is generated in real time, so that the system can realize multi-platform propagation feature collaborative verification, reduce the false hot spot misjudgment rate and high-value event missed detection rate, and improve the false hot spot filtering rate and high-value event recall rate. BRIEF DESCRIPTION OF DRAWINGS

[0070] Fig. 1 It is a schematic diagram of the overall system architecture of the application.

[0071] Fig. 2 It is a method flowchart of the application. DETAILED DESCRIPTION

[0072] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0073] Please refer to Figs. 1-2 The internet hot spot data mining system and method based on artificial intelligence, the method comprises the following steps:

[0074] S1, real-time collection of multi-source heterogeneous data by distributed crawler, including data news title, social topic and forum post, adding timestamp label;

[0075] S2, performing word segmentation, stop word filtering and symbol standardization on multi-source heterogeneous data to generate a structured text set;

[0076] S3, generating 128-dimensional semantic vectors of the text by using a pre-trained BERT model, and performing L2 normalization processing;

[0077] S4, calculating the similarity between texts and generating associated labels, when the similarity is higher than the dynamic threshold, generating associated labels;

[0078] S5. Build a dynamic relationship graph with text as nodes and associated labels as edges, and assign nodes with similarity higher than a threshold to the same initial cluster;

[0079] S6. Use the priority queue merging algorithm to iteratively merge the initial clusters: take the two clusters with the highest similarity as the merging targets, and update the semantic center vector of the merged cluster until the similarity between all clusters is lower than the merging threshold;

[0080] S7. Calculate the popularity index of each merged cluster, where the popularity index is dynamically calculated based on the number of texts in the cluster, the time decay factor, and the weight coefficient of the communication platform;

[0081] S8, extract the core keywords and time evolution characteristics of hot data;

[0082] S9. Sort the hotspot data in descending order of popularity index, generate a visual hotspot ranking list, and output a multi-dimensional analysis report;

[0083] S10. Continuously optimize the similarity calculation model and heat index weight parameters based on user feedback data.

[0084] Step S1 specifically includes:

[0085] S11. Configure the API interface cluster to connect to the news aggregation platform API, social media open interface, and forum RSS subscription feed;

[0086] S12. Use asynchronous IO mechanism to concurrently collect data streams from multiple platforms, and add source platform ID and timestamp metadata to the collected data;

[0087] S13. Establish a data cleaning pipeline to filter out noise data that contains more than 30% of advertising text, repeated content, and non-verbal symbols.

[0088] The text preprocessing in step S2 includes:

[0089] S21. Use a multi-granularity word segmentation model to semantically segment the text. The word segmentation model integrates dictionary matching and BiLSTM sequence tagging algorithm.

[0090] S22. Load a multi-domain stop word library to perform stop word filtering, where the stop word library includes general stop words, domain-specific stop words, and dynamically generated high-frequency low-value words;

[0091] S23. Perform standardized conversions on special symbols, including full-width to half-width conversions, currency symbol unification, and semantic encoding conversions of emoticons.

[0092] The deep learning model processing in step S3 includes:

[0093] S31. Load the pre-trained BERT model and add a domain adaptation layer on top of the model. The adaptation layer includes a bidirectional LSTM and an attention mechanism.

[0094] S32, using contrastive learning strategy to fine-tune model parameters, with the training goal of maximizing the similarity of similar text vectors and minimizing the similarity of heterogeneous text vectors;

[0095] S33. Extract the last hidden state of the model as a 128-dimensional semantic vector and perform L2 normalization to generate a unit vector.

[0096] The similarity calculation in step S4 is:

[0097] S41. Calculate the base value based on cosine similarity:

[0098]

[0099] in is the semantic vector of text i, j, is the L2 norm of the vector, sim base is the basic similarity value;

[0100] S41, superposition edit distance compensation factor:

[0101]

[0102] Where α is the adjustable mixing coefficient, L i , L j are text length, sim final is the final similarity value, sim base is the basic similarity value, edit distance is the Levenshtein edit distance function, t i is the original content of text i, t j is the original content of text j.

[0103] The cluster merging algorithm of step S6 includes:

[0104] S61, construct a maximum heap to store cluster pair similarities, with the top element of the heap being the cluster pair with the highest current similarity;

[0105] S62: When the top similarity is greater than the merging threshold, perform cluster merging and generate a new cluster.

[0106] S63. Calculate the new cluster center vector as the weighted average of the original cluster vectors:

[0107]

[0108] where n i is the number of texts in cluster i, n jis the number of texts in cluster j, is the semantic center vector of the new cluster, is the center vector of cluster i, is the center vector of cluster j;

[0109] S64, updating the similarity value between the new cluster and the remaining clusters, and maintaining the maximum heap structure;

[0110] S65. Set time decay constraints: clusters older than 48 hours will not participate in the merge operation.

[0111] The heat index in step S7 is calculated as:

[0112]

[0113] Where HotIndex is the hot index value, N is the total number of valid texts in the cluster, T is the time difference of the latest text, β is the time decay coefficient, and w p is the propagation weight of platform p, c p is the number of texts from platform p, and P is the platform set.

[0114] The hotspot data extraction in step S8 includes:

[0115] S81, use TextRank algorithm to extract high-frequency keywords and key phrases within the cluster;

[0116] S82. Construct a time slice distribution map to identify the characteristics of the outbreak, duration, and subsidence periods of hot events;

[0117] S83. Generate a cross-platform propagation path map, marking the core propagation nodes and secondary propagation links.

[0118] The system includes:

[0119] Data collection module: Configure a multi-threaded crawler framework and API adapter to achieve real-time collection and timestamp marking of multi-source heterogeneous data;

[0120] Semantic Understanding Module: This module includes a preprocessing unit and a deep learning vectorization unit, which performs text cleaning and semantic vector generation.

[0121] Dynamic clustering module: realizes similarity calculation, relationship graph construction and iterative cluster merging functions;

[0122] Hotness evaluation module: calculates multi-dimensional hotness index and performs hotspot screening;

[0123] Visual output module: generates hotspot rankings, keyword cloud maps and dynamic demonstrations of communication paths;

[0124] Feedback optimization module: collects user behavior data and adjusts model parameters.

[0125] Also includes:

[0126] The dynamic clustering module contains a priority queue processor to maintain the similarity ranking between clusters;

[0127] The heat evaluation module integrates a time decay calculator and a cross-platform weight configurator;

[0128] The visual output module supports interactive data drilling, response time range screening and platform source filtering operations;

[0129] The feedback optimization module establishes a parameter adjustment closed loop. When the deviation between the user's click behavior and the system's recommended hot spots exceeds the threshold, the similarity model retraining process is automatically triggered.

[0130] Example 1: Semantic calibration solution for cross-platform data collection:

[0131] In the cross-platform hot data collection scenario, the present invention solves the problem of data representation deviation by deploying a multi-source semantic calibration unit. The specific implementation process first constructs a dynamic semantic feature library: establishes an institutional authority feature baseline for news platforms such as Xinhuanet, configures an emoticon encoding comparison table for social media such as Weibo, and loads professional terminology dictionaries such as blockchain for forum platforms such as Zhihu. When implementing real-time offset monitoring, the system calculates the spatial angle difference between the current text vector and the reference vector, and automatically triggers the calibration mechanism when the difference exceeds the preset threshold of 0.2. In the dynamic correction stage, a generative adversarial network is used to generate a semantic compensation vector, which is superimposed on the original data with 30% compensation vectors and then output to the downstream processing module. After verification by multi-source data testing, the solution reduces the cross-platform data representation difference by 82%, and the hotspot clustering similarity calculation error is reduced from the baseline value of 15.7% to 3.4%.

[0132] Example 2: Dynamic tracking solution for hotspot propagation trajectory:

[0133] This embodiment runs the semantic evolution tracking engine to achieve precise control. The specific implementation includes three key stages: in the spatiotemporal feature extraction link, the system automatically captures the core keywords every 2 hours to build a state matrix, and fully records the spatiotemporal distribution characteristics of the keyword vector; the offset detection stage calculates the main direction angle between the current state matrix and the initial matrix, and immediately determines that a semantic offset has occurred when the angle deviation is greater than 15 degrees; at this time, the model dynamic update mechanism is triggered, and the incremental learning algorithm is used to adjust the neural network weight parameters of the LSTM-CRF model at a learning rate of 0.01. In actual tests of public health emergencies, the system completed the response within 5 minutes after the semantic offset occurred, and the evolutionary path recognition accuracy reached 93.6%, an increase of 15.4 percentage points compared with traditional methods.

[0134] Example 3: Multi-platform hotspot authenticity verification solution:

[0135] This embodiment establishes a three-level confidence fusion system. First, a multi-dimensional evaluation matrix is ​​constructed: news authority is scored based on media rank; social communication depth is calculated by multiplying the forwarding level and the diffusion coefficient; and forum sentiment intensity is calculated by multiplying the sentiment polarity and the sentiment density. The confidence fusion mechanism uses a weighted ratio of 40% for authority, 30% for communication depth, and 30% for sentiment intensity. A comprehensive confidence score is generated through normalization. A high-value hotspot is identified when the score exceeds a threshold of 0.8. Anomaly blocking rules are also configured to automatically transfer suspected fraudulent hotspots with a communication depth exceeding 10 but an authority score below 2 to a manual review queue. In actual monitoring during the 315 Consumer Rights Protection Day, this solution achieved a fraudulent hotspot filtering rate of 97.3% and a high-value event recall rate of 89.4%, a 17.8 percentage point improvement over the baseline system.

[0136] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0137] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An Internet hotspot data mining method based on artificial intelligence, characterized by: The method comprises the following steps: S1. Collect multi-source heterogeneous data in real time through distributed crawlers, including data news headlines, social topics, and forum posts, and add timestamp tags; S2. Perform word segmentation, stop word filtering, and symbol standardization on multi-source heterogeneous data to generate a structured text set; S3, using the pre-trained BERT model to generate a 128-dimensional semantic vector of the text, which is then L2 normalized; S4, calculating the similarity between texts and generating associated tags, generating associated tags when the similarity is higher than a dynamic threshold; S5. Build a dynamic relationship graph with text as nodes and associated labels as edges, and assign nodes with similarity higher than a threshold to the same initial cluster; S6. Use the priority queue merging algorithm to iteratively merge the initial clusters: take the two clusters with the highest similarity as the merging targets, and update the semantic center vector of the merged cluster until the similarity between all clusters is lower than the merging threshold; S7. Calculate the popularity index of each merged cluster, where the popularity index is dynamically calculated based on the number of texts in the cluster, the time decay factor, and the weight coefficient of the communication platform; S8, extract the core keywords and time evolution characteristics of hot data; S9. Sort the hotspot data in descending order of popularity index, generate a visual hotspot ranking list, and output a multi-dimensional analysis report; S10. Continuously optimize the similarity calculation model and heat index weight parameters based on user feedback data.

2. The method for mining Internet hotspot data based on artificial intelligence according to claim 1, characterized in that: The step S1 specifically includes: S11. Configure the API interface cluster to connect to the news aggregation platform API, social media open interface, and forum RSS subscription feed; S12. Use asynchronous IO mechanism to concurrently collect data streams from multiple platforms, and add source platform ID and timestamp metadata to the collected data; S13. Establish a data cleaning pipeline to filter out noise data that contains more than 30% of advertising text, repeated content, and non-verbal symbols.

3. The method for mining Internet hotspot data based on artificial intelligence according to claim 1, characterized in that: The text preprocessing of step S2 includes: S21. Use a multi-granularity word segmentation model to perform semantic segmentation on the text, wherein the word segmentation model integrates dictionary matching and BiLSTM sequence tagging algorithm; S22. Load a multi-domain stop word library to perform stop word filtering, wherein the stop word library includes general stop words, domain-specific stop words, and dynamically generated high-frequency low-value words; S23. Perform standardized conversions on special symbols, including full-width to half-width conversions, currency symbol unification, and semantic encoding conversions of emoticons.

4. The method for mining Internet hotspot data based on artificial intelligence according to claim 1, wherein: The deep learning model processing in step S3 includes: S31. Load the pre-trained BERT model and add a domain adaptation layer on top of the model. The adaptation layer includes a bidirectional LSTM and an attention mechanism. S32, using contrastive learning strategy to fine-tune model parameters, with the training goal of maximizing the similarity of similar text vectors and minimizing the similarity of heterogeneous text vectors; S33. Extract the last hidden state of the model as a 128-dimensional semantic vector and perform L2 normalization to generate a unit vector.

5. The method for mining Internet hotspot data based on artificial intelligence according to claim 1, characterized in that: The similarity calculation in step S4 is as follows: S41. Calculate the base value based on cosine similarity: in is the semantic vector of text i, j, is the L2 norm of the vector, sim base is the basic similarity value; S41, superposition edit distance compensation factor: Where α is the adjustable mixing coefficient, L i 、L j are text length, sim final is the final similarity value, sim base is the basic similarity value, edit distance is the Levenshtein edit distance function, t i is the original content of text i, t j is the original content of text j.

6. The method for mining Internet hotspot data based on artificial intelligence according to claim 1, characterized in that: The cluster merging algorithm of step S6 includes: S61, construct a maximum heap to store cluster pair similarities, with the top element of the heap being the cluster pair with the highest current similarity; S62: When the top similarity is greater than the merging threshold, perform cluster merging and generate a new cluster. S63. Calculate the new cluster center vector as the weighted average of the original cluster vectors: where n i is the number of texts in cluster i, n j is the number of texts in cluster j, is the semantic center vector of the new cluster, is the center vector of cluster i, is the center vector of cluster j; S64, updating the similarity value between the new cluster and the remaining clusters, and maintaining the maximum heap structure; S65. Set time decay constraints: clusters older than 48 hours will not participate in the merge operation.

7. The method for mining Internet hotspot data based on artificial intelligence according to claim 1, characterized in that: The heat index of step S7 is calculated as: Where HotIndex is the hot index value, N is the total number of valid texts in the cluster, T is the time difference of the latest text, β is the time decay coefficient, and w p is the propagation weight of platform p, c p is the number of texts from platform p, and P is the platform set.

8. The method for mining Internet hotspot data based on artificial intelligence according to claim 1, characterized in that: The hotspot data extraction in step S8 includes: S81, use TextRank algorithm to extract high-frequency keywords and key phrases within the cluster; S82. Construct a time slice distribution map to identify the characteristics of the outbreak, duration, and subsidence periods of hot events; S83. Generate a cross-platform propagation path map, marking the core propagation nodes and secondary propagation links.

9. An Internet hotspot data mining system based on artificial intelligence, characterized by: The system comprises: Data collection module: Configure a multi-threaded crawler framework and API adapter to achieve real-time collection and timestamp marking of multi-source heterogeneous data; Semantic Understanding Module: This module includes a preprocessing unit and a deep learning vectorization unit, which performs text cleaning and semantic vector generation. Dynamic clustering module: realizes similarity calculation, relationship graph construction and iterative cluster merging functions; Hotness evaluation module: calculates multi-dimensional hotness index and performs hotspot screening; Visual output module: generates hotspot rankings, keyword cloud maps and dynamic demonstrations of communication paths; Feedback optimization module: collects user behavior data and adjusts model parameters.

10. The artificial intelligence-based Internet hotspot data mining system according to claim 9, characterized in that: Also includes: The dynamic clustering module includes a priority queue processor to maintain the similarity ranking between clusters; The heat evaluation module integrates a time decay calculator and a cross-platform weight configurator; The visual output module supports interactive data drilling, response time range screening and platform source filtering operations; The feedback optimization module establishes a parameter adjustment closed loop, and automatically triggers the similarity model retraining process when the deviation between the user's click behavior and the system's recommended hot spots exceeds a threshold.

Images