Subject hotspot dynamic monitoring method and system

Abstract

The invention relates to the technical field of subject hot spot monitoring, and discloses a subject hot spot dynamic monitoring method and system, and the monitoring method comprises the following steps: S1, determining a monitoring target and a monitoring range, determining a subject field needing to be monitored and a specific direction, carrying out the data collection, carrying out the preprocessing of the collected data, and carrying out the processing of the collected data; carrying out format unification and standardization processing on data from different sources; based on the big data operation and maintenance technology, the subject hot spot dynamic state can be monitored in real time, the real-time performance and accuracy of the data are ensured, the collected data can be efficiently processed and analyzed through the big data analysis and mining technology, valuable information and insight are extracted, the research direction is grasped, and the research efficiency is improved. The big data operation and maintenance system can collect data from various sources, ensures comprehensive coverage of subject hotspot dynamics, can accurately predict the subject hotspot dynamics through big data analysis and a machine learning algorithm, and provides a basis for scientific research and decision making.

Description

Technical Field

[0001] The present invention relates to the technical field of subject hotspot monitoring, and in particular to a subject hotspot dynamic monitoring method and system. Background Art

[0002] Dynamic monitoring of subject hot spots can track the development trends and changes of subjects in real time, ensuring that schools or research institutions can keep abreast of the latest developments in subjects. Through monitoring, key indicator data such as high-end talents, scientific research projects, award-winning results, academic papers, etc. in the subject field can be obtained to provide strong support for the development of the subject. Based on the data and analysis results of dynamic monitoring of subject hot spots, schools or research institutions can formulate development plans and decisions more scientifically. These decisions can be adjusted and optimized in a targeted manner according to the weaknesses and advantages of the subject, thereby improving the overall competitiveness of the subject. Dynamic monitoring of subject hot spots not only focuses on the actual status of the subject, but also pays attention to its development history and development trends. Through comparison and analysis, it can inspire the enterprising spirit of teachers and students in the subject, promote their continuous pursuit of excellence, and improve the teaching quality and scientific research level of the subject. Dynamic monitoring of subject hot spots helps to discover the intersections and integration points between different disciplines. Through interdisciplinary cooperation and exchanges, it can promote the generation of new knowledge and the improvement of innovation capabilities, injecting new vitality into the development of the subject. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for dynamically monitoring subject hot spots to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a method for dynamically monitoring subject hot spots, the monitoring method comprising the following steps:

[0005] S1. Determine the objectives and scope of monitoring, clarify the subject areas and specific directions that need to be monitored, collect data from academic journals and databases, conference proceedings, social media and news media, patent databases, and academic social networks, pre-process the collected data, remove duplicates, fill in missing values, correct erroneous data, integrate the processed data, unify and standardize the format of data from different sources, and then store the integrated data in the database;

[0006] S2. Analyze the data, use natural language processing technology to extract keywords, topics, author information, calculate various indicators, use data visualization tools to generate charts and reports to help intuitively understand the data, use machine learning algorithms to cluster the data, identify different research directions and hot areas, and discover the long-term trends and short-term fluctuations of discipline development through the analysis of time series data. Use association rule mining technology to find the correlation between different research fields;

[0007] S3. Establish a real-time monitoring system, use the streaming processing framework to process real-time data, identify abnormal data points through anomaly detection algorithms, and promptly discover potential research hotspots or emergencies. Display the analysis results through a dashboard for users to view and understand, and generate regular reports to summarize the hotspot dynamics and development trends over a period of time.

[0008] S4. Collect user feedback on system functions and results, and continuously optimize system functions and algorithms based on the feedback to improve user experience and accuracy.

[0009] Preferably, in step S1, the step of preprocessing the data includes:

[0010] Use Python's Pandas library to load data into the working environment and view the basic information, statistical summary, and distribution of the data;

[0011] Use Pandas’ duplicated() method to identify duplicate records, use Pandas’ drop_duplicates() method to delete duplicate records, ensure that each record in the dataset is unique, use Pandas’ isnull() to identify missing values, and check the number and distribution of missing values;

[0012] Identify erroneous data by comparing data validation rules. After filling missing values ​​and correcting erroneous data, recheck the basic information, statistical summary and distribution of the data to ensure the consistency and accuracy of the data. Use data validation rules and visualization tools to compare with other data sources to recheck the quality and completeness of the data.

[0013] Save the preprocessed data to a new file or database for subsequent analysis. During the preprocessing process, regularly back up the original data and intermediate results to prevent data loss or damage.

[0014] Preferably, the extraction step using natural language processing technology in step S2 includes:

[0015] Preprocess the text to remove noise and irrelevant information, cut the text into individual words, and remove stop words in the text;

[0016] For feature extraction, the frequency of each word in the document is counted to measure the ability of a word to distinguish documents in the document set. The TF-IDF algorithm is used to combine word frequency and inverse document frequency to calculate the score of each word. The words with high scores are used as keywords. Then, the TextRank algorithm is used to sort the text based on the graph. The relationship between local words is used to sort the subsequent keywords. The extracted keywords are filtered, sorted, deduplicated and merged to obtain the final keyword list.

[0017] Use the LSA algorithm to extract topics from texts, map the texts to a low-dimensional topic space, and then extract topics. Perform post-processing on the extracted topics, merge similar topics, and remove irrelevant topics.

[0018] Conduct in-depth grammatical and semantic analysis of the text, including part-of-speech tagging and named entity recognition. Named entity recognition can identify specific types of entities in the text and match the identified entities with a known author database to find possible author information. The text contains information related to the author's name, which is directly extracted and then verified to ensure its accuracy.

[0019] Preferably, in step S2, the step of using association rule mining technology to find the correlation between different research fields includes:

[0020] Determine the research field and collect data related to these research fields. The data should include the identification of different research fields and the possible association information between them. Clean and organize the collected data, remove duplicate, invalid or noisy data, and convert the data into a format suitable for association rule mining.

[0021] Select appropriate association rule mining algorithms based on data characteristics and mining requirements, determine support and confidence thresholds, and use these two parameters to filter frequent item sets and generate association rules. Support represents the frequency of item sets in a data set, and confidence represents the accuracy of association rules. Use the selected algorithms and parameters to mine data and generate association rules that meet support and confidence thresholds.

[0022] Filter out the rules that are strongly related to the research field from the mined association rules, filter them according to the support, confidence and lift indicators of the rules, interpret the filtered association rules, and clarify the correlation between the different research fields they represent;

[0023] The mined association rules are displayed in a graphical way.

[0024] Preferably, the step of using a stream processing framework to process real-time data in step S3 includes:

[0025] According to the specific needs of processing real-time data, select a suitable stream processing framework, build and configure the operating environment required by the stream processing framework, including hardware resources, software dependencies, and network configuration, and connect the real-time data source to the stream processing framework;

[0026] Clean the incoming real-time data, remove invalid, duplicate or noisy data, ensure data quality, convert the data into a format suitable for subsequent processing and analysis, and use the data streaming computing function provided by the streaming processing framework to process and analyze the data in real time;

[0027] According to the characteristics of the data and the application scenario, select the appropriate anomaly detection algorithm, use historical data to train the anomaly detection algorithm, generate an initialization model, input real-time data into the trained anomaly detection model, perform real-time anomaly detection, and mark or record the detected abnormal data points for subsequent analysis;

[0028] Filter out data related to research hotspots or emergencies from detected abnormal data points, conduct in-depth analysis of the filtered abnormal data, explain the causes and meanings of the abnormalities in combination with domain knowledge, analyze the distribution, frequency, and trend of abnormal data points to discover potential research hotspots or emergencies, and formulate corresponding response strategies or decision plans based on the analysis results;

[0029] Monitor the operating status of the streaming processing framework and anomaly detection system in real time to ensure the stability and reliability of the system. Optimize the performance of the streaming processing framework and anomaly detection algorithm based on the system's operating conditions and processing efficiency. Continuously optimize and update the streaming processing framework and anomaly detection system as data continues to accumulate and application scenarios change.

[0030] Preferably, in step S4, the analysis results and hot information are organized into a knowledge base to facilitate subsequent research and reference, and the hot information is released through academic websites and social media to promote the dissemination and sharing of knowledge.

[0031] A system for dynamically monitoring academic hotspots includes a data acquisition module, a data preprocessing unit, a central processing unit, a real-time monitoring module, a system maintenance and optimization module, a hotspot identification unit, a knowledge management unit, a visualization display module and a user interaction module. The output end of the data acquisition module is electrically connected to the input end of the data preprocessing unit, the output end of the data preprocessing unit is electrically connected to the input end of the central processing unit, the output end of the central processing unit is electrically connected to the input ends of the hotspot identification unit, the visualization display module, the knowledge management unit and the real-time monitoring module respectively, the output end of the real-time monitoring module is electrically connected to the input end of the system maintenance and optimization module, the output end of the visualization display module is electrically connected to the input end of the user interaction module, the data acquisition module can collect raw data from multiple data sources, the real-time monitoring module can process and monitor data streams in real time, and promptly discover new hotspots and abnormal situations, the system maintenance and optimization module can continuously optimize system performance and functions, and collect user feedback on system functions and results, the visualization display module can display analysis results to users in a graphical manner, and the user interaction module can provide a user-friendly interface so that users can easily query and operate data.

[0032] Preferably, the data preprocessing unit includes a data cleaning module, a data integration module, a data conversion module and a data reduction module. The data cleaning module is the core part of the data preprocessing unit and is responsible for eliminating noise in the data and correcting its inconsistent errors. The data integration module is responsible for merging data from multiple data sources together to form a complete data set. The data conversion module is responsible for format conversion, unit conversion and other processing on the data to meet the needs of subsequent analysis. The data reduction module aims to reduce the size of the data while minimizing the impact on the final mining results.

[0033] Preferably, the hotspot identification unit includes a topic identification module, a hotspot extraction module and a trend analysis module. The topic identification module uses natural language processing to perform text analysis and topic extraction on the collected data, identifies key themes and topics in the data, and provides a basis for subsequent hotspot analysis. The hotspot extraction module uses statistical analysis and data mining techniques based on the results of the topic identification module to extract hot topics and trends from the data. The trend analysis module uses time series analysis and prediction technology to predict and analyze the development trends of hot topics.

[0034] Preferably, the knowledge management unit includes a knowledge storage module, a knowledge retrieval module, a knowledge sharing module, a knowledge updating and maintenance module and an intelligent recommendation module. The knowledge acquisition module is responsible for automatically collecting knowledge related to subject hotspots from various data sources. The knowledge storage module uses database technology to store the collected knowledge in a structured manner to facilitate subsequent retrieval and analysis. The knowledge retrieval module uses search engines and information retrieval technology to provide users with convenient knowledge retrieval services. The knowledge sharing module supports users to share and disseminate knowledge in the knowledge base, including internal sharing and external publishing. The knowledge updating and maintenance module is responsible for regularly updating the knowledge in the knowledge base to ensure the accuracy and timeliness of the knowledge. At the same time, it is also responsible for processing user feedback and opinions, and maintaining and optimizing the knowledge base. The intelligent recommendation module uses machine learning algorithms and recommendation technologies to recommend relevant knowledge and resources to users based on their interests and behaviors.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention is based on big data operation and maintenance technology, which can monitor the dynamics of subject hot spots in real time, ensure the real-time and accuracy of data, and help to timely discover and track the latest developments and trends in the subject field. By using big data analysis and mining technology, the collected data can be efficiently processed and analyzed to extract valuable information and insights, which helps users quickly understand subject hot spots and grasp research directions. The big data operation and maintenance system can collect data from multiple sources to ensure comprehensive coverage of subject hot spots. Through big data analysis and machine learning algorithms, the dynamics of subject hot spots can be accurately predicted, which helps users understand future research directions and trends in advance and provide a basis for scientific research decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0038] Figure 2 is a system block diagram of the present invention;

[0039] Figure 3 is a system block diagram of a data preprocessing unit in the present invention;

[0040] Figure 4 is a system block diagram of a hotspot identification unit in the present invention;

[0041] Figure 5 It is a system block diagram of the knowledge management unit in the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] A method for dynamically monitoring subject hot spots, the monitoring method comprising the following steps:

[0044] S1. Determine the objectives and scope of monitoring, clarify the subject areas and specific directions that need to be monitored, collect data from academic journals and databases, conference proceedings, social media and news media, patent databases, and academic social networks, pre-process the collected data, remove duplicates, fill in missing values, correct erroneous data, integrate the processed data, unify and standardize the format of data from different sources, and then store the integrated data in the database;

[0045] S2. Analyze the data, use natural language processing technology to extract keywords, topics, author information, calculate various indicators, use data visualization tools to generate charts and reports to help intuitively understand the data, use machine learning algorithms to cluster the data, identify different research directions and hot areas, and discover the long-term trends and short-term fluctuations of discipline development through the analysis of time series data. Use association rule mining technology to find the correlation between different research fields;

[0046] S3. Establish a real-time monitoring system, use the streaming processing framework to process real-time data, identify abnormal data points through anomaly detection algorithms, and promptly discover potential research hotspots or emergencies. Display the analysis results through a dashboard for users to view and understand, and generate regular reports to summarize the hotspot dynamics and development trends over a period of time.

[0047] S4. Collect user feedback on system functions and results, and continuously optimize system functions and algorithms based on the feedback to improve user experience and accuracy.

[0048] In step S1, the step of preprocessing the data includes:

[0049] Use Python's Pandas library to load data into the working environment and view the basic information, statistical summary, and distribution of the data;

[0050] Use Pandas’ duplicated() method to identify duplicate records, use Pandas’ drop_duplicates() method to delete duplicate records, ensure that each record in the dataset is unique, use Pandas’ isnull() to identify missing values, and check the number and distribution of missing values;

[0051] Identify erroneous data by comparing data validation rules. After filling missing values ​​and correcting erroneous data, recheck the basic information, statistical summary and distribution of the data to ensure the consistency and accuracy of the data. Use data validation rules and visualization tools to compare with other data sources to recheck the quality and completeness of the data.

[0052] Save the preprocessed data to a new file or database for subsequent analysis. During the preprocessing process, regularly back up the original data and intermediate results to prevent data loss or damage.

[0053] The extraction steps using natural language processing technology in step S2 include:

[0054] Preprocess the text to remove noise and irrelevant information, cut the text into individual words, and remove stop words in the text;

[0055] For feature extraction, the frequency of each word in the document is counted to measure the ability of a word to distinguish documents in the document set. The TF-IDF algorithm is used to combine word frequency and inverse document frequency to calculate the score of each word. The words with high scores are used as keywords. Then, the TextRank algorithm is used to sort the text based on the graph. The relationship between local words is used to sort the subsequent keywords. The extracted keywords are filtered, sorted, deduplicated and merged to obtain the final keyword list.

[0056] Use the LSA algorithm to extract topics from texts, map the texts to a low-dimensional topic space, and then extract topics. Perform post-processing on the extracted topics, merge similar topics, and remove irrelevant topics.

[0057] Conduct in-depth grammatical and semantic analysis of the text, including part-of-speech tagging and named entity recognition. Named entity recognition can identify specific types of entities in the text and match the identified entities with a known author database to find possible author information. The text contains information related to the author's name, which is directly extracted and then verified to ensure its accuracy.

[0058] In step S2, using association rule mining technology, the steps of finding the correlation between different research fields include:

[0059] Determine the research field and collect data related to these research fields. The data should include the identification of different research fields and the possible association information between them. Clean and organize the collected data, remove duplicate, invalid or noisy data, and convert the data into a format suitable for association rule mining.

[0060] Select appropriate association rule mining algorithms based on data characteristics and mining requirements, determine support and confidence thresholds, and use these two parameters to filter frequent item sets and generate association rules. Support represents the frequency of item sets in a data set, and confidence represents the accuracy of association rules. Use the selected algorithms and parameters to mine data and generate association rules that meet support and confidence thresholds.

[0061] Filter out the rules that are strongly related to the research field from the mined association rules, filter them according to the support, confidence and lift indicators of the rules, interpret the filtered association rules, and clarify the correlation between the different research fields they represent;

[0062] The mined association rules are displayed in a graphical way.

[0063] The steps of using the stream processing framework to process real-time data in step S3 include:

[0064] According to the specific needs of processing real-time data, select a suitable stream processing framework, build and configure the operating environment required by the stream processing framework, including hardware resources, software dependencies, and network configuration, and connect the real-time data source to the stream processing framework;

[0065] Clean the incoming real-time data, remove invalid, duplicate or noisy data, ensure data quality, convert the data into a format suitable for subsequent processing and analysis, and use the data streaming computing function provided by the streaming processing framework to process and analyze the data in real time;

[0066] According to the characteristics of the data and the application scenario, select the appropriate anomaly detection algorithm, use historical data to train the anomaly detection algorithm, generate an initialization model, input real-time data into the trained anomaly detection model, perform real-time anomaly detection, and mark or record the detected abnormal data points for subsequent analysis;

[0067] Filter out data related to research hotspots or emergencies from detected abnormal data points, conduct in-depth analysis of the filtered abnormal data, explain the causes and meanings of the abnormalities in combination with domain knowledge, analyze the distribution, frequency, and trend of abnormal data points to discover potential research hotspots or emergencies, and formulate corresponding response strategies or decision plans based on the analysis results;

[0068] Monitor the operating status of the streaming processing framework and anomaly detection system in real time to ensure the stability and reliability of the system. Optimize the performance of the streaming processing framework and anomaly detection algorithm based on the system's operating conditions and processing efficiency. Continuously optimize and update the streaming processing framework and anomaly detection system as data continues to accumulate and application scenarios change.

[0069] In step S4, the analysis results and hot information are organized into a knowledge base to facilitate subsequent research and reference, and the hot information is released through academic websites and social media to promote the dissemination and sharing of knowledge.

[0070] A system for dynamically monitoring academic hotspots includes a data acquisition module, a data preprocessing unit, a central processing unit, a real-time monitoring module, a system maintenance and optimization module, a hotspot identification unit, a knowledge management unit, a visualization display module and a user interaction module. The output end of the data acquisition module is electrically connected to the input end of the data preprocessing unit, the output end of the data preprocessing unit is electrically connected to the input end of the central processing unit, the output end of the central processing unit is electrically connected to the input ends of the hotspot identification unit, the visualization display module, the knowledge management unit and the real-time monitoring module respectively, the output end of the real-time monitoring module is electrically connected to the input end of the system maintenance and optimization module, the output end of the visualization display module is electrically connected to the input end of the user interaction module, the data acquisition module can collect raw data from multiple data sources, the real-time monitoring module can process and monitor data streams in real time, and promptly discover new hotspots and abnormal situations, the system maintenance and optimization module can continuously optimize system performance and functions, and collect user feedback on system functions and results, the visualization display module can display analysis results to users in a graphical manner, and the user interaction module can provide a user-friendly interface so that users can easily query and operate data.

[0071] The data preprocessing unit includes a data cleaning module, a data integration module, a data conversion module and a data reduction module. The data cleaning module is the core part of the data preprocessing unit and is responsible for eliminating the noise in the data and correcting its inconsistent errors. The data integration module is responsible for merging data from multiple data sources together to form a complete data set. The data conversion module is responsible for format conversion, unit conversion and other processing of the data to meet the needs of subsequent analysis. The data reduction module aims to reduce the size of the data while minimizing the impact on the final mining results.

[0072] The hotspot identification unit includes a topic identification module, a hotspot extraction module and a trend analysis module. The topic identification module uses natural language processing to perform text analysis and topic extraction on the collected data, identifies key themes and topics in the data, and provides a basis for subsequent hotspot analysis. The hotspot extraction module, based on the results of the topic identification module, uses statistical analysis and data mining techniques to extract hot topics and trends from the data. The trend analysis module uses time series analysis and prediction technology to predict and analyze the development trends of hot topics.

[0073] The knowledge management unit includes knowledge storage module, knowledge retrieval module, knowledge sharing module, knowledge update and maintenance module and intelligent recommendation module. The knowledge acquisition module is responsible for automatically collecting knowledge related to subject hotspots from various data sources. The knowledge storage module uses database technology to store the collected knowledge in a structured manner to facilitate subsequent retrieval and analysis. The knowledge retrieval module uses search engines and information retrieval technology to provide users with convenient knowledge retrieval services. The knowledge sharing module supports users to share and disseminate knowledge in the knowledge base, including internal sharing and external publishing. The knowledge update and maintenance module is responsible for regularly updating the knowledge in the knowledge base to ensure the accuracy and timeliness of the knowledge. At the same time, it is also responsible for processing user feedback and opinions, and maintaining and optimizing the knowledge base. The intelligent recommendation module uses machine learning algorithms and recommendation technologies to recommend relevant knowledge and resources to users based on their interests and behaviors.

[0074] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0075] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for dynamic monitoring of subject hot spots, characterized in that: The monitoring method includes the following steps: S1. Determine the monitoring objectives and scope, clarify the subject areas and specific directions that need to be monitored, collect data, pre-process the collected data, unify and standardize the data from different sources, and then store the integrated data in the database; S2. Analyze the data, calculate various indicators, use data visualization tools to generate charts and reports, use machine learning algorithms to cluster the data, identify different research directions and hot areas, and use association rule mining technology to find the correlation between different research fields; S3. Establish a real-time monitoring system, use the stream processing framework to process real-time data, and identify abnormal data points through anomaly detection algorithms; S4. Collect user feedback on system functions and results, and continuously optimize system functions and algorithms based on the feedback to improve user experience and accuracy.

2. A method for dynamic monitoring of subject hot spots according to claim 1, characterized in that: In step S1, the step of preprocessing the data includes: Use Python's Pandas library to load data into the working environment and view the basic information, statistical summary, and distribution of the data; Use Pandas’ duplicated() method to identify duplicate records, use Pandas’ drop_duplicates() method to delete duplicate records, ensure that each record in the dataset is unique, use Pandas’ isnull() to identify missing values, and check the number and distribution of missing values; Identify erroneous data by comparing data validation rules. After filling missing values ​​and correcting erroneous data, recheck the basic information, statistical summary and distribution of the data to ensure the consistency and accuracy of the data. Use data validation rules and visualization tools to compare with other data sources to recheck the quality and completeness of the data. Save the preprocessed data to a new file or database for subsequent analysis. During the preprocessing process, regularly back up the original data and intermediate results to prevent data loss or damage.

3. A method for dynamic monitoring of subject hot spots according to claim 1, characterized in that: The extraction step using natural language processing technology in step S2 includes: Preprocess the text to remove noise and irrelevant information, cut the text into individual words, and remove stop words in the text; For feature extraction, the frequency of each word in the document is counted to measure the ability of a word to distinguish documents in the document set. The TF-IDF algorithm is used to combine word frequency and inverse document frequency to calculate the score of each word. The words with high scores are used as keywords. Then, the TextRank algorithm is used to sort the text based on the graph. The relationship between local words is used to sort the subsequent keywords. The extracted keywords are filtered, sorted, deduplicated and merged to obtain the final keyword list. Use the LSA algorithm to extract topics from texts, map the texts to a low-dimensional topic space, and then extract topics. Perform post-processing on the extracted topics, merge similar topics, and remove irrelevant topics. Conduct in-depth grammatical and semantic analysis of the text, including part-of-speech tagging and named entity recognition. Named entity recognition can identify specific types of entities in the text and match the identified entities with a known author database to find possible author information. The text contains information related to the author's name, which is directly extracted and then verified to ensure its accuracy.

4. A method for dynamically monitoring subject hot spots according to claim 1, characterized in that: In step S2, the steps of using association rule mining technology to find the association between different research fields include: Determine the research field and collect data related to these research fields. The data should include the identification of different research fields and the possible association information between them. Clean and organize the collected data, remove duplicate, invalid or noisy data, and convert the data into a format suitable for association rule mining. Select appropriate association rule mining algorithms based on data characteristics and mining requirements, determine support and confidence thresholds, and use these two parameters to filter frequent item sets and generate association rules. Support represents the frequency of item sets in a data set, and confidence represents the accuracy of association rules. Use the selected algorithms and parameters to mine data and generate association rules that meet support and confidence thresholds. Filter out the rules that are strongly related to the research field from the mined association rules, filter them according to the support, confidence and lift indicators of the rules, interpret the filtered association rules, and clarify the correlation between the different research fields they represent; The mined association rules are displayed in a graphical way.

5. The method for dynamic monitoring of subject hot spots according to claim 1 is characterized by: The step of using the stream processing framework to process real-time data in step S3 includes: According to the specific needs of processing real-time data, select a suitable stream processing framework, build and configure the operating environment required by the stream processing framework, including hardware resources, software dependencies, and network configuration, and connect the real-time data source to the stream processing framework; Clean the incoming real-time data, remove invalid, duplicate or noisy data, ensure data quality, convert the data into a format suitable for subsequent processing and analysis, and use the data streaming computing function provided by the streaming processing framework to process and analyze the data in real time; According to the characteristics of the data and the application scenario, select the appropriate anomaly detection algorithm, use historical data to train the anomaly detection algorithm, generate an initialization model, input real-time data into the trained anomaly detection model, perform real-time anomaly detection, and mark or record the detected abnormal data points for subsequent analysis; Filter out data related to research hotspots or emergencies from detected abnormal data points, conduct in-depth analysis of the filtered abnormal data, explain the causes and meanings of the abnormalities in combination with domain knowledge, analyze the distribution, frequency, and trend of abnormal data points to discover potential research hotspots or emergencies, and formulate corresponding response strategies or decision plans based on the analysis results; Monitor the operating status of the streaming processing framework and anomaly detection system in real time to ensure the stability and reliability of the system. Optimize the performance of the streaming processing framework and anomaly detection algorithm based on the system's operating conditions and processing efficiency. Continuously optimize and update the streaming processing framework and anomaly detection system as data continues to accumulate and application scenarios change.

6. A method for dynamic monitoring of subject hot spots according to claim 1, characterized in that: In step S4, the analysis results and hot information are organized into a knowledge base to facilitate subsequent research and reference, and the hot information is released through academic websites and social media to promote the dissemination and sharing of knowledge.

7. A system for dynamic monitoring of subject hot spots, comprising a data acquisition module, a data preprocessing unit, a central processing unit, a real-time monitoring module, a system maintenance and optimization module, a hot spot identification unit, a knowledge management unit, a visualization display module and a user interaction module, characterized in that: The output end of the data acquisition module is electrically connected to the input end of the data preprocessing unit, the output end of the data preprocessing unit is electrically connected to the input end of the central processing unit, the output end of the central processing unit is electrically connected to the input ends of the hotspot identification unit, the visualization display module, the knowledge management unit and the real-time monitoring module respectively, the output end of the real-time monitoring module is electrically connected to the input end of the system maintenance and optimization module, the output end of the visualization display module is electrically connected to the input end of the user interaction module, the data acquisition module can collect raw data from multiple data sources, the real-time monitoring module can process and monitor data streams in real time, and promptly discover new hotspots and abnormal situations, the system maintenance and optimization module can continuously optimize system performance and functions, and collect user feedback on system functions and results, the visualization display module can display the analysis results to the user in a graphical manner, and the user interaction module can provide a user-friendly interface so that users can easily query and operate data.

8. A subject hotspot dynamic monitoring system according to claim 7, characterized in that: The data preprocessing unit includes a data cleaning module, a data integration module, a data conversion module and a data reduction module. The data cleaning module is the core part of the data preprocessing unit and is responsible for eliminating noise in the data and correcting its inconsistent errors. The data integration module is responsible for merging data from multiple data sources together to form a complete data set. The data conversion module is responsible for format conversion, unit conversion and other processing on the data to meet the needs of subsequent analysis. The data reduction module aims to reduce the size of the data while minimizing the impact on the final mining results.

9. A subject hotspot dynamic monitoring system according to claim 7, characterized in that: The hot spot identification unit includes a topic identification module, a hot spot extraction module and a trend analysis module. The topic identification module uses natural language processing to perform text analysis and topic extraction on the collected data, identifies key themes and topics in the data, and provides a basis for subsequent hot spot analysis. The hot spot extraction module uses statistical analysis and data mining technology based on the results of the topic identification module to extract hot topics and trends from the data. The trend analysis module uses time series analysis and prediction technology to predict and analyze the development trend of hot topics.

10. A subject hotspot dynamic monitoring system according to claim 7, characterized in that: The knowledge management unit includes a knowledge storage module, a knowledge retrieval module, a knowledge sharing module, a knowledge update and maintenance module and an intelligent recommendation module. The knowledge acquisition module is responsible for automatically collecting knowledge related to subject hotspots from various data sources. The knowledge storage module uses database technology to store the collected knowledge in a structured manner to facilitate subsequent retrieval and analysis. The knowledge retrieval module uses search engines and information retrieval technology to provide users with convenient knowledge retrieval services. The knowledge sharing module supports users to share and disseminate knowledge in the knowledge base, including internal sharing and external publishing. The knowledge update and maintenance module is responsible for regularly updating the knowledge in the knowledge base to ensure the accuracy and timeliness of the knowledge. At the same time, it is also responsible for processing user feedback and opinions, and maintaining and optimizing the knowledge base. The intelligent recommendation module uses machine learning algorithms and recommendation technologies to recommend relevant knowledge and resources to users based on their interests and behaviors.

Images