Hotspot mining system and method based on Internet financial information, and medium
By designing a multi-module Internet financial information hotspot mining system, the problem of massive financial information integration and analysis is solved, efficient integration and precise mining of financial information is achieved, information quality and analysis accuracy are improved, and more scientific decision-making and in-depth research are supported.
Patent Information
- Application Number
- CN202411641565.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-02-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
It is difficult for existing technology to effectively integrate and analyze massive and scattered Internet financial information, resulting in untimely and inaccurate information acquisition, and it is difficult to accurately tap financial hot spots.
A hot spot mining system based on Internet financial information was designed, including information collection module, text preprocessing module, hot spot evaluation index calculation module, hot spot mining and analysis module and visual display module. The system collects financial information through network crawlers, performs text preprocessing and semantic analysis, calculates hot spot evaluation indicators, and conducts in-depth analysis and visual display.
It realizes efficient integration and precise mining of financial information, improves information quality and analysis accuracy, saves users' information search and sorting time, and supports more scientific decision-making and in-depth research.
Smart Images

Figure CN119537665A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of Internet financial hotspot mining systems, and in particular to a hotspot mining system, method and medium based on Internet financial information. Background Art
[0002] In today's digital age, financial information on the Internet is exploding. Investors, financial institutions, and financial researchers are in urgent need of timely and accurate access to hot financial information in order to make reasonable investment decisions, formulate business strategies, or conduct academic research. However, in the face of massive and complex Internet financial information, existing information acquisition and analysis methods have many shortcomings.
[0003] On the one hand, information sources are scattered and diverse. Financial news websites, forums, social media, and various research report platforms all contain a wealth of financial information, but this information is scattered in different corners of the Internet and lacks an effective integration mechanism. Investors often need to spend a lot of time switching between multiple platforms, making it difficult to fully and quickly grasp all relevant information.
[0004] On the other hand, the quality of information varies. It is difficult to distinguish true from false financial information on the Internet. These bad information not only interferes with users' judgment of real hot spots, but may even lead to wrong decisions.
[0005] In addition, traditional information analysis methods are mostly based on manual experience and simple statistical methods, which are difficult to cope with large-scale, highly dynamic Internet financial information. Manual analysis cannot process massive amounts of data in a timely manner, and is prone to miss hot events with strong timeliness, and there are limitations in the depth and accuracy of analysis. For example, it is difficult for manual analysis to conduct comprehensive and in-depth mining and accurate prediction of complex market trends and financial events related to multiple factors.
[0006] With the continuous development of information technology, such as big data processing technology, natural language processing technology and machine learning algorithms, the technology has gradually matured, providing technical feasibility for solving the above problems. By building an intelligent hotspot mining system, it is possible to integrate scattered financial information sources, use advanced algorithms to screen high-quality information and accurately mine hotspots, meet the market's demand for efficient acquisition and in-depth analysis of financial hotspot information, and promote scientific decision-making and in-depth research in the financial field. Summary of the invention
[0007] The present invention proposes a hotspot mining system, method and medium based on Internet financial information to solve the problems mentioned in the above-mentioned prior art.
[0008] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a hot spot mining system based on Internet financial information, comprising:
[0009] The information collection module deploys multiple web crawlers, sets crawling rules for financial news websites, financial forums, social media financial sections, and industry research report platforms, and dynamically adjusts the collection cycle according to the update frequency of different information sources;
[0010] The text preprocessing module is connected to the information collection module to clean the collected text information, remove HTML tags, special characters, stop words, and perform stem extraction or word form restoration on the text; use semantic analysis technology to identify and correct typos and grammatical errors in the text, and standardize professional financial terms; divide long texts into paragraphs or sentences based on semantics, and classify and annotate the text, and store the preprocessed text data in the intermediate database;
[0011] The hotspot evaluation index calculation module reads the preprocessed text data from the intermediate database, calculates the weights of different categories of information based on text classification, including macroeconomic information weights set to 0.3, industry dynamics to 0.25, company financial reports to 0.2, investment strategy to 0.15, and others to 0.1; counts the frequency of keywords in the text, uses the TF-IDF algorithm to calculate the importance of keywords, and calculates the timeliness score based on the information release time. The formula is: hotspot index = category weight × (keyword frequency × TF-IDF value) × timeliness score, and stores the calculation results in the evaluation index database;
[0012] The hotspot mining and analysis module sets the hotspot threshold according to the results of the hotspot evaluation index calculation module. When the hotspot index exceeds the threshold, it is determined to be hot information. The hotspot information is deeply analyzed, including using the sentiment analysis algorithm to judge the sentiment tendency of the information and clustering the related hotspot information into one category through the text clustering algorithm.
[0013] Visual display module, obtain the hotspot analysis report generated by the hotspot mining and analysis module.
[0014] Preferably, the web crawler in the information collection module has the functions of intelligently identifying verification codes and breaking through IP restrictions, and is deployed in a distributed environment; it supports the collection of multiple data formats, including HTML, XML, and JSON, and encrypts and transmits the collected data.
[0015] Preferably, the semantic analysis technology in the text preprocessing module is built based on a deep learning framework, including a BERT model, and the training data covers a large amount of text in the financial field; the standardization of professional financial terms is based on authoritative financial dictionaries and industry standards, and the stop word list combines general stop words and financial stop words.
[0016] Preferably, the TF-IDF algorithm in the hotspot evaluation index calculation module considers the characteristics of vocabulary in the financial field during calculation and assigns higher weights to professional terms; the timeliness score calculation adopts an exponential decay function, including the highest score within 1 hour after release and gradually decreasing over time; the hotspot threshold can be dynamically adjusted according to user needs and historical data.
[0017] Preferably, the sentiment analysis algorithm in the hotspot mining and analysis module adopts a model structure based on the combination of convolutional neural network CNN and recurrent neural network RNN, and the training data is annotated with accurate sentiment labels; the text clustering algorithm adopts DBSCAN or K-Means++ algorithm, and clusters according to the similarity of hotspot information, and the similarity calculation comprehensively considers text content, keywords, and release time factors.
[0018] Preferably, a hotspot mining method based on Internet financial information comprises the following steps:
[0019] The information collection step is to start the web crawler to collect Internet financial information through the information collection module according to the preset information source list and collection rules, monitor the network status in real time during the collection process, automatically retry or switch to the backup collection node in case of network failure, remove duplicates and filter invalid information from the collected information, and store it according to the information category and source. The storage format adopts a structured data format, including a relational database table structure, and records the collection log, including the collection time, collection quantity, and collection failure reasons, so as to facilitate subsequent analysis and optimization of the collection strategy;
[0020] Text preprocessing step: Use the text preprocessing module to read the stored financial information and perform text cleaning, including noise removal and standardization. Use natural language processing technology to perform part-of-speech tagging and named entity recognition, including identifying company names and stock codes. Perform syntactic analysis on the text according to semantic and grammatical rules, such as analyzing sentence structure and subject-verb-object relationships. Convert the processed text into vector representation, including word vector or sentence vector. The vector model used is Word2Vec or GloVe. Store the text data after vector representation in the intermediate repository.
[0021] The hotspot evaluation index calculation step is to read the text vector data from the intermediate repository and calculate the topic relevance score of the text, including using the LDA topic model to calculate the relevance to the financial topic, and combining the keyword weight and information timeliness to calculate the hotspot evaluation index. The keyword weight calculation comprehensively considers the word frequency, inverse document frequency and lexical semantic importance. The timeliness calculation refers to the information release time and the current time interval. The hotspot evaluation index is obtained by weighted summation. The calculation result is compared with the preset threshold to screen out potential hotspot information. The threshold setting can be adjusted according to historical data and user needs.
[0022] Hotspot mining and analysis steps: Use the hotspot mining and analysis module to conduct in-depth analysis on potential hotspot information, including using sentiment analysis models to determine the sentiment polarity of information, including positive, negative, and neutral, analyzing sentiment intensity, and using text clustering methods to group similar hotspot information together. Mining the correlation between hot events, including the connections between related companies and industries, generating hot event maps, showing the development context and influencing factors of hot events, and predicting the development trend of hot events, including the trend of heat change and the trend of expansion of the scope of influence.
[0023] The visualization and push step displays the results of the hotspot mining and analysis steps through a visualization module. The display forms include dynamic charts and interactive interfaces. Users can choose to view different types of hotspot information and hotspot trends in different time periods through the operation interface. Hotspot content can be pushed to users based on their subscription information and interest preferences.
[0024] Preferably, in the information collection step, different collection resources are allocated according to the characteristics and importance of different information sources, including allocating more bandwidth and number of crawlers to authoritative financial news websites; the information source list is updated regularly, new financial information sources are added, invalid or low-quality information sources are deleted, and the collected information is backed up once a day, and the backup data is stored in an off-site storage device.
[0025] Preferably, in the text preprocessing step, the part-of-speech tagging and named entity recognition models are expanded and optimized according to the characteristics of financial terminology, including adding the names of newly emerging financial products; in the syntactic analysis process, the grammatical rules of the financial field are combined, including the analysis of sentences related to financial statements.
[0026] Preferably, in the hotspot evaluation index calculation step, external knowledge graphs are introduced, including company equity relationship graphs and industry upstream and downstream relationship graphs, to enhance the relevance and accuracy of hotspot evaluation; hotspot thresholds are re-evaluated and adjusted regularly, and optimized according to the historical distribution of hotspot events and current market environment changes, such as lowering the threshold during periods of large market fluctuations to increase hotspot sensitivity.
[0027] Preferably, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the hotspot mining method based on Internet financial information are implemented.
[0028] Compared with the prior art, the present invention has the following beneficial effects:
[0029] The present invention has significant beneficial effects. First, it realizes efficient integration and accurate mining of financial information. The system can automatically collect financial information from multiple different sources, screen and analyze it through intelligent algorithms, quickly and accurately identify hot events and trends, save users a lot of information search and sorting time, and enable users to grasp market trends in a timely manner and make more informed investment decisions or adjust research directions.
[0030] Secondly, it improves the quality of information and the accuracy of analysis. Through text preprocessing and hotspot evaluation index calculation, false and misleading information is effectively removed, and the importance and relevance of information are scientifically evaluated. Combined with in-depth analysis and visual display, users can gain an in-depth understanding of the overall picture, emotional tendencies, and potential impacts of hot events from multiple dimensions, providing more reliable data support for decision-making.
[0031] Furthermore, it has good adaptability and scalability. The system can dynamically adjust parameters such as collection strategies and hot spot thresholds according to market changes and user needs to meet the needs of financial hot spot mining in different periods and fields. At the same time, the modular design facilitates subsequent function expansion and algorithm optimization, which can continuously improve system performance.
[0032] In addition, it promotes the widespread dissemination and sharing of information. The visual display and push functions enable hot information to be delivered to subscribers in a timely manner. Whether professional investors or ordinary financial enthusiasts, they can easily obtain valuable financial hot content, which enhances the transparency and circulation of market information and is conducive to the healthy development of the entire financial field. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A schematic block diagram of a hotspot mining system based on Internet financial information proposed by the present invention;
[0034] Figure 2 This is a schematic block diagram of a hotspot mining method based on Internet financial information proposed by the present invention. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise" and "counterclockwise" indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.
[0037] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, and it can be the internal connection of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0038] Reference Figure 1-2 :A hot spot mining system based on Internet financial information, comprising:
[0039] The information collection module deploys multiple web crawlers, sets specific crawling rules for financial news websites, financial forums, social media financial sections, and industry research report platforms, and dynamically adjusts the collection cycle according to the update frequency of different information sources. For example, financial news websites are crawled once an hour, and social media financial sections are monitored and crawled in real time. The collected information is preliminarily screened to remove duplicate and invalid information, such as advertisements and spam. At the same time, key information is extracted, including information title, release time, publisher, content summary, and the source of information is marked. The screened information is stored in the local database to ensure data integrity and accuracy.
[0040] The text preprocessing module is connected to the information collection module to clean the collected text information, remove HTML tags, special characters, stop words, and perform stem extraction or word form restoration on the text, such as restoring "running" to "run"; it uses semantic analysis technology to identify and correct typos and grammatical errors in the text, and standardizes professional financial terms, such as unifying the expressions of "price-earnings ratio" and "price-earnings ratio"; it divides long texts into paragraphs or sentences based on semantics to improve subsequent processing efficiency, and at the same time classifies and annotates the text, such as macroeconomics, industry trends, company financial reports, and investment strategies, and stores the preprocessed text data in an intermediate database.
[0041] The hotspot evaluation index calculation module reads the preprocessed text data from the intermediate database, and calculates the weights of different categories of information based on text classification. For example, the weight of macroeconomic information is set to 0.3, industry dynamics is 0.25, company financial reports is 0.2, investment strategy is 0.15, and others are 0.1; the frequency of occurrence of keywords in the text is counted, and the TF-IDF algorithm is used to calculate the importance of keywords. The timeliness score is calculated in combination with the information release time. For example, the closer the release time, the higher the score. The hotspot evaluation index is calculated comprehensively through a specific formula. The formula is: hotspot index = category weight × (keyword frequency × TF-IDF value) × timeliness score, and the calculation results are stored in the evaluation index database;
[0042] Term frequency (TF) indicates the number of times a word appears in a document, and inverse document frequency (IDF) is used to measure the importance of a word. In this system, considering the characteristics of vocabulary in the financial field, the TF-IDF algorithm is improved to more accurately calculate the importance of keywords;
[0043] Suppose the document set is D, and the frequency of keyword t in document d is tf t,d , the total number of documents is N, and the number of documents containing keyword t is df t ;
[0044] The improved TF-IDF formula is: Where α is an adjustment factor (the value range is 0-1, and can be adjusted according to the characteristics of vocabulary distribution in the financial field. For example, for high-frequency and important financial terms, the α value can be appropriately increased).
[0045] Among them, β is the weight coefficient (the value range is 0-1, which is used to adjust the impact of the overall frequency of occurrence of a keyword in the entire document collection on its importance. If a keyword appears frequently and is more important in the entire document collection, the β value can be appropriately increased).
[0046] The hotspot mining and analysis module sets the hotspot threshold based on the results of the hotspot evaluation index calculation module. When the hotspot index exceeds the threshold, it is judged as hot information. In-depth analysis of the hotspot information is performed, including the use of sentiment analysis algorithms to determine the emotional tendency of the information, such as positive, negative, and neutral. Related hotspot information is clustered into one category through text clustering algorithms, such as hot events in the same industry. The propagation path, propagation speed, and impact range of hotspot information are analyzed, such as the number of reposts and comments on social media, and a hotspot analysis report is generated. The report includes a hotspot overview, heat trend, sentiment tendency, and related impacts.
[0047] The visualization display module obtains the hot spot analysis report generated by the hot spot mining and analysis module, and displays the hot spot information in the form of intuitive charts, such as a bar chart showing the hot spot popularity ranking, a line chart showing the trend of popularity over time, and a map showing the geographical distribution of hot events. It supports interactive operations, and users can click on the chart to view detailed information, such as the original text of the hot news and related comments. It also provides a hot spot information push function, which can push hot content to subscribed users, such as through emails and text messages.
[0048] In the present invention, the web crawler in the information collection module has the functions of intelligently identifying verification codes and breaking through IP restrictions, and can be deployed in a distributed environment to improve collection efficiency; it supports collection of multiple data formats, such as HTML, XML, and JSON, and encrypts and transmits the collected data to ensure data security.
[0049] In the present invention, the semantic analysis technology in the text preprocessing module is constructed based on a deep learning framework, such as the BERT model, and the training data covers a large amount of text in the financial field; the standardization processing of professional financial terms is based on authoritative financial dictionaries and industry standards, and the stop word list combines general stop words and specific stop words in the financial field.
[0050] In the present invention, the TF-IDF algorithm in the hotspot evaluation index calculation module takes into account the characteristics of vocabulary in the financial field during calculation and assigns higher weights to professional terms; the timeliness score calculation adopts an exponential decay function, such as the highest score within 1 hour after release and gradually decreases over time; the hotspot threshold can be dynamically adjusted according to user needs and historical data.
[0051] In the present invention, the sentiment analysis algorithm in the hotspot mining and analysis module adopts a model structure based on the combination of convolutional neural network (CNN) and recurrent neural network (RNN), and the training data is annotated with accurate sentiment labels; the text clustering algorithm adopts DBSCAN or K-Means++ algorithm, and clusters according to the similarity of hotspot information, and the similarity calculation comprehensively considers the text content, keywords, and release time factors.
[0052] In the present invention, a hotspot mining method based on Internet financial information includes the following steps:
[0053] The information collection step is to start the web crawler to collect Internet financial information through the information collection module as described in claim 1 according to the preset information source list and collection rules, monitor the network status in real time during the collection process, automatically retry or switch to the backup collection node in case of network failure, deduplicate the collected information, filter out invalid information, and classify and store it according to the information category and source. The storage format adopts a structured data format, such as a relational database table structure, and records the collection log, including the collection time, collection quantity, and collection failure reasons, so as to facilitate subsequent analysis and optimization of the collection strategy.
[0054] The text preprocessing step uses the text preprocessing module to read the stored financial information and perform text cleaning, including noise removal and standardization. It uses natural language processing technology to perform part-of-speech tagging and named entity recognition, such as identifying company names and stock codes. It performs syntactic analysis on the text according to semantic and grammatical rules, such as analyzing sentence structure and subject-verb-object relationships. The processed text is converted into a vector representation, such as a word vector or a sentence vector. The vector model used is Word2Vec or GloVe, and the text data after vector representation is stored in an intermediate repository.
[0055] The steps for calculating the hot spot evaluation index are as follows: read the text vector data from the intermediate repository, calculate the topic relevance score of the text, such as using the LDA topic model to calculate the relevance to financial topics, and calculate the hot spot evaluation index in combination with keyword weight and information timeliness. The keyword weight calculation comprehensively considers word frequency, inverse document frequency and lexical semantic importance. The timeliness calculation refers to the information release time and the current time interval. The hot spot evaluation index is obtained by weighted summation. The calculation result is compared with the preset threshold to screen out potential hot spot information. The threshold setting can be adjusted according to historical data and user needs.
[0056] Hotspot mining and analysis steps: Use the hotspot mining and analysis module to conduct in-depth analysis on potential hotspot information, including using sentiment analysis models to determine the sentiment polarity of information, such as positive, negative, and neutral, analyzing sentiment intensity, and using text clustering methods to group similar hotspot information together, mining the correlation between hot events, such as the connection between related companies and industries, generating hot event maps, showing the development context and influencing factors of hot events, and predicting the development trend of hot events, such as the trend of heat change and the trend of expansion of the scope of influence.
[0057] The visualization and push step displays the results of the hotspot mining and analysis steps through a visualization display module. The display forms include dynamic charts and interactive interfaces. Users can choose to view different types of hotspot information and hotspot trends in different time periods through the operation interface. The hotspot content is pushed to users based on their subscription information and interest preferences. The push methods include instant messaging tool push and mobile application push. The push content includes hotspot summaries and detailed report links.
[0058] In the present invention, in the information collection step, different collection resources are allocated according to the characteristics and importance of different information sources, such as allocating more bandwidth and crawlers to authoritative financial news websites; the information source list is updated regularly, new financial information sources are added, invalid or low-quality information sources are deleted, and the collected information is backed up once a day, and the backup data is stored in an off-site storage device.
[0059] In the present invention, in the text preprocessing step, according to the characteristics of financial terminology, the part-of-speech tagging and named entity recognition models are expanded and optimized, such as adding the names of newly emerging financial products; in the syntactic analysis process, the grammatical rules of the financial field are combined to improve the accuracy of syntactic analysis, such as the analysis of sentences related to financial statements; in the vector model training process, the latest financial news and report data are added, the vector representation is updated, and the accuracy and timeliness of the text vector are improved.
[0060] In the present invention, in the hotspot evaluation index calculation step, external knowledge graphs are introduced, such as company equity relationship graphs and industry upstream and downstream relationship graphs, to enhance the relevance and accuracy of hotspot evaluation; hotspot thresholds are re-evaluated and adjusted regularly, and optimized according to the historical distribution of hotspot events and current market environment changes, such as lowering the threshold during periods of large market fluctuations to increase hotspot sensitivity.
[0061] In the present invention, a computer-readable storage medium stores a computer program, characterized in that when the computer program is executed by a processor, the steps of a hotspot mining method based on Internet financial information are implemented.
[0062] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can replace or change the technical solution and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A hotspot mining system based on Internet financial information, characterized in that: include: The information collection module deploys multiple web crawlers, sets crawling rules for financial news websites, financial forums, social media financial sections, and industry research report platforms, and dynamically adjusts the collection cycle according to the update frequency of different information sources; The text preprocessing module is connected to the information collection module to clean the collected text information, remove HTML tags, special characters, stop words, and perform stem extraction or word form restoration on the text; Use semantic analysis technology to identify and correct typos and grammatical errors in texts, and standardize professional financial terms; Segment long text into paragraphs or sentences based on semantics, classify and annotate the text, and store the preprocessed text data in an intermediate database; The hotspot evaluation index calculation module reads the preprocessed text data from the intermediate database, and calculates the weights of different categories of information based on text classification, including macroeconomic information weights set to 0.3, industry dynamics to 0.25, company financial reports to 0.2, investment strategy to 0.15, and others to 0.1; the frequency of occurrence of keywords in the text is counted, and the TF-IDF algorithm is used to calculate the importance of keywords. The timeliness score is calculated in combination with the information release time. The formula is: hotspot index = category weight × (keyword frequency × TF-IDF value) × timeliness score, and the calculation results are stored in the evaluation index database; The hotspot mining and analysis module sets the hotspot threshold according to the results of the hotspot evaluation index calculation module. When the hotspot index exceeds the threshold, it is determined to be hot information. The hotspot information is deeply analyzed, including using the sentiment analysis algorithm to judge the sentiment tendency of the information and clustering the related hotspot information into one category through the text clustering algorithm. Visual display module, obtain the hotspot analysis report generated by the hotspot mining and analysis module.
2. The hotspot mining system based on Internet financial information according to claim 1 is characterized in that: The web crawler in the information collection module has the functions of intelligently identifying verification codes and breaking through IP restrictions, and is deployed in a distributed environment; it supports the collection of multiple data formats, including HTML, XML, and JSON, and encrypts the transmission of the collected data.
3. The hot spot mining system based on Internet financial information according to claim 1 is characterized in that: The semantic analysis technology in the text preprocessing module is built based on a deep learning framework, including the BERT model, and the training data covers a large amount of financial texts; the standardization of professional financial terms is based on authoritative financial dictionaries and industry standards, and the stop word list combines general stop words and financial stop words.
4. The hot spot mining system based on Internet financial information according to claim 1 is characterized in that: The TF-IDF algorithm in the hotspot evaluation index calculation module considers the characteristics of financial vocabulary during calculation and gives higher weight to professional terms; The timeliness score calculation uses an exponential decay function, including the highest score within 1 hour after release and gradually decreasing over time; Hotspot thresholds can be adjusted dynamically based on user needs and historical data.
5. The hotspot mining system based on Internet financial information according to claim 1 is characterized in that: The sentiment analysis algorithm in the hotspot mining and analysis module adopts a model structure based on the combination of convolutional neural network CNN and recurrent neural network RNN, and the training data is annotated with accurate sentiment labels; the text clustering algorithm adopts DBSCAN or K-Means++ algorithm, and clusters according to the similarity of hotspot information. The similarity calculation comprehensively considers text content, keywords, and release time factors.
6. A method for mining hot spots based on Internet financial information using any one of claims 1 to 5, characterized in that: The following steps are involved: The information collection step is to start the web crawler to collect Internet financial information through the information collection module according to the preset information source list and collection rules, monitor the network status in real time during the collection process, automatically retry or switch to the backup collection node in case of network failure, remove duplicates and filter invalid information from the collected information, and store it according to the information category and source. The storage format adopts a structured data format, including a relational database table structure, and records the collection log, including the collection time, collection quantity, and collection failure reasons, so as to facilitate subsequent analysis and optimization of the collection strategy; Text preprocessing step: Use the text preprocessing module to read the stored financial information and perform text cleaning, including noise removal and standardization. Use natural language processing technology to perform part-of-speech tagging and named entity recognition, including identifying company names and stock codes. Perform syntactic analysis on the text according to semantic and grammatical rules, such as analyzing sentence structure and subject-verb-object relationships. Convert the processed text into vector representation, including word vector or sentence vector. The vector model used is Word2Vec or GloVe. Store the text data after vector representation in the intermediate repository. The hotspot evaluation index calculation step is to read the text vector data from the intermediate repository and calculate the topic relevance score of the text, including using the LDA topic model to calculate the relevance to the financial topic, and combining the keyword weight and information timeliness to calculate the hotspot evaluation index. The keyword weight calculation comprehensively considers the word frequency, inverse document frequency and lexical semantic importance. The timeliness calculation refers to the information release time and the current time interval. The hotspot evaluation index is obtained by weighted summation. The calculation result is compared with the preset threshold to screen out potential hotspot information. The threshold setting can be adjusted according to historical data and user needs. Hotspot mining and analysis steps: For potential hotspot information, use the hotspot mining and analysis module to conduct in-depth analysis, including using sentiment analysis models to determine the sentiment polarity of information, including positive, negative, and neutral, analyzing sentiment intensity, using text clustering methods to group similar hotspot information together, mining the correlation between hot events, including the connection between related companies and industries, generating hot event maps, showing the development context and influencing factors of hot events, and predicting the development trend of hot events, including the trend of heat change and the trend of expansion of the scope of influence; The visualization and push step displays the results of the hotspot mining and analysis steps through a visualization module. The display forms include dynamic charts and interactive interfaces. Users can choose to view different types of hotspot information and hotspot trends in different time periods through the operation interface. Hotspot content can be pushed to users based on their subscription information and interest preferences.
7. The hotspot mining method based on Internet financial information according to claim 6 is characterized in that: In the information collection step, different collection resources are allocated according to the characteristics and importance of different information sources, including allocating more bandwidth and number of crawlers to authoritative financial news websites; regularly update the information source list, add new financial information sources, delete invalid or low-quality information sources, and back up the collected information once a day. The backup data is stored in off-site storage devices.
8. The hotspot mining method based on Internet financial information according to claim 6 is characterized in that: In the text preprocessing step, based on the characteristics of financial terminology, the part-of-speech tagging and named entity recognition models are expanded and optimized, including the addition of new names of financial products; in the syntactic analysis process, the grammatical rules of the financial field are combined, including the analysis of sentences related to financial statements.
9. The hotspot mining method based on Internet financial information according to claim 6, characterized in that: In the calculation step of hotspot evaluation indicators, external knowledge graphs are introduced, including company equity relationship graphs and industry upstream and downstream relationship graphs, to enhance the relevance and accuracy of hotspot evaluation; hotspot thresholds are re-evaluated and adjusted regularly, and optimized according to the historical distribution of hotspot events and current market environment changes, such as lowering the threshold during periods of large market fluctuations to increase hotspot sensitivity.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the hot spot mining method based on Internet financial information are implemented as described in claims 6-9.
Citation Information
Cited By
Geographic information mining method and system based on big data
CN119884215A
AI intelligent manuscript creation method and system based on historical manuscripts
CN120561355A
An AI intelligent manuscript creation method and system based on historical manuscripts
CN120561355B
Hotspot information monitoring and processing system and method
CN121146846A
Content analysis method, system and equipment based on intelligent information acquisition and medium
CN121188531A