Public opinion popularity prediction system based on Transform + LSTM (Long Short Term Memory) hybrid architecture
The public opinion heat prediction system based on the Transformer+LSTM hybrid architecture solves the problems of weak modeling ability and insufficient processing of time series information in traditional methods in public opinion analysis, and achieves accurate prediction of public opinion heat, especially its superiority in long texts and complex semantic relationships.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AIR FORCE COMM SERGEANT SCHOOL OF PLA
- Filing Date
- 2025-12-08
- Publication Date
- 2026-05-15
AI Technical Summary
Traditional public opinion analysis methods suffer from weak modeling capabilities and insufficient processing of time-series information when dealing with large-scale text data, making it difficult to accurately predict the intensity of public opinion.
A public opinion heat prediction system based on a Transformer+LSTM hybrid architecture is adopted. News data is collected by web crawling tools, multi-dimensional features are extracted and standardized, the Transformer module is used to capture sequence dependencies, and the LSTM module is combined to learn the dynamic changes of time series data. Finally, the predicted value is generated through a fully connected layer.
It achieves more comprehensive and accurate prediction of public opinion heat, especially showing unique superiority in handling long texts and complex semantic relationships, combining the powerful semantic representation capability of Transformer and the temporal context memory capability of LSTM.
Smart Images
Figure CN122048339A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of public opinion heat prediction technology, and more specifically, to a public opinion heat prediction system based on a Transformer+LSTM hybrid architecture. Background Technology
[0002] Public opinion monitoring and technical analysis are crucial information processing technologies in today's internet environment. With the rapid development of social media and online platforms, a massive amount of user-generated text data is constantly emerging, containing a wealth of public opinion information. Traditional public opinion analysis methods typically rely on simple text feature extraction and statistical analysis, which suffer from weak modeling capabilities and insufficient processing of time-series data. Therefore, how to extract valuable information from massive amounts of data and accurately predict public opinion trends has become an urgent problem to be solved.
[0003] In recent years, deep learning methods have been widely applied in the field of natural language processing due to their powerful pattern recognition and feature learning capabilities. The Transformer model, with its self-attention mechanism, excels in handling long-distance dependencies and rich contextual information, making it suitable for sentiment analysis tasks involving large-scale text data. However, simply using the Transformer model may be insufficient in capturing temporal changes in data, especially in applications such as sentiment trend prediction that require reflecting the dynamic characteristics of time-series data. On the other hand, traditional time-series models such as LSTM (Long Short-Term Memory) can better preserve temporal information when processing time-series data, but often have limitations in representing heterogeneous textual information. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, this invention provides a public opinion heat prediction system based on a Transformer+LSTM hybrid architecture, which has the advantage of accurately predicting public opinion heat.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a public opinion heat prediction system based on a Transformer+LSTM hybrid architecture, comprising a database, a data acquisition device, a feature extraction device, a feature processing device, and a public opinion prediction device. The output of the data acquisition device is connected to the input of the feature extraction device, the output of the feature extraction device is connected to the input of the feature processing device, and the output of the feature processing device is connected to the input of the public opinion prediction device. The data acquisition device, the feature extraction device, the feature processing device, and the public opinion prediction device are all bidirectionally connected to the database.
[0006] The data acquisition device uses web crawlers to crawl news data and store it in a database;
[0007] Feature extraction device, which uses extraction tools to extract features from news data in multiple dimensions;
[0008] The feature processing device performs feature standardization and normalization on the extracted dimensional features, and then processes the standardized and normalized dimensional features through the Transformer module. During the processing, the sequence dependency relationship is obtained through the Transformer module and the LSTM module.
[0009] The public opinion prediction device uses the encoder in the Transformer module to process the dimensional features, activates them with the ReLU activation function, and then feeds them into a fully connected layer to generate the final prediction value.
[0010] As a preferred embodiment of the present invention, the feature processing device includes a feature processing module, a calculation processing module, and a sequence relationship module. The output end of the feature processing module is connected to the input end of the calculation processing module, and the input end of the calculation processing module is connected to both the sequence relationship module and the input end of the public opinion prediction device.
[0011] The feature processing module uses MinMaxScaler for feature standardization and constructs a time window sequence. It maps the original features to the model's hidden expansion dimension and then normalizes them.
[0012] The calculation and processing module processes the standardized and normalized dimensional features through the Transformer module and inputs the processing results into the input terminal of the public opinion prediction device.
[0013] The sequence relation module captures sequence dependencies during the processing by the encoder in the Transformer module, and inputs the processing results into the LSTM module to obtain long-term sequence dependencies.
[0014] As a preferred embodiment of the present invention, the Transformer module includes an encoder and a decoder. Six encoders are superimposed on a single encoder and the Transformer architecture is used for data processing. Six decoders are superimposed on a single decoder and the Transformer architecture is used for data processing.
[0015] As a preferred technical solution of the present invention, the extracted dimensional features are four dimensions: number of comments, proportion of negative sentiment, user activity, and number of reposts.
[0016] As a preferred embodiment of the present invention, the data acquisition device includes a data interface module, a strategy setting module, a template creation module, and an information acquisition module:
[0017] The data interface module provides a standard interface for data interaction with big data platforms.
[0018] The strategy setting module is used to set the collection strategy for network information collection;
[0019] The template creation module is used to create templates for collecting network information.
[0020] The information collection module is used to collect network popularity information data according to the set collection strategy and collection template.
[0021] As a preferred embodiment of the present invention, it further includes a preprocessing device, wherein the input end and the output end of the preprocessing device are respectively connected to the output end of the data acquisition device and the input end of the feature extraction device;
[0022] The preprocessing device can divide the collected data according to certain rules and transmit the divided data to the feature extraction device.
[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0024] This invention integrates the Transformer and LSTM models, combining their advantages. It leverages the powerful semantic representation capabilities of the Transformer while retaining the temporal context memory capabilities of the LSTM when processing sequential data. This hybrid architecture can predict public opinion trends more comprehensively and accurately, demonstrating unique superiority, especially when dealing with long texts and complex semantic relationships. Attached Figure Description
[0025] Figure 1 This is a schematic diagram of the structure of the present invention;
[0026] Figure 2 This is a schematic diagram of the pretreatment device structure of the present invention;
[0027] Figure 3 This is a schematic diagram of the data acquisition device and feature processing device of the present invention. Detailed Implementation
[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0029] like Figures 1 to 3As shown, this invention provides a public opinion heat prediction system based on a Transformer+LSTM hybrid architecture, including a database, a data acquisition device, a feature extraction device, a feature processing device, and a public opinion prediction device. The output of the data acquisition device is connected to the input of the feature extraction device, the output of the feature extraction device is connected to the input of the feature processing device, and the output of the feature processing device is connected to the input of the public opinion prediction device. The data acquisition device, the feature extraction device, the feature processing device, and the public opinion prediction device are all bidirectionally connected to the database.
[0030] The data acquisition device uses web crawlers to crawl news data and store it in a database;
[0031] Feature extraction device, which uses extraction tools to extract features from news data in multiple dimensions;
[0032] The feature processing device performs feature standardization and normalization on the extracted dimensional features, and then processes the standardized and normalized dimensional features through the Transformer module. During the processing, the sequence dependency relationship is obtained through the Transformer module and the LSTM module.
[0033] The public opinion prediction device uses the encoder in the Transformer module to process the dimensional features, activates them with the ReLU activation function, and then feeds them into a fully connected layer to generate the final prediction value.
[0034] The Transformer module includes an encoder and a decoder. Six encoders are superimposed on a single encoder and the Transformer architecture is used for data processing. Six decoders are superimposed on a single decoder and the Transformer architecture is used for data processing.
[0035] It's important to note that the Transformer model is a deep learning model, particularly popular in natural language processing. It introduces a novel mechanism—self-attention—that enables the model to process sequential data more efficiently and effectively. The Transformer is entirely based on attention mechanisms and does not use recurrent neural networks or convolutional neural networks. Its core concept is:
[0036] Self-attention mechanism: This allows the model to directly establish dependencies between arbitrary positions within a sequence, thereby better understanding the contextual relationships of the data. Self-attention can process data at all positions in parallel, which improves computational efficiency.
[0037] Multi-head attention: The model learns different representations of the data simultaneously, with each "head" focusing on a different part of the sequence. This mechanism can capture multiple levels of dependencies in the sequence.
[0038] Feedforward neural network: Performs non-linear transformation on each token to improve the model's expressive power.
[0039] Positional encoding: Since Transformer does not contain loop structures, positional encoding is used to preserve word order information.
[0040] The essence of a Transformer is composed of encoders and decoders. Multiple encoders can be stacked on top of each other; the older Transformer architecture stacked six, and the same applies to decoders. Within each encoder and decoder, there are corresponding functional layers to process the specific business data.
[0041] Input represents the source text, which is transformed through an input embedding layer. This layer acts like a dictionary, looking up a corresponding vector for each token. After obtaining the token embedding, a positional encoding is added. This positional encoding helps the model understand the relative positions of the tokens. The token embedding and positional encoding then enter the multi-head attention module.
[0042] This module allows the model to focus on different parts of the input sequence simultaneously. Residual connections and layer normalization modules are applied after the attention mechanism. These structures and techniques effectively prevent gradient vanishing, thus helping the model training converge and enabling the training of deeper networks.
[0043] The feature processing device includes a feature processing module, a calculation processing module, and a sequence relationship module. The output of the feature processing module is connected to the input of the calculation processing module, and the input of the calculation processing module is connected to both the sequence relationship module and the input of the public opinion prediction device.
[0044] The feature processing module uses MinMaxScaler for feature standardization and constructs a time window sequence. It maps the original features to the model's hidden expansion dimension and then normalizes them.
[0045] The calculation and processing module processes the standardized and normalized dimensional features through the Transformer module and inputs the processing results into the input terminal of the public opinion prediction device.
[0046] The sequence relation module captures sequence dependencies during the processing by the encoder in the Transformer module, and inputs the processing results into the LSTM module to obtain long-term sequence dependencies.
[0047] The extracted dimensional features are four dimensions: number of comments, percentage of negative sentiment, user activity, and number of reposts.
[0048] The data acquisition device includes a data interface module, a strategy setting module, a template creation module, and an information acquisition module.
[0049] The data interface module provides a standard interface for data interaction with big data platforms.
[0050] The strategy setting module is used to set the collection strategy for network information collection;
[0051] The template creation module is used to create templates for collecting network information.
[0052] The information collection module is used to collect network popularity information data according to the set collection strategy and collection template.
[0053] The data collection strategy includes the depth, scope, frequency, and priority of the collected information. The data collection template includes the following fields: username, title, content, time, source, keywords, search trends, geographic scope, and target audience.
[0054] As a preferred embodiment of the present invention, it further includes a preprocessing device, wherein the input end and the output end of the preprocessing device are respectively connected to the output end of the data acquisition device and the input end of the feature extraction device;
[0055] The preprocessing device can divide the collected data according to certain rules and transmit the divided data to the feature extraction device.
[0056] Preprocessing methods include: dividing the data to be processed into groups according to time order or according to topic popularity, to obtain x groups of the first dataset about x topics.
[0057] Working principle and usage process of this invention:
[0058] News data is crawled and stored in a database using a data acquisition device. Then, a feature extraction device extracts features in four dimensions: number of comments, proportion of negative sentiment, user activity, and number of reposts. The feature processing device uses MinMaxScaler to standardize the features and constructs a time window sequence. By mapping the original features to the model's hidden expansion dimension and then normalizing them, the input is processed by the encoder of the Transformer module to capture sequence dependencies. The output of the Transformer module is fed into the LSTM module to learn the dynamic changes of the time series data. After activation using the ReLU activation function, it is fed into a fully connected layer to generate the final predicted value.
[0059] By fusing the Transformer and LSTM models, the advantages of both can be combined. This approach leverages the powerful semantic representation capabilities of the Transformer while retaining the temporal contextual memory capabilities of the LSTM when processing sequential data. This hybrid architecture can predict public opinion trends more comprehensively and accurately, demonstrating unique superiority, especially when dealing with long texts and complex semantic relationships.
[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0061] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A public opinion heat prediction system based on a Transformer+LSTM hybrid architecture, characterized by: It includes a database, a data acquisition device, a feature extraction device, a feature processing device, and a public opinion prediction device. The output of the data acquisition device is connected to the input of the feature extraction device, the output of the feature extraction device is connected to the input of the feature processing device, and the output of the feature processing device is connected to the input of the public opinion prediction device. The data acquisition device, the feature extraction device, the feature processing device, and the public opinion prediction device are all bidirectionally connected to the database. The data acquisition device uses web crawlers to crawl news data and store it in a database; Feature extraction device, which uses extraction tools to extract features from news data in multiple dimensions; The feature processing device performs feature standardization and normalization on the extracted dimensional features, and then processes the standardized and normalized dimensional features through the Transformer module. During the processing, the sequence dependency relationship is obtained through the Transformer module and the LSTM module. The public opinion prediction device uses the encoder in the Transformer module to process the dimensional features, activates them with the ReLU activation function, and then feeds them into a fully connected layer to generate the final prediction value.
2. The public opinion heat prediction system based on the Transformer+LSTM hybrid architecture according to claim 1, characterized in that: The data acquisition device includes a data interface module, a strategy setting module, a template creation module, and an information acquisition module. The data interface module provides a standard interface for data interaction with big data platforms. The strategy setting module is used to set the collection strategy for network information collection; The template creation module is used to create templates for collecting network information. The information collection module is used to collect network popularity information data according to the set collection strategy and collection template.
3. The public opinion heat prediction system based on the Transformer+LSTM hybrid architecture according to claim 1, characterized in that: The extracted dimensional features are four dimensions: number of comments, percentage of negative sentiment, user activity, and number of reposts.
4. The public opinion heat prediction system based on the Transformer+LSTM hybrid architecture according to claim 1, characterized in that: The feature processing device includes a feature processing module, a calculation processing module, and a sequence relationship module. The output of the feature processing module is connected to the input of the calculation processing module, and the input of the calculation processing module is connected to both the sequence relationship module and the input of the public opinion prediction device. The feature processing module uses MinMaxScaler for feature standardization and constructs a time window sequence. It maps the original features to the model's hidden expansion dimension and then normalizes them. The calculation and processing module processes the standardized and normalized dimensional features through the Transformer module and inputs the processing results into the input terminal of the public opinion prediction device. The sequence relation module captures sequence dependencies during the processing by the encoder in the Transformer module, and inputs the processing results into the LSTM module to obtain long-term sequence dependencies.
5. The public opinion heat prediction system based on the Transformer+LSTM hybrid architecture according to claim 4, characterized in that: The Transformer module includes an encoder and a decoder. Six encoders are superimposed on a single encoder and the Transformer architecture is used for data processing. Six decoders are superimposed on a single decoder and the Transformer architecture is used for data processing.
6. The public opinion heat prediction system based on the Transformer+LSTM hybrid architecture according to claim 1, characterized in that: It also includes a preprocessing device, the input and output of which are respectively connected to the output of the data acquisition device and the input of the feature extraction device; The preprocessing device can divide the collected data according to certain rules and transmit the divided data to the feature extraction device.