Lightweight integrated model-based public opinion popularity prediction method

By deploying lightweight integrated models on the server side and browser side, the resource constraints and data security problems in public opinion heat prediction are solved, efficient and real-time public opinion heat prediction is achieved, and user experience and data protection are improved.

CN119047645AActive Publication Date: 2024-11-29SICHUAN JIUZHOU SOFTWARE CO LTD

Patent Information

Application Number
CN202411199383.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-29
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

In the prediction of public opinion popularity in the existing technology, deep learning models require a large amount of training data and computing resources, which are difficult to implement in resource-constrained environments. The traditional deployment methods are easily affected by network latency and data privacy security issues, and cannot meet high real-time and data protection requirements.

Method used

By deploying lightweight integrated models on the server side and the browser side, using the computing power on the server side for model training, and deploying lightweight models on the browser side, localized inference of the model is realized, network delays and data transmission risks are avoided, and the risk of network delays and data transmission is ensured. Prediction accuracy and real-time.

Benefits of technology

It significantly improves the response speed of the prediction system, reduces data transmission risks, enhances the privacy and security protection of user data, meets the needs of high real-time application, and realizes the unified application and adaptability of the model in a multi-end environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119047645A_ABST
    Figure CN119047645A_ABST
Patent Text Reader

Abstract

The invention discloses a public opinion popularity prediction method based on a lightweight integrated model. The method comprises the following steps: constructing a public opinion popularity prediction model based on a recurrent neural network; respectively deploying the public opinion popularity prediction model to a server side and a browser side through a deep learning framework to obtain a complete prediction model and a lightweight integration model; training of the complete prediction model is completed through the training set, and model parameters of the trained complete prediction model are loaded to the lightweight integrated model; and obtaining public opinion data to be predicted, and inputting the data into the complete prediction model or the lightweight integration model to obtain a public opinion popularity prediction result. According to the method, the lightweight public opinion popularity prediction model is deployed at the browser side, so that localized reasoning of the model is realized, the problem of network delay caused by real-time prediction depending on the server side is avoided, the response speed of the prediction system is remarkably improved, the requirements of high-real-time application scenes are met, and the method and the device are suitable for popularization and application. And the demand of transmitting user data to the server side is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of big data processing technology, and in particular to a public opinion heat prediction method based on a lightweight integrated model. Background Art

[0002] In the era of information explosion, with the rapid development of social media, news media and other digital platforms, the public's attention to various events, topics or entities is increasing. Public opinion heat has become a key indicator to measure these concerns, and its accurate prediction is of great significance to government agencies, enterprises, news media and social media platforms. Accurate public opinion heat prediction can not only help relevant institutions to understand social hot spots in a timely manner, but also provide strong support for policy formulation, brand management and market promotion.

[0003] Traditional methods for predicting the heat of public opinion mainly rely on statistical analysis or shallow machine learning models. These methods usually perform poorly when dealing with high-dimensional data or complex nonlinear relationships, resulting in insufficient prediction accuracy and difficulty in meeting the needs of practical applications. In recent years, with the rapid development of deep learning technology, its powerful feature extraction capabilities and model generalization capabilities have been widely used in public opinion heat prediction, especially in the field of natural language processing (NLP), where models based on the Transformer framework have shown excellent performance. However, these deep learning models usually require a large amount of training data and computing resources, which are difficult to implement in many practical application scenarios, especially in resource-constrained environments.

[0004] In order to avoid the problem of network delay and improve the real-time response capability, the public opinion heat prediction model often needs to be deployed in the client's browser to realize the localized reasoning of the model. However, due to the limited computing power of the browser, it is obviously unable to support the training of complex models. Therefore, the existing technology usually chooses to deploy the prediction model on the server side and relies on the communication between the browser and the server to obtain the prediction results. On the one hand, this traditional deployment method is susceptible to network delays or network instability, resulting in reduced response speed, unable to meet the requirements of high real-time applications, and may reduce the user experience; on the other hand, transmitting data to the server for processing may cause data privacy and security issues, especially when processing sensitive information, user data is at risk of being intercepted or leaked during transmission. Summary of the invention

[0005] In order to solve the deficiencies in the above-mentioned prior art, the present invention aims to provide a method for predicting the popularity of public opinion based on a lightweight integrated model. By deploying the model on the server side and the browser side respectively, combining the advantages of both, the powerful computing power of the server side is used to train the model efficiently, and the lightweight model is deployed on the browser side to achieve flexible real-time prediction capabilities. The model parameters trained on the server side can be directly loaded to the browser side to ensure the uniformity and consistency of the model in a multi-terminal environment, and meet the multiple requirements for prediction accuracy and real-time performance in different application scenarios.

[0006] In order to achieve the above-mentioned invention object, the technical solution provided by the present invention includes:

[0007] The method for predicting the heat of public opinion based on a lightweight integrated model includes the following steps:

[0008] A public opinion heat prediction model is constructed based on a recurrent neural network, wherein the public opinion heat prediction model includes a plurality of parallel basic networks and a multi-head attention layer that converts the output of the basic network into a two-dimensional matrix form as input;

[0009] The public opinion heat prediction model is deployed to the server and browser respectively through a deep learning framework to obtain a complete prediction model and a lightweight integrated model;

[0010] The training set is used to complete the training of the complete prediction model, and the model parameters of the trained complete prediction model are loaded into the lightweight integrated model;

[0011] Obtain the public opinion data to be predicted, input the complete prediction model or lightweight integrated model, and obtain the public opinion heat prediction results.

[0012] Preferably, the basic network includes several groups of basic models whose structural complexity is proportional to the length of the time window during data extraction, and the basic models with the same time window length have the same structure.

[0013] Preferably, the method of converting the output of the basic network into a two-dimensional matrix form comprises:

[0014] The basic network includes N groups of basic models for N time window lengths, each group includes M basic models; the output of the basic network is constructed as an M x N two-dimensional matrix.

[0015] Preferably, the method of inputting the two-dimensional matrix into the multi-head attention layer includes: performing a linear transformation on the two-dimensional matrix, converting it into a query matrix, a key matrix and a value matrix respectively, and inputting them into the multi-head attention layer.

[0016] Preferably, the training set and the public opinion data to be predicted need to be preprocessed after being obtained, and the preprocessing method includes:

[0017] Store the data to be processed in chronological order;

[0018] Divide the data to be processed by topic popularity to obtain x sets of first data sets about x topics; the characteristic data of the first data set includes: the number of reposts, comments, likes, readings, entities associated with the topic, collections, and media within a unit time;

[0019] The data in the first data set are divided into a number of time windows respectively to obtain y pieces of first data; the label of the data in each time window is the label of the next time step of the last time step of the current time window.

[0020] Preferably, the deployment method of the entire prediction model and the lightweight integration model includes:

[0021] The public opinion heat prediction model is deployed to the server side through the Tensorflow framework to obtain a complete prediction model; the public opinion heat prediction model is deployed to the browser side through the Tensorflow.js framework to obtain a lightweight integrated model; the complete prediction model and the lightweight integrated model have the same structure.

[0022] Beneficial Effects

[0023] 1. The present invention realizes localized reasoning of the model by deploying a lightweight public opinion heat prediction model on the browser side, avoids the network delay problem caused by relying on the server side for real-time prediction, significantly improves the response speed of the prediction system, meets the needs of high real-time application scenarios, reduces the need to transmit user data to the server side, thereby reducing the risk of data interception or leakage during transmission, enhances the privacy and security protection of user data, and avoids the response delay problem caused by network instability, ensuring that users can obtain timely feedback during use, thereby significantly improving the user experience.

[0024] 2. The present invention has good portability. By deploying models of the same structure on the server side and the browser side respectively, the unified application of the model on multiple terminals is realized, so that the system can run efficiently in different scenarios, enhancing the adaptability and flexibility of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a method for predicting the heat of public opinion based on a lightweight integrated model in a preferred embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the relationship between various models and the signal transmission direction in a method for predicting the heat of public opinion based on a lightweight integrated model in a preferred embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the structure of a public opinion heat prediction model in another preferred embodiment of the present invention; DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described below in conjunction with the accompanying drawings. In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention.

[0029] Example

[0030] like Figure 1-Figure 2 As shown, this embodiment provides a method for predicting the heat of public opinion based on a lightweight integrated model, comprising the steps of:

[0031] S1. Construct a public opinion heat prediction model based on a recurrent neural network, wherein the public opinion heat prediction model includes several parallel basic networks and a multi-head attention layer that converts the output of the basic network into a two-dimensional matrix form as input.

[0032] The recurrent neural network (RNN) is an artificial neural network structure used to process sequence data. This series of networks includes network structures such as basic RNN (Recurrent Neural Network), long short-term memory network (LSTM) and gated recurrent unit (GRU). Since this type of network has great advantages in processing time series, the present invention uses it as the basis, such as Figure 3 As shown, the present invention does not further limit the specific type to be selected, but in order to achieve the purpose of the present invention, the public opinion heat prediction model needs to include several parallel basic networks and a multi-head attention layer that converts the output of the basic network into a two-dimensional matrix form as input.

[0033] In this embodiment, in order to effectively extract data features when processing time windows of different lengths, the design of the basic network needs to flexibly adapt to changes in the length of the time window. Therefore, in some preferred embodiments, the basic network includes several groups of basic models whose structural complexity is proportional to the length of the time window when extracting data, that is, the longer the time window, the more complex the structure of the basic model. The basic models with the same time window length have the same structure. Specifically, the structural complexity of the basic model is mainly reflected in hyperparameters such as the number of network layers and the number of hidden nodes. For the basic models processing the same time window length, the structural design is unified to ensure that the model can stably and efficiently extract data features under a specific time window length. At the same time, since the data features corresponding to different time window lengths have different complexities and information amounts, the basic model needs to be adjusted for these differences to ensure the prediction accuracy in the entire data processing process.

[0034] A key advantage of this design is that it can adapt to different data processing requirements by adjusting the complexity of the basic model, thereby avoiding unnecessary computational overhead while maintaining high prediction accuracy. This not only improves the flexibility and adaptability of the model, but also makes it more suitable for application in resource-constrained environments. In addition, this basic network composed of multiple groups of basic models also has good scalability. When faced with new data features or changing time window requirements, optimization can be achieved by adjusting or increasing the number or complexity of the basic model without making major modifications to the entire network structure.

[0035] When processing data of different lengths of time windows, the output of the basic network may have different shapes. In order to unify these outputs and facilitate input into the multi-head attention layer, it is considered to convert the output of the basic network into a two-dimensional matrix form. In some preferred embodiments, considering that the data features are extracted through time windows of different lengths, a preferred two-dimensional matrix conversion method is provided, which specifically includes: the basic network includes N groups of basic models for N time window lengths, and each group of basic models specifically processes a time window of a specific length to capture the data features under the time window length. Each group of basic models includes M basic models, which have the same network structure; the output of the basic network is constructed as an M x N two-dimensional matrix. In this matrix, each row (M) represents the output of a group of basic models, and each column (N) corresponds to the feature extraction results of different time window lengths. Specifically, each row of the matrix contains the output of the same group of basic models under different time windows; and each column of the matrix contains the output of different groups of basic models for the same time window length. Through this arrangement, it can be ensured that the feature information contained in the matrix is ​​comprehensive and representative.

[0036] Furthermore, in order to match the characteristics of the multi-head attention layer, in some preferred embodiments, a method for inputting the two-dimensional matrix into the multi-head attention layer is also provided, specifically including: performing a linear transformation on the two-dimensional matrix, converting it into a query matrix, a key matrix and a value matrix respectively, and inputting the multi-head attention layer. Among them, the linear transformation can be achieved by matrix multiplication, that is, applying different weight matrices to the two-dimensional matrix to generate a new feature representation. The purpose of this process is to map the input two-dimensional matrix to a higher dimensional space in order to better capture the important features in the data.

[0037] The query matrix (Query Matrix, Q) is used to determine the target features that need to be paid attention to in the attention mechanism. The key matrix (Key Matrix, K) contains the feature representation of each input element, which is used to match the query matrix to calculate the relevance. The value matrix (Value Matrix, V) stores the feature information that needs to be transmitted, and the final output is obtained by weighting it with the attention weight. The above matrix is ​​input into the multi-head attention layer, and the importance of each element is calculated through dot product operation and weighted summation, and the final attention weight is generated.

[0038] Those skilled in the art should know that after the multi-head attention layer processing, the model usually needs to be further processed by the superposition and normalization components and the feedforward network layer to obtain the final feature results. In this process, the output of the multi-head attention layer will be superimposed with the original input through the residual connection (skip connection), and then normalized (such as Layer Norm) to ensure the stability of the model and the efficiency of training. After these processes, the key information in the feature results is further strengthened, while the noise or unimportant features are effectively suppressed, so that the model can focus more on the most valuable part of the data. Then, the feature results will pass through the feedforward network layer (usually including one or more fully connected layers) to further extract and combine high-order features and enhance the prediction ability of the model. Finally, the processed feature results will introduce nonlinear factors through activation functions (such as Relu, Swish, etc.) to enable the model to capture more complex feature relationships. The choice of activation function can be adjusted according to specific application requirements to optimize the performance of the model. After these processing steps, the model finally generates a prediction result. Those skilled in the art can design the specific structure and parameter configuration of the output part by themselves based on existing technical knowledge and specific application scenarios. The present invention does not further limit the design of the output part, and allows flexible adjustment to adapt to different needs and conditions.

[0039] S2. Deploy the public opinion heat prediction model to the server and browser respectively through a deep learning framework to obtain a complete prediction model and a lightweight integrated model.

[0040] Deep learning framework refers to a software tool or library used to design, train and deploy deep learning models. These frameworks provide a series of predefined functions, modules and APIs, allowing developers to efficiently build complex neural network models without having to implement the underlying mathematical operations or optimization algorithms from scratch. In the present invention, the deployment of the public opinion heat prediction model mainly considers the cross-platform deployment capability, that is, deploying the same model architecture in different computing environments. On the server side, its powerful computing power is used to process large-scale data and perform complex training tasks; on the browser side, it is necessary to run a lightweight model to achieve real-time reasoning and localized calculations, avoiding network delays. In some preferred embodiments, consider deploying the public opinion heat prediction model to the server side through the Tensorflow framework to obtain a complete prediction model; deploying the public opinion heat prediction model to the browser side through the Tensorflow.js framework to obtain a lightweight integrated model; the complete prediction model and the lightweight integrated model have the same structure. Among them, TensorFlow and TensorFlow.js are deep learning frameworks developed and maintained by Google, and the two have good compatibility. TensorFlow can perform efficient model training on the server side, while TensorFlow.js allows model reasoning on the browser side. Specifically, TensorFlow.js provides an API optimized specifically for the JavaScript environment, which can directly load models trained by TensorFlow and perform reasoning operations in the browser, which is very important for applications that require real-time response on the client side. This seamless cross-platform support enables the model to be deployed and run in a variety of environments, meeting the needs of the present invention for multi-terminal applications. It should be understood that the present invention can also use other deep learning frameworks such as PyTorch and MXNet to implement this step, and its specific framework selection and implementation method can be reasonably designed by those skilled in the art based on its user manual and the purpose of the present invention.

[0041] S3. Use the training set to complete the training of the complete prediction model, and load the model parameters of the trained complete prediction model into the lightweight integrated model.

[0042] The training of various models in the present invention can be carried out according to the existing technology in the field, and the present invention has no intention to further limit the training process. Those skilled in the art will know that the key prerequisite for model training is the preparation of the training set. In some preferred embodiments, in order to better collect training data that meets the characteristics of public opinion prediction and ensure effective preliminary screening of the data to be predicted in practical applications, the following example of a training set processing method is proposed. It should be pointed out that the data processing method proposed below is not only applicable to the preparation of training sets, but also to the preparation of data to be predicted. Therefore, the subsequent description will no longer distinguish between the processing process of the training set and the data to be predicted.

[0043] S31. Select at least one social platform as a public opinion collection platform to collect public opinion-related data, including mainstream social platforms such as Weibo, WeChat, Twitter, Facebook, etc. The collected data includes but is not limited to: the number of forwarding, comment, like, reading, number of entities associated with the topic, number of collections, number of media, topic popularity ranking, etc. of the article or topic per unit time. Preferably, the data is collected according to historical hot topics.

[0044] S32. Store the data to be processed in chronological order.

[0045] S33. Using the topic popularity ranking as label data, the data to be processed is divided according to the topic popularity to obtain x groups of first data sets about x topics, that is, each group of first data sets includes data of one topic.

[0046] S34. Divide the data in the first data set into several time windows respectively to obtain y first data; wherein the lengths of the time windows may be different, and the label of the data in each time window is the label of the next time step of the last time step of the current time window. The time window refers to a portion of data intercepted from a time series. When processing a time series, the data is divided into multiple time windows, each time window containing a set of continuous time steps. The label is the target of model prediction, which is the value predicted at a certain time point in the future.

[0047] The last time step of the current time window means: Assuming that a time window contains data from time point t1 to tn, then the last time step of the current time window is tn. The label of the next time step: Assuming that the time series from t1 to tn+1 is continuous, then the next time step is the label corresponding to tn+1, that is, the value of tn+1.

[0048] The meaning of this step is: for all feature data in a time window (from t1 to tn), the corresponding label (or target value) is the value at the next time point tn+1 immediately after the end of the time window. The model will learn to predict this future value (label) from the data (feature data) in the time window.

[0049] Furthermore, the model is not trained on the lightweight integrated model on the browser side, but relies on the complete prediction model on the server side to complete the training. After the training is completed, the model parameters of the complete prediction model are retained and loaded into the lightweight integrated model. Since the complete prediction model and the lightweight integrated model have the same structure, this consistency enables the model parameters to be seamlessly transferred from the server side to the browser side. Specifically, parameter loading can be achieved through API calls and other methods.

[0050] S4. Obtain the public opinion data to be predicted, input the complete prediction model or lightweight integrated model, and obtain the public opinion heat prediction result.

[0051] It should be understood that the complete prediction model is deployed on the server side. When the prediction task requires high accuracy, or needs to process large-scale data, or needs to update and train the public opinion heat prediction model, it can be considered to input the public opinion data to be predicted into the complete prediction model. Utilizing the powerful computing power and abundant resources of the server, the complete prediction model can handle more complex computing tasks, thereby providing more accurate prediction results. In addition, model updates and training on the server side can also be carried out efficiently in this environment to ensure that the model is always in the optimal state.

[0052] The lightweight integrated model is deployed on the browser side and is suitable for scenarios that require instant feedback or real-time prediction. When the lightweight integrated model is input, since it runs locally on the client, it can significantly improve the response speed and provide fast prediction results. This localization process also effectively avoids network delays and data leakage problems that may occur during data transmission, thereby enhancing the privacy and security of the data.

[0053] Therefore, users can flexibly choose to use a complete prediction model or a lightweight integrated model according to their actual needs. For tasks that require high precision and complex calculations, a complete prediction model can be selected; while in scenarios that emphasize real-time and data privacy, a lightweight integrated model is a more ideal choice. This flexibility of choice ensures the applicability and practicality of the present invention in different application scenarios.

[0054] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for predicting the heat of public opinion based on a lightweight integrated model, characterized in that: Includes steps: A public opinion heat prediction model is constructed based on a recurrent neural network, wherein the public opinion heat prediction model includes a plurality of parallel basic networks and a multi-head attention layer that converts the output of the basic network into a two-dimensional matrix form as input; The public opinion heat prediction model is deployed to the server and browser respectively through a deep learning framework to obtain a complete prediction model and a lightweight integrated model; The training set is used to complete the training of the complete prediction model, and the model parameters of the trained complete prediction model are loaded into the lightweight integrated model; Obtain the public opinion data to be predicted, input the complete prediction model or lightweight integrated model, and obtain the public opinion heat prediction results.

2. The method for predicting the heat of public opinion based on a lightweight integrated model according to claim 1, characterized in that: The basic network includes several groups of basic models whose structural complexity is proportional to the time window length during data extraction, and the basic models with the same time window length have the same structure.

3. The method for predicting the heat of public opinion based on a lightweight integrated model according to claim 1, characterized in that: The method for converting the output of the basic network into a two-dimensional matrix form includes: the basic network includes N groups of basic models for N time window lengths, each group includes M basic models; the output of the basic network is constructed as an M x N two-dimensional matrix.

4. The method for predicting the heat of public opinion based on a lightweight integrated model according to claim 1, characterized in that: The method of inputting the two-dimensional matrix into the multi-head attention layer includes: performing a linear transformation on the two-dimensional matrix, converting it into a query matrix, a key matrix and a value matrix respectively, and inputting them into the multi-head attention layer.

5. The method for predicting the heat of public opinion based on a lightweight integrated model according to claim 1, characterized in that: The training set and the public opinion data to be predicted need to be preprocessed after being obtained, and the preprocessing method includes: Store the data to be processed in chronological order; Divide the data to be processed by topic popularity to obtain x sets of first data sets about x topics; the characteristic data of the first data set includes: the number of reposts, comments, likes, readings, entities associated with the topic, collections, and media within a unit time; The data in the first data set are divided into a number of time windows respectively to obtain y pieces of first data; the label of the data in each time window is the label of the next time step of the last time step of the current time window.

6. The method for predicting the heat of public opinion based on a lightweight integrated model according to claim 1, characterized in that: The deployment method of the entire prediction model and the lightweight integration model includes: The public opinion heat prediction model is deployed to the server side through the Tensorflow framework to obtain a complete prediction model; the public opinion heat prediction model is deployed to the browser side through the Tensorflow.js framework to obtain a lightweight integrated model; the complete prediction model and the lightweight integrated model have the same structure.

Citation Information

Patent Citations

  • Time sequence prediction method based on news public opinions

    CN110222149A

  • Network public opinion prediction method based on microblog data

    CN113779382A

  • Short-term load prediction method based on hybrid neural network model

    CN115238999A

  • Public opinion sentiment analysis method based on time sequence-popularity double-attention mechanism fused with bidirectional long-short term memory network

    CN116562298A

  • Catalytic cracking unit key index modeling method based on time sequence feature extraction

    WO2024021536A1

Cited By

  • Consistency data set generation method and device, equipment and storage medium

    CN119939676A

  • Public opinion popularity early anomaly detection method and system based on attention auto-encoder

    CN120162549A