Information data analysis report arrangement and generation method based on large model

Abstract

The invention discloses an information data analysis report arrangement and generation method based on a large model, and relates to the technical information retrieval field, and the method comprises the following steps: building a network news comprehensive sample database; performing semantic recognition and screening on the network news comprehensive samples to obtain topic information parameters of the network news comprehensive samples; performing scene vector analysis on the topic information parameters to obtain a topic scene vector of the network news comprehensive sample; determining hot search topic information parameters, analyzing a keyword change trend in the hot search topic information parameters, and evaluating a public opinion trend vector of a hot search topic scene; evaluating a preference index between the network news comprehensive sample and the hot search topic; generating a network news comprehensive report corresponding to the preference according to the preference index between the network news comprehensive sample and the hot search topic; the method has the advantages that the news editing manuscript output efficiency is improved, and the news content quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of information retrieval technology, and in particular to a method for collating and generating information data analysis reports based on a large model. Background Art

[0002] The information data analysis report compilation of the big model refers to the systematic sorting, analysis and summary of the massive information data processed by the big model, extracting key data points, trends, correlations and insights, and forming a clearly structured and easy-to-understand report so that decision makers can quickly grasp the information behind the data and provide a basis for planning and business decisions.

[0003] Based on the rapid development of the Internet and the growth of domestic mobile users, hot news information is no longer limited to journalists' field investigations, but appears randomly as individual users upload their records to the Internet. However, hot news events usually have a trend shift, that is, content similar to hot news is more likely to attract the attention of individual users. Therefore, a decision-making solution to assist journalists is proposed. Summary of the invention

[0004] In order to solve the above technical problems, a method for compiling and generating information data analysis reports based on a large model is provided. This technical solution solves the above-mentioned rapid development of the Internet and the growth of domestic mobile users. Hot news information is no longer limited to journalists' field investigations, but appears randomly as personal users upload their records to the Internet. However, hot news events usually have a trend transfer, that is, content similar to hot news is more likely to receive the attention of individual users. Therefore, a set of decision-making solutions to assist journalists is proposed.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: A method for collating and generating information data analysis reports based on a large model, comprising: Based on the background API interface of the Internet and social media, obtain comprehensive sample data of real-time news and build a comprehensive sample database of online news; Based on the online news comprehensive sample database, semantic recognition and screening of online news comprehensive samples are performed to obtain topic information parameters of online news comprehensive samples; According to the topic information parameters of the network news comprehensive sample, scene vector analysis is performed on the topic information parameters to obtain the theme scene vector of the network news comprehensive sample; Obtain comprehensive sample data of historical hot search news, determine hot search topic information parameters, analyze keyword change trends in hot search topic information parameters, and evaluate the public opinion trend vector of hot search topic scenarios; Fitting analysis is performed on the theme scene vectors of the comprehensive sample of online news and the opinion trend vectors of the hot search theme scenes to evaluate the preference index between the comprehensive sample of online news and the hot search topics; According to the preference index between the comprehensive sample of online news and the hot search topics, a comprehensive report of online news corresponding to the preference is generated; ; In the formula, G k is the preference index between the comprehensive sample of online news and the kth hot search topic, P(z k ∣d i ) is the kth topic scene vector of the i-th network news comprehensive sample, Y k (t) is the public opinion trend vector of the kth hot search theme scene in t unit time, and L is the total number of theme scene vectors; Preferably, based on the online news comprehensive sample database, semantic recognition and screening of the online news comprehensive sample is performed to obtain topic information parameters of the online news comprehensive sample, specifically including: Based on the online news comprehensive sample database, semantic recognition and screening of online news comprehensive samples are performed to obtain topic information parameters of online news comprehensive samples; Preprocess the noise data in the comprehensive sample of online news, and split it into image data and text data to obtain news image data and news text data; Substitute the news image data into the CNN convolutional neural network model, use the convolution layer to perform convolution and pooling operations on the news impact data, obtain the news image feature map, and input it into the feature extraction layer for global average. Substitute the output of the residual block of the pooling layer into the full connection layer and the softmax layer to generate the feature map of the feature extraction layer, and flatten the feature map into a one-dimensional array and perform normalization processing to obtain the news image feature vector; Substitute the news text data into the BERT model, use the input embedding layer to map the word vector, paragraph vector and position vector of the news text data, obtain the news text embedding vector, and input it into the Transformer encoder for multi-head self-attention mechanism and feedforward neural network training, and output the news text feature vector; Based on the news image feature vector and the news text feature vector, feature concatenation is performed in NumPy to obtain the topic information parameters of the comprehensive sample of online news.

[0006] Preferably, according to the topic information parameters of the network news comprehensive sample, scene vector analysis is performed on the topic information parameters to obtain the subject scene vector of the network news comprehensive sample, specifically including: Based on the topic information parameters of the comprehensive sample of online news, a topic information parameter matrix of the comprehensive sample of online news is constructed; Based on each element in the topic information parameter matrix of the comprehensive sample of online news, a posterior probability analysis of the topic is performed to obtain the topic scenario corresponding to the comprehensive sample of online news; Verify the likelihood function value of each element in the topic information parameter matrix for the subject scenario, and evaluate the classification probability of the subject scenario of the comprehensive sample of online news; The classification probability of the theme scene of the comprehensive sample of online news is iterated several times until the likelihood function value reaches convergence, and the theme scene vector of the comprehensive sample of online news is obtained; The subject scene vector of the comprehensive online news sample is specifically: ; In the formula, P(z k ∣d i ) is the kth topic scene vector of the i-th network news comprehensive sample, P(w j ∣z k ) is the probability distribution of the jth topic information parameter in the kth topic scenario, P(z k ∣w ij ,d i ) is the probability distribution of the kth topic scenario given the jth topic information parameter of the ith online news comprehensive sample, δ(w,w ij ) is the indicator function, when w is equal to w ij When , its value is 1, otherwise it is 0, n is the total number of comprehensive samples of online news, and m is the total number of topic information.

[0007] Preferably, obtaining comprehensive sample data of historical hot search news, determining hot search topic information parameters, analyzing keyword change trends in the hot search topic information parameters, and evaluating the public opinion trend vector of the hot search topic scene specifically include: Based on the comprehensive sample data of historical hot search news, the keywords in the hot search topic information parameters are marked according to the unit time, and the topic information time series parameters of historical hot search news are constructed; According to the time series parameters of the subject information of historical hot search news, the TF-IDF value of each keyword is calculated to determine the subject information word heat distribution value of each historical hot search news; Based on text sentiment analysis, the sentiment tendency of the topic information word heat distribution value of each historical hot search news is evaluated, and the sentiment tendency distribution value of the topic information word heat distribution value of each historical hot search news is obtained; Based on ARIMA time series autoregression, a public opinion trend analysis model for historical hot search news is constructed; The topic information vocabulary heat distribution value of each historical hot search news and the topic information vocabulary sentiment tendency distribution value of each historical hot search news are substituted into the public opinion trend analysis model of historical hot search news, and the objective function is constructed based on the difference between the predicted value and the actual value. The prediction error function of the training result is used as the end target to generate the public opinion trend vector of the hot search topic scene.

[0008] The public opinion trend analysis model of the historical hot search news is specifically: ; Where Y k (t) is the opinion trend vector of the kth hot search topic scene in t unit time, is the predicted value of the zth word heat distribution of the topic information of the hot search news in the historical t unit time, is the predicted value of the sentiment tendency distribution of the zth word in the topic information of the hot search news in the historical t unit time, is the actual value of the zth word heat distribution of the hot search news topic information in the historical t unit time, is the actual value of the sentiment tendency distribution of the zth word in the topic information of the hot search news in the historical t unit time, u is the total number of words in the topic information of the hot search news, and T is the total duration.

[0009] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a large-model-based information data analysis report compilation and generation scheme, based on multi-source data analysis, using real-time data from online news and social media, combined with semantic recognition and vector analysis algorithms, to automatically identify and extract topic information, evaluate public opinion trends through historical hot search data, and perform matching and fitting analysis between topics and public opinion trends, thereby generating an accurate online news comprehensive report. The beneficial effects are: improving the efficiency of news editors in producing manuscripts and improving the quality of news content. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flow chart of a method for collating and generating information data analysis reports based on a large model; Figure 2 A flow chart of the method for obtaining topic information parameters of a comprehensive sample of online news; Figure 3 Flowchart of the method for obtaining the theme scene vector of the comprehensive sample of online news; Figure 4 Flowchart of the opinion trend vector method for evaluating hot search topic scenarios. DETAILED DESCRIPTION

[0011] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0012] Reference Figure 1 As shown, a method for collating and generating information data analysis reports based on a large model includes: Based on the background API interface of the Internet and social media, obtain comprehensive sample data of real-time news and build a comprehensive sample database of online news; Based on the online news comprehensive sample database, semantic recognition and screening of online news comprehensive samples are performed to obtain topic information parameters of online news comprehensive samples; According to the topic information parameters of the network news comprehensive sample, scene vector analysis is performed on the topic information parameters to obtain the theme scene vector of the network news comprehensive sample; Obtain comprehensive sample data of historical hot search news, determine hot search topic information parameters, analyze keyword change trends in hot search topic information parameters, and evaluate the public opinion trend vector of hot search topic scenarios; Fitting analysis is performed on the theme scene vectors of the comprehensive sample of online news and the opinion trend vectors of the hot search theme scenes to evaluate the preference index between the comprehensive sample of online news and the hot search topics; According to the preference index between the comprehensive sample of online news and the hot search topics, a comprehensive report of online news corresponding to the preference is generated; ; In the formula, G k is the preference index between the comprehensive sample of online news and the kth hot search topic, P(z k ∣d i ) is the kth topic scene vector of the i-th network news comprehensive sample, Y k (t) is the public opinion trend vector of the kth hot search theme scene in t unit time, and L is the total number of theme scene vectors.

[0013] This solution is based on multi-source data analysis, using real-time data from online news and social media, combined with semantic recognition and vector analysis algorithms, to automatically identify and extract topic information, evaluate public opinion trends through historical hot search data, and perform matching and fitting analysis between topics and public opinion trends, thereby generating an accurate online news comprehensive report. The beneficial effects are: improving the efficiency of news editors in publishing and improving the quality of news content.

[0014] Reference Figure 2 As shown, based on the online news comprehensive sample database, the semantic recognition and screening of the online news comprehensive sample is performed to obtain the topic information parameters of the online news comprehensive sample, which specifically include: Based on the online news comprehensive sample database, semantic recognition and screening of online news comprehensive samples are performed to obtain topic information parameters of online news comprehensive samples; Preprocess the noise data in the comprehensive sample of online news, and split it into image data and text data to obtain news image data and news text data; Substitute the news image data into the CNN convolutional neural network model, use the convolution layer to perform convolution and pooling operations on the news impact data, obtain the news image feature map, and input it into the feature extraction layer for global average. Substitute the output of the residual block of the pooling layer into the full connection layer and the softmax layer to generate the feature map of the feature extraction layer, and flatten the feature map into a one-dimensional array and perform normalization processing to obtain the news image feature vector; Substitute the news text data into the BERT model, use the input embedding layer to map the word vector, paragraph vector and position vector of the news text data, obtain the news text embedding vector, and input it into the Transformer encoder for multi-head self-attention mechanism and feedforward neural network training, and output the news text feature vector; Based on the news image feature vector and the news text feature vector, feature concatenation is performed in NumPy to obtain the topic information parameters of the comprehensive sample of online news.

[0015] It is understandable that since the online news comprehensive sample contains independent text data or image data, when processing the topic information vector of the online news comprehensive sample, it is necessary to create two independent private variables to encapsulate the image data and text data in the online news comprehensive sample to avoid mutual contamination between data.

[0016] Reference Figure 3 As shown, according to the topic information parameters of the network news comprehensive sample, the scene vector analysis is performed on the topic information parameters to obtain the theme scene vector of the network news comprehensive sample, which specifically includes: Based on the topic information parameters of the comprehensive sample of online news, a topic information parameter matrix of the comprehensive sample of online news is constructed; Based on each element in the topic information parameter matrix of the comprehensive sample of online news, a posterior probability analysis of the topic is performed to obtain the topic scenario corresponding to the comprehensive sample of online news; Verify the likelihood function value of each element in the topic information parameter matrix for the subject scenario, and evaluate the classification probability of the subject scenario of the comprehensive sample of online news; The classification probability of the theme scene of the comprehensive sample of online news is iterated several times until the likelihood function value reaches convergence, and the theme scene vector of the comprehensive sample of online news is obtained; The subject scene vector of the comprehensive online news sample is specifically: ; In the formula, P(z k ∣d i ) is the kth topic scene vector of the i-th network news comprehensive sample, P(w j ∣z k ) is the probability distribution of the jth topic information parameter in the kth topic scenario, P(z k ∣w ij ,d i ) is the probability distribution of the kth topic scenario given the jth topic information parameter of the ith online news comprehensive sample, δ(w,w ij ) is the indicator function, when w is equal to w ij When , its value is 1, otherwise it is 0, n is the total number of comprehensive samples of online news, and m is the total number of topic information.

[0017] This scheme uses the topic information of online news to construct a parameter matrix, determines the theme scene of the news through matrix analysis, and calculates the likelihood function value to verify the accuracy. After multiple iterations of optimization, the theme scene vector of the news is finally obtained, realizing accurate and efficient extraction of news theme information.

[0018] Reference Figure 4 As shown, obtaining comprehensive sample data of historical hot search news, determining hot search topic information parameters, analyzing keyword change trends in hot search topic information parameters, and evaluating the public opinion trend vector of the hot search topic scene specifically include: Based on the comprehensive sample data of historical hot search news, the keywords in the hot search topic information parameters are marked according to the unit time, and the topic information time series parameters of historical hot search news are constructed; According to the time series parameters of the subject information of historical hot search news, the TF-IDF value of each keyword is calculated to determine the subject information word heat distribution value of each historical hot search news; Based on text sentiment analysis, the sentiment tendency of the topic information word heat distribution value of each historical hot search news is evaluated, and the sentiment tendency distribution value of the topic information word heat distribution value of each historical hot search news is obtained; Based on ARIMA time series autoregression, a public opinion trend analysis model for historical hot search news is constructed; Substitute the topic information word heat distribution value of each historical hot search news and the topic information word sentiment tendency distribution value of each historical hot search news into the public opinion trend analysis model of historical hot search news, construct the objective function based on the difference between the predicted value and the actual value, and use the prediction error function of the training result as the end target to generate the public opinion trend vector of the hot search topic scene; The public opinion trend analysis model of the historical hot search news is specifically: ; Where Y k (t) is the opinion trend vector of the kth hot search topic scene in t unit time, is the predicted value of the zth word heat distribution of the topic information of the hot search news in the historical t unit time, is the predicted value of the sentiment tendency distribution of the zth word in the topic information of the hot search news in the historical t unit time, is the actual value of the zth word heat distribution of the hot search news topic information in the historical t unit time, is the actual value of the sentiment tendency distribution of the zth word in the topic information of the hot search news in the historical t unit time, u is the total number of words in the topic information of the hot search news, and T is the total duration.

[0019] This solution collects comprehensive sample data of historical hot search news, performs time series tagging on the keywords in the hot search topic information parameters, calculates the TF-IDF value of each keyword to measure its heat distribution, and further combines the text sentiment tendency analysis to obtain the sentiment tendency distribution of each keyword. Subsequently, the ARIMA time series autoregressive model is used to build a public opinion trend analysis model, and the keyword heat distribution and sentiment tendency distribution values ​​are substituted into the model for training, with the goal of minimizing the prediction error, and finally generates the public opinion trend vector of the hot search topic scene to provide decision support for journalists.

[0020] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for arranging and generating information data analysis reports based on a large model, characterized in that: include: Based on the background API interface of the Internet and social media, obtain comprehensive sample data of real-time news and build a comprehensive sample database of online news; Based on the online news comprehensive sample database, semantic recognition and screening of online news comprehensive samples are performed to obtain topic information parameters of online news comprehensive samples; According to the topic information parameters of the network news comprehensive sample, scene vector analysis is performed on the topic information parameters to obtain the theme scene vector of the network news comprehensive sample; Obtain comprehensive sample data of historical hot search news, determine hot search topic information parameters, analyze keyword change trends in hot search topic information parameters, and evaluate the public opinion trend vector of hot search topic scenarios; Fitting analysis is performed on the theme scene vectors of the comprehensive sample of online news and the opinion trend vectors of the hot search theme scenes to evaluate the preference index between the comprehensive sample of online news and the hot search topics; According to the preference index between the comprehensive sample of online news and the hot search topics, a comprehensive report of online news corresponding to the preference is generated; ； In the formula, G k is the preference index between the comprehensive sample of online news and the kth hot search topic, P(z k ∣d i ) is the kth topic scene vector of the i-th network news comprehensive sample, Y k (t) is the public opinion trend vector of the kth hot search theme scene in t unit time, and L is the total number of theme scene vectors.

2. The method for arranging and generating information data analysis reports based on a large model according to claim 1, characterized in that: Based on the online news comprehensive sample database, the semantic recognition and screening of online news comprehensive samples are carried out to obtain the topic information parameters of the online news comprehensive samples, including: Based on the online news comprehensive sample database, semantic recognition and screening of online news comprehensive samples are performed to obtain topic information parameters of online news comprehensive samples; Preprocess the noise data in the comprehensive sample of online news, and split it into image data and text data to obtain news image data and news text data; Substitute the news image data into the CNN convolutional neural network model, use the convolution layer to perform convolution and pooling operations on the news impact data to obtain the news image feature map, and input it into the feature extraction layer for global average. Substitute the output of the residual block of the pooling layer into the full connection layer and the softmax layer to generate the feature map of the feature extraction layer, and flatten the feature map into a one-dimensional array and perform normalization to obtain the news image feature vector; Substitute the news text data into the BERT model, use the input embedding layer to map the word vector, paragraph vector and position vector of the news text data, obtain the news text embedding vector, and input it into the Transformer encoder for multi-head self-attention mechanism and feedforward neural network training, and output the news text feature vector; Based on the news image feature vector and the news text feature vector, feature concatenation is performed in NumPy to obtain the topic information parameters of the comprehensive sample of online news.

3. The method for arranging and generating information data analysis reports based on a large model according to claim 2 is characterized in that: According to the topic information parameters of the network news comprehensive sample, the scene vector analysis is performed on the topic information parameters to obtain the theme scene vector of the network news comprehensive sample, which specifically includes: Based on the topic information parameters of the comprehensive sample of online news, a topic information parameter matrix of the comprehensive sample of online news is constructed; Based on each element in the topic information parameter matrix of the comprehensive sample of online news, a posterior probability analysis of the topic is performed to obtain the topic scenario corresponding to the comprehensive sample of online news; Verify the likelihood function value of each element in the topic information parameter matrix for the subject scenario, and evaluate the classification probability of the subject scenario of the comprehensive sample of online news; The classification probability of the theme scene of the comprehensive sample of online news is iterated several times until the likelihood function value converges, and the theme scene vector of the comprehensive sample of online news is obtained.

4. The method for arranging and generating information data analysis reports based on a large model according to claim 3 is characterized in that: The subject scene vector of the comprehensive online news sample is specifically: ； In the formula, P(z k ∣d i ) is the kth topic scene vector of the i-th network news comprehensive sample, P(w j ∣z k ) is the probability distribution of the jth topic information parameter in the kth topic scenario, P(z k ∣w ij ,d i ) is the probability distribution of the kth topic scenario given the jth topic information parameter of the ith online news comprehensive sample, δ(w,w ij ) is the indicator function, when w is equal to w ij When , its value is 1, otherwise it is 0, n is the total number of comprehensive samples of online news, and m is the total number of topic information.

5. The method for arranging and generating information data analysis reports based on a large model according to claim 4 is characterized in that: Obtain comprehensive sample data of historical hot search news, determine hot search topic information parameters, analyze keyword change trends in hot search topic information parameters, and evaluate the public opinion trend vector of hot search topic scenarios. Specifically include: Based on the comprehensive sample data of historical hot search news, the keywords in the hot search topic information parameters are marked according to the unit time, and the topic information time series parameters of historical hot search news are constructed; According to the time series parameters of the subject information of historical hot search news, the TF-IDF value of each keyword is calculated to determine the subject information word heat distribution value of each historical hot search news; Based on text sentiment analysis, the sentiment tendency of the topic information word heat distribution value of each historical hot search news is evaluated, and the sentiment tendency distribution value of the topic information word heat distribution value of each historical hot search news is obtained; Based on ARIMA time series autoregression, a public opinion trend analysis model for historical hot search news is constructed; The topic information vocabulary heat distribution value of each historical hot search news and the topic information vocabulary sentiment tendency distribution value of each historical hot search news are substituted into the public opinion trend analysis model of historical hot search news, and the objective function is constructed based on the difference between the predicted value and the actual value. The prediction error function of the training result is used as the end target to generate the public opinion trend vector of the hot search topic scene.

6. The method for arranging and generating information data analysis reports based on a large model according to claim 5 is characterized in that: The public opinion trend analysis model of the historical hot search news is specifically as follows: ； Where Y k (t) is the opinion trend vector of the kth hot search topic scene in t unit time, is the predicted value of the zth word heat distribution of the topic information of the hot search news in the historical t unit time, is the predicted value of the sentiment tendency distribution of the zth word in the topic information of the hot search news in the historical t unit time, is the actual value of the zth word heat distribution of the hot search news topic information in the historical t unit time, is the actual value of the sentiment tendency distribution of the zth word in the topic information of the hot search news in the historical t unit time, u is the total number of words in the topic information of the hot search news, and T is the total duration.

Images