A brand influence dynamic monitoring method based on big data driving

Abstract

The present application relates to a kind of brand influence dynamic monitoring method based on big data driving, comprising the following steps: collecting the webpage data of various network media and various forums by network crawler and carrying out pretreatment to webpage data;Media public opinion sentiment classification model is built, and the public opinion information in the webpage data of various network media after pretreatment is classified by media public opinion sentiment classification model;Hot topic mining model is built, and hot topic in the webpage data of various forums after pretreatment is mined by hot topic mining model, meanwhile, hot topic tracking model is built, and hot topic is tracked by hot topic tracking model;Brand influence trend prediction model is built, and the influence trend of the brand is predicted by the public opinion information related to the brand corresponding to the model and relevant hot topic, which provides early warning capability for brand party.

Description

TECHNICAL FIELD

[0001] The application relates to a brand influence dynamic monitoring method based on big data driving and belongs to the technical field of information management. BACKGROUND

[0002] For brand influence evaluation methods and measurement techniques of different brand types, current research mainly has two directions, one is focused on the market and financial perspective, and brand influence key factors are extracted and relevant data is obtained for measurement and calculation, and the other direction is to set different brand influence indexes according to research results, and evaluation is performed through expert scoring, however, these research results are more from the perspective of market and consumers;

[0003] The prior art such as patent No. CN112598468A discloses a consumer product public opinion internet influence index evaluation method, which comprises the following steps: inputting target related keywords in a mapping function; performing mapping processing on the keywords to obtain a keyword set; classifying Sina Weibo, Xiaohongshu and the like into microblog platforms, and classifying Heima complaint platform and consumer quality network into network media platforms; obtaining microblog information; collecting microblog descriptive information; calculating microblog platform influence value; obtaining network media information; collecting network media descriptive information; obtaining website ranking and website relevance degree rating index; calculating network media platform influence value; and calculating network influence total value according to a formula. The application has the beneficial effects of comprehensive information collection, wide monitoring and analysis dimensions, and can truly reflect the internet public opinion influence of the brand name, product type name and product quality fault type name to be evaluated.

[0004] However, the above scheme only calculates the influence of specific platforms on brand influence, which cannot represent the overall evaluation of all media and users on the internet, lacks certain comprehensiveness, and results in inaccurate influence calculation results. SUMMARY

[0005] In order to solve the problems existing in the prior art, the application provides a brand influence dynamic monitoring method based on big data driving.

[0006] The technical scheme of the application is as follows:

[0007] A brand influence dynamic monitoring method based on big data driving comprises the following steps:

[0008] Collecting webpage data of various network media and various forums through a network crawler and preprocessing the webpage data;

[0009] Building a media public opinion sentiment classification model, and performing sentiment classification on public opinion information in the preprocessed webpage data of various network media through the media public opinion sentiment classification model;

[0010] The hot topic mining model is constructed, and the hot topic mining model is used to mine the hot topics in the preprocessed web page data of various forums, and a hot topic tracking model is constructed, and the hot topic tracking model is used to track the hot topics.

[0011] A brand influence trend prediction model is constructed, which predicts the influence trend of the brand by classifying the sentiment of the public opinion information related to the brand and predicting the hot topics.

[0012] As a preferred embodiment of the present application, the pre-processing steps of the web page data are as follows:

[0013] Filter the web page tags in the web page data to obtain text data, remove punctuation and spaces in the text data, then cut the sentences in the text data into word items by the jieba word segmentation algorithm, filter the word items without specific meaning based on a pre-set stop word table, and finally convert all the text data into word vector form.

[0014] As a preferred embodiment of the present application, the media public opinion sentiment classification model is constructed based on a TextCNN-GRU model, and the specific steps of sentiment classification are as follows:

[0015] Based on the preprocessed various media web page data, a training set and a test set are constructed;

[0016] The training set is input into the media public opinion sentiment classification model to train the model, and the performance of the media public opinion sentiment classification model is evaluated by the test set. If the performance evaluation result of the media public opinion sentiment classification model is less than the pre-set performance threshold, the parameters of the media public opinion sentiment classification model are adjusted and re-trained by the training set, and the performance is evaluated again. The operation is repeated until the performance of the media public opinion sentiment classification model is greater than the pre-set performance threshold, and the training is stopped. The trained media public opinion sentiment classification model is obtained;

[0017] The public opinion information in various media web page data is classified by the trained media public opinion sentiment classification model.

[0018] As a preferred embodiment of the present application, the hot topic mining model mines the hot topics by the following steps:

[0019] Each post in the preprocessed various forum web page data is stored in a document and constitutes a document set, and all word pairs in the document set are generated by combining all word pairs in the document set;

[0020] The BTM topic model is obtained by modeling the word pairs through the BTM algorithm, and the initial number of topics of the BTM topic model is set, the topics are extracted from the word pairs through the BTM topic model, and the number of topics is continuously adjusted through the KL distance as an evaluation index of topic similarity, and when the average similarity between the topics extracted by the BTM topic model is the minimum, the trained BTM topic model is obtained;

[0021] After the topics of the document set are extracted through the trained BTM topic model, the following steps are used to determine the hot topic:

[0022] The topic importance index TI is set j (t i ) represents the importance of the ith topic t i in the jth document d j , which is considered through the following two factors:

[0023] That is, the ratio of the number of times that the topic t i appears in the document d j to the number of times that the topic t i appears in all documents M I j (t i ) and the ratio of the number of times that the topic t i appears in the document d j to the number of times that all other topics k appear in the document d j T j (t i ), I j (t i ) and T j (t i ) are calculated as shown in the following formula:

[0024]

[0025] Therefore, the calculation method of the topic importance index T j (t i ) is shown in the following formula:

[0026] TI j (t i )=I j (t i )·T j (t i )

[0027] The higher the value of the topic importance of the topic t i , the greater the amount of information contained in the topic t i , and the more representative it is, that is, the hot topic, and the importance of the topic t i in the entire document set is calculated through the following formula:

[0028]

[0029] wherein: TI(t i ) represents the importance of the subject t i in the entire document set.

[0030] As a preferred embodiment of the present application, the hot topic tracking model tracks hot topics through the following steps:

[0031] Divide the media articles in each type of network media webpage data in time sequence in different time windows, and identify hot topics in each time window;

[0032] Calculate the topic intensity of each hot topic in the time window through the average value of the distribution probability of the hot topic on each media article in the time window, and the topic intensity δ z of the hot topic z is calculated as follows:

[0033]

[0034] wherein: D i represents the number of media articles contained in the time window i; d j represents the jth media article in the time window i; represents the probability that the media article d j belongs to the hot topic z;

[0035] Calculate the proportion of the same feature words between two hot topics through the Jaccard coefficient, then calculate the probability that the feature words of two hot topics appear in the same media article through the word co-occurrence algorithm, and combine the two to build the JW algorithm to calculate whether the two hot topics are associated, and then simultaneously track the content of the associated hot topics.

[0036] As a preferred embodiment of the present application, the step of calculating the proportion of the same feature words between two hot topics through the Jaccard coefficient is:

[0037] Let the feature word vector set of the hot topic z1 be the feature word vector set of the hot topic z2 be wherein, n is the total number of feature words, and the proportion of the same feature words between two hot topics is calculated through the following formula:

[0038]

[0039]

[0040] ​The calculation formula of the probability that two hot topic characteristic words appear in the same media article at the same time through the word co-occurrence algorithm is as follows:

[0041]

[0042] Wherein: f is the total number of time windows; w is the total number of articles in the corresponding time window; ‖Segment‖ represents the number of articles in the two adjacent time windows where the hot topics z1 and z2 are located; represents the media articles containing the characteristic words in the two adjacent time windows at the same time;

[0043] The calculation formula of the JW algorithm obtained by combining the above formula is as follows:

[0044] JW(z1,z2)=γJaccard(z1,z2)+(1-γ)Wordco-occurrence(z1,z2)

[0045] Wherein: γ represents the weight;

[0046] The threshold is set, and when the value calculated by the JW algorithm of two hot topics is greater than the threshold, it is judged that the two hot topics have relevance.

[0047] As a preferred embodiment of the present application, the brand influence trend prediction model is based on a stacking ensemble learning model, which is composed of two layers of prediction models, the first layer of prediction models is composed of several types of sub-prediction models, and the second layer of prediction models is composed of an extreme gradient boosting model;

[0048] The first layer prediction model construction step is:

[0049] A plurality of different types of initial prediction models are selected in advance, and the models are trained through the pre-constructed training set;

[0050] After training, the error difference degree r of each initial prediction model is calculated through the Pearson correlation coefficient, and the specific formula is as follows: xy

[0051]

[0052] Wherein: represents the average value of the samples in the training set; x i represents the i-th sample in the training set; m represents the total number of samples in the training set or the test set; y i represents the i-th sample in the test set; represents the average value of the samples in the test set;

[0053] ​A preset error difference threshold is set, and an initial prediction model with an error difference greater than the threshold is taken as a sub-prediction model of the first layer prediction model.

[0054] As a preferred embodiment of the present application, the training step of the brand influence trend prediction model is:

[0055] Brand-related public opinion information is classified and hot topics are constructed into a training set, and the training set is divided into sub-training sets corresponding to the number of prediction models;

[0056] For any sub-prediction model in the first layer prediction model, any sub-training set is randomly selected as a test set, and the remaining training set is input into the prediction model for training. The sub-prediction model is evaluated by the corresponding test set, and the parameters of the sub-prediction model are adjusted according to the evaluation result to complete the training. The prediction result of each sub-prediction model is obtained by predicting the trained sub-prediction model.

[0057] A new training set is constructed by the prediction results of all sub-prediction models, and the new training set is input into the second layer prediction model for training to obtain the trained brand influence trend prediction model.

[0058] On the other hand, the present application also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the method according to any one of the embodiments of the present application.

[0059] In another aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to realize the method according to any one of the embodiments of the present application.

[0060] The present application has the following advantages:

[0061] 1. The present application predicts the influence trend of a brand based on machine learning through public opinion information data in all network media and forums on the Internet, providing early warning capabilities for brand owners. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 The present application is a method flowchart;

[0063] Figure 2 The present application is a media public opinion sentiment classification model structure diagram;

[0064] Figure 3 The present application is a hot topic mining flowchart;

[0065] Figure 4 The present application is a BTM topic model word pair generation flowchart;

[0066] Figure 5 A brand influence trend prediction model structure diagram of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0068] It should be understood that the step numbers used herein are only for the convenience of description, and are not limited to the execution sequence of the steps.

[0069] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0070] The terms "comprise" and "include" indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0071] The term "and / or" means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0072] Embodiment one:

[0073] Referring to Figure 1 A brand influence dynamic monitoring method based on big data driving includes the following steps:

[0074] Collecting web page data of various network media and various forums through a web crawler and pre-processing the web page data;

[0075] Building a media public opinion sentiment classification model, and classifying the sentiment of the public opinion information in the pre-processed web page data of various network media through the media public opinion sentiment classification model;

[0076] Building a hot topic mining model, and mining hot topics in the pre-processed web page data of various forums through the hot topic mining model, and building a hot topic tracking model, and tracking the hot topics through the hot topic tracking model;

[0077] A brand influence trend prediction model is constructed, which predicts the brand influence trend by classifying the sentiment of brand-related public opinion information and predicting hot topics.

[0078] In the embodiment, the brand influence score is calculated by setting a brand influence monitoring index.

[0079] As a preferred embodiment of the present embodiment, the specific steps of preprocessing the webpage data are as follows:

[0080] Filtering webpage labels in webpage data to obtain text data, removing punctuation and spaces in the text data, then cutting the sentences in the text data into word items by using the jieba word segmentation algorithm, filtering word items without specific meaning based on a preset stop word table, and finally converting all text data into word vector form;

[0081] Specifically, in the present embodiment, the read() function in Python is used to read data; in the second step, the compile() method in the re library is used to write a regular expression to filter out information irrelevant to text emotion analysis, wherein the expression [\u4e00-\u9fa5] can filter out punctuation in the text, and the expression [(?<=\>).*?(?=\<)] is used to filter out webpage labels in the crawler data; in the third step, the jieba library of Python is called to perform Chinese word segmentation on the text, and the sentences are cut into word items in units of words; in the fourth step, the stop_words stop word table is loaded to filter out words without specific meaning in the text; in the fifth step, the strip() function of Python is finally used to remove spaces in the text, and the corpus is read into the memory at one time and passed into jieba.cut(file.read()).

[0082] The first step of generating word vectors is to construct a word dictionary for training data, and the Counter is used to count the words. When constructing the word dictionary, only words with a frequency greater than 1 in the corpus are retained, wherein <pad>and <unk>are two initialized tokens, <pad>to fill in the blanks of a sentence, <unk>In this step, we get a dictionary containing 10384 words to replace the words that do not appear in the corpus; the second step needs to construct the word-to-token mapping and token-to-word mapping, using the Tokenizer class, receiving the size of the dictionary through the vocab_size parameter, after setting the size of the dictionary, those low-frequency words will be filtered out. Therefore, when generating the vector of the text later, the word vector of the low-frequency word outside the dictionary will no longer appear; the third step, instantiate an object t of Tokenizer, get the corresponding relationship between index and word in the dictionary through the t.word_index() method; the fourth step, use the t.texts_to_sequences() method to convert each sentence and paragraph in the text into an index represented vector, when converting the sentence vector represented by the index, since each sentence is of different length, it is not convenient for batch processing when training the neural network model later, therefore, a padding operation is needed to unify the length of all sentences, the method used is pad_sequences() method to fill the part of the sentence that is less than the length with 0, and cut off the part of the sentence that is more than the length. The fifth step, finally load the pre-trained word vector, which needs to be matched with the index in the dictionary to build an embedding matrix, that is, the index can be used to find the corresponding word vector, which completes the word vector technology and makes the text into a vector form that can be processed by the computer, laying a foundation for the subsequent sentiment analysis model.

[0083] As a preferred embodiment of the present embodiment, the media public opinion sentiment classification model is constructed based on a TextCNN-GRU model, and the specific steps of sentiment classification are as follows:

[0084] Based on the preprocessed various types of media webpage data, a training set and a test set are constructed;

[0085] The training set is input into the media public opinion sentiment classification model to train the model, and then the performance of the media public opinion sentiment classification model is evaluated through the test set. If the performance evaluation result of the media public opinion sentiment classification model is less than the preset performance threshold, the parameters of the media public opinion sentiment classification model are adjusted, and the training set is trained again to evaluate the performance. The operation is repeated until the performance of the media public opinion sentiment classification model is greater than the preset performance threshold, and the training is stopped. Thus, the trained media public opinion sentiment classification model is obtained.

[0086] The public opinion information in various types of media webpage data is classified through the trained media public opinion sentiment classification model.

[0087] Referring to Figure 2 In the present embodiment, the specific structure of the TextCNN-GRU model is as follows:

[0088] The model includes ten parts of embedding layer, convolution layer, nonlinear activation function layer, pooling layer, Dropout layer, Flatten layer, GRU layer, Dropout layer, fully connected layer and Sigmoid activation function layer;

[0089] (1) Embedding layer

[0090] The word2vec word vector transforms the dimension of the input vector through the embedding layer. Since the convolution operation requires a channels dimension in the Text-CNN model, after the embedding is constructed, the shape is actually (batch_size, vocab_size, embedding_size), but the convolution in TensorFlow requires a dimension of (batch_size, heights, widths, channels), so the channels dimension is added by the expand_dims method.

[0091] (2) Convolution layer

[0092] Also known as a convolution filter, it can extract features and capture local information of text during convolution. Since we use multiple filters, the convolution and pooling operations need to be processed for each filter. First, pooled_outputs is defined to store the output results of the convolution and pooling operations for each filter. For each filter, filter_size represents the size of the convolution kernel, i.e., the size of the sliding window, strides represents the convolution step, and padding takes the value of 'VALID' to indicate that only simple convolution is performed without padding 0. After the convolution operation, the relu function is used for activation.

[0093] (3) Pooling layer

[0094] The pooling layer is generally located after the convolution layer. It mainly extracts some local optimal features from the features obtained by the convolution layer, constantly reduces the amount of features extracted by the convolution layer, reduces the complexity of the model, prevents overfitting as much as possible, and discards redundant feature quantities while retaining useful information. The pooling layer is implemented based on the tf.nn.max_pool method, where pool_size represents the size of the pooling window, which is related to the length of the sentence vector and the size of the convolution kernel. Different filter_size corresponds to different pool_size, strides represents the pooling step, and padding takes the value of 'VALID' for simple pooling without padding. The output size is smaller than the input size.

[0095] (4) Dropout layer

[0096] Overfitting is a common problem when training neural networks. To prevent overfitting, we can improve the performance of the neural network by preventing the co-action of feature detectors. During forward propagation, we can stop the activation of a neuron with a certain probability p, reducing the over-reliance on some local features and making the model more general. In implementation, we use the tf.nn.dropout method, where the parameter keep_prob is the probability of each element being preserved.

[0097] (5) Flatten layer

[0098] The Flatten layer is used to convert the dimension of the input value, that is, to convert the two-dimensional features obtained by the previous layers into one-dimensional features. We use the tf.nn.flatten function.

[0099] (6) GRU layer

[0100] GRU has a simpler structure than LSTM, which does not contain the output gate and cell state in LSTM, and the functions of the forget gate and input gate in LSTM are realized by the update gate and reset gate. The update gate (reset gate) and reset gate can preserve long-distance sequence information and will not fade over time or be eliminated because it is irrelevant to prediction. We call the tf.nn.rnn_cell.GRUCell function to implement it. The important parameters of this function are: num_unit, which represents the number of neurons in the GRU cell, units, which represents the output dimension, and activation, which represents the activation function selected when implementing this layer.

[0101] (7) Fully connected layer

[0102] The fully connected layer is used to connect all the features captured by the previous convolutional and pooling layers and to weight and splice the output feature values. The fully connected layer is usually at the end of the CNN. It specifically connects each neuron of the previous layer to the neurons of this layer one by one, integrates the features extracted by the previous layers, and learns the high-level meaning of the features. We implement it based on the tf.layers.dense function, and the most important parameter is rate, which represents the proportion of neurons that need to be discarded during training.

[0103] (8) Classifier

[0104] The classifier is usually followed by the last fully connected layer, and the softmax function is usually used to output a number between 0 and 1, representing the classification label, i.e., the probability of the sample being classified into a certain class. The error update value is obtained by comparing the model output label with the true label of the sample, and is updated through stochastic gradient descent and backpropagation to the previous layers, updating the parameters in the network structure in a timely manner.

[0105] As a preferred embodiment of the present embodiment, refer to Figure 3 The hot topic mining model mines hot topics through steps as follows:

[0106] Each post in the preprocessed forum webpage data of each category is stored in a document and constitutes a document set, and all word pairs (also referred to as biterms) are generated by combining all word pairs in the document set two by two;

[0107] The BTM algorithm is used to model the word pairs to obtain a BTM topic model, and the initial number of topics of the BTM topic model is set; the BTM topic model extracts topics according to the word pairs, and the KL distance is used as an evaluation index of topic similarity to continuously adjust the number of topics; when the average similarity between the topics extracted by the BTM topic model is the minimum, the trained BTM topic model is obtained;

[0108] The BTM and the LDA are both topic models, and can generate keywords in each topic and probability distribution of each topic in a new document. The difference between the BTM and the LDA is that the BTM introduces the idea of word pairs, that is, each word pair generates a topic, and even if it is a short text, the number of word pairs is relatively large, thereby solving the problem of sparseness of document generation topics of the LDA model on short texts. The word pair refers to two words appearing in the same segment after preprocessing, and can be regarded as any two unordered co-occurrence words appearing in a document in a short text. The data generation process of the BTM can be described as a process in which each biterm is sequentially taken out from a specific topic of a corpus. The topics are sampled from all data in the corpus, and then judged according to the information of the word pairs, thereby obtaining the distribution of each topic in the text. According to the description of the paper author, the BTM performs better than the LDA on short texts, and even if it is a long text, the effect of the BTM is not weaker than that of the LDA;

[0109] The word pair generation process is as shown in Figure 4 The joint distribution probability formula of the word pair b=(b i , b j ) under the topic z calculated according to the model is as follows:

[0110]

[0111] The probability of the entire corpus is as follows:

[0112]

[0113] The biggest difference between BTM and traditional topic model is that BTM does not model the generation process of documents. Therefore, the proportion of topics in the document cannot be directly obtained in the topic learning process. In order to infer the topic in the document, it is assumed that the proportion of document topics is equal to the expected value of the topic in the biterm generated by the document, as shown in the following formula:

[0114] P(z|d)=∑ b P(z|b)P(b|d)

[0115] P(z|b) can be expressed by the Bayesian formula based on the parameter value predicted by BTM, as shown in the following formula:

[0116]

[0117] Where P(z)=θ z , P(w i |z)=Ф i|z , and P(b|d) can be estimated by the empirical distribution of biterm in the document, as shown in the following formula:

[0118]

[0119] Where n d (b) is the frequency of biterm b in document d. In short text, P(b|d) is approximately a uniform distribution. BTM model needs to obtain the values of θ and Ф according to existing parameters and corpus;

[0120] In this embodiment, Gibbs sampling is selected to infer the parameters of BTM;

[0121] First, the initial state is randomly selected for the Markov chain, and then the conditional distribution P(z|z i , B, α, β) of each biterm b=(w j , w b ) is calculated, where z b represents the topic distribution of all biterms except b, and B represents the word pair data set of all biterms. The joint probability formula of the whole data is as follows:

[0122]

[0123] Where: n z represents the number of times that biterm b is assigned to topic z, n w|z is the number of times that word w is assigned to topic z, and α and β are Dirichlet priors in LDA. It should be noted that when a biterm b is assigned to a topic z, the words w i and w j will be assigned to the topic at the same time;

[0124] Finally, the distribution of topic words Φ and the global topic distribution θ can be inferred by counting the biterm and word co-occurrence of the topic distribution, as shown in the following formula:

[0125]

[0126] The selection of the number of topics is as follows:

[0127] In the topic model, the selection of the number of topics K directly affects the quality of the model, thereby interfering with the clustering effect, so the setting of the parameter K is particularly important. A good model should maximize the degree of topic differentiation, so that the topics of each category do not interfere with each other, and the text topics can be clearly highlighted. In the study of text clustering, the main clustering assumption is that documents of the same class have a larger similarity, while documents of different classes have a smaller similarity. Therefore, when the average similarity between topics is the smallest, the topic model can be optimal.

[0128] The similarity between text vectors is often calculated using Euclidean distance or cosine similarity, but in order to preserve the characteristics of the topic probability distribution, a similarity function that can measure the distance between probability distributions, i.e., the Kullback-Leibler (KL) distance, is selected as the measurement standard. The calculation of the KL distance is shown in the following formula:

[0129]

[0130] where z i and z j are the probability distributions of two different topics, and represent the probabilities corresponding to the kth document topic z i and the topic z j , respectively, and N is the total number of documents. From the formula of the KL distance, it can be seen that for all k, KL(zi||zj)≠KL(zj||zi). Therefore, this embodiment uses another commonly used symmetric version of the KL distance, i.e., the Jensen-Shannon distance. The Jensen-Shannon distance is shown in the following formula:

[0131]

[0132] Then, the overall average similarity is calculated using the JS distance, as shown in the following formula:

[0133]

[0134] After extracting the topics of the document set by the trained BTM topic model, the following steps are used to determine the popular topics:

[0135] Setting the theme importance index TI j (t i ) represents the importance of the ith theme t i in the jth document d j , which is considered by the following two factors:

[0136] , namely the ratio of the number of times that the theme t i appears in the document d j to the number of times that the theme t i appears in all documents M, I j (t i ) and the ratio of the number of times that the theme t i appears in the document d j to the number of times that all other themes k appear in the document d j , T j (t i ), the calculation methods of I j (t i ) and T j (t i ) are shown in the following formula:

[0137]

[0138] Therefore, the calculation method of the theme importance index T j (t i ) is shown in the following formula:

[0139] TI j (t i ) = I j (t i ) · T j (t i )

[0140] The higher the value of the theme importance of the theme t i , the greater the amount of information contained in the theme t i , the more representative it is, i.e. a hot topic, and the importance of the theme t i in the entire document set is calculated by the following formula:

[0141]

[0142] wherein TI(t i ) represents the importance of the theme t i in the entire document set.

[0143] As a preferred embodiment of the present embodiment, the hot topic mining model mines hot topics through the following steps:

[0144] The pre-processed post data of each type of forum webpage is stored in a document and constitutes a document set, and all word pairs in the document set are combined to generate all possible word pairs;

[0145] The BTM topic model is obtained by modeling the word pairs through the BTM algorithm, and the initial number of topics of the BTM topic model is set. The topics are extracted from the word pairs through the BTM topic model, and the KL distance is used as an evaluation index of topic similarity to continuously adjust the number of topics. When the average similarity between the topics extracted by the BTM topic model is the smallest, the trained BTM topic model is obtained.

[0146] After the topics of the document set are extracted through the trained BTM topic model, the following steps are used to determine the hot topic:

[0147] The topic importance index TI is set j (t i ), which represents the importance of the ith topic t i in the jth document d j . This index considers the following two factors:

[0148] That is, the ratio of the number of occurrences of the topic t i in the document d j to the number of occurrences of the topic t i in all documents M I j (t i ) and the ratio of the number of occurrences of the topic t i in the document d j to the number of occurrences of all other topics k in the document d j T j (t i ). The calculation methods of I j (t i ) and T j (t i ) are shown in the following formulas:

[0149]

[0150] Therefore, the calculation method of the topic importance index T j (t i ) is shown in the following formula:

[0151] TI j (t i )=I j (t i )·T j (t i )

[0152] The topic t i The higher the value of the importance of the theme t i The more information it contains, the more representative it is, that is, a hot topic, and the importance of the theme t i In the entire document set:

[0153]

[0154] Where: TI(t i ) represents the importance of the theme t i In the entire document set.

[0155] 5. The brand influence dynamic monitoring method based on big data driving according to claim 1, wherein the hot topic tracking model tracks hot topics through the following steps:

[0156] Divide the media articles in various network media webpage data in time sequence in different time windows, and identify hot topics in each time window;

[0157] Calculate the topic intensity of each hot topic in the time window through the average value of the distribution probability of the hot topic on each media article in any time window, and the topic intensity δ z of the hot topic z is calculated as follows:

[0158]

[0159] Where: D i represents the number of media articles contained in the time window i; d j represents the jth media article in the time window i; represents the probability that the media article d j belongs to the hot topic z;

[0160] Calculate the proportion of the same feature words between two hot topics through the Jaccard coefficient, calculate the probability that the feature words of two hot topics appear in the same media article through the word co-occurrence algorithm, and construct the JW algorithm to calculate whether two hot topics are associated, and then track the content of the associated hot topics at the same time.

[0161] As a preferred embodiment of the present embodiment, the step of calculating the proportion of the same feature words between two hot topics through the Jaccard coefficient is:

[0162] Let the feature word vector set of the hot topic z1 be The feature word vector set of the hot topic z2 is Wherein, n is the feature word

[0163] The total number, the proportion of the same feature words between two hot topics is calculated by the following formula:

[0164]

[0165] The calculation formula of the probability that the feature words of two hot topics appear in the same media article is as follows:

[0166]

[0167] Wherein: f is the total number of time windows; w is the total number of articles in the corresponding time window; ‖Segment‖ represents the number of articles in the two adjacent time windows where the hot topics z1 and z2 are located; represents the media articles containing the feature words in the two adjacent time windows;

[0168] The calculation formula of the JW algorithm is obtained by combining the above formula as follows:

[0169] JW(z1,z2)=γJaccard(z1,z2)+(1-γ)Word co-occurrence(z1,z2)

[0170] Wherein: γ represents the weight;

[0171] A threshold value is set, and when the value calculated by the JW algorithm of two hot topics is greater than the threshold value, it is determined that the two hot topics have relevance.

[0172] As a preferred embodiment of the present embodiment, refer to Figure 5 The brand influence trend prediction model is based on a stacking ensemble learning model, which is composed of two layers of prediction models. The first layer of prediction model is composed of several types of sub-prediction models, and the second layer of prediction model is composed of an extreme gradient boosting model.

[0173] The first layer prediction model construction step is:

[0174] In order to obtain the optimal prediction effect, it is necessary to select a model with large difference as a base learning machine in the first layer of the stacking model. This is because different algorithm models essentially observe data from different data space angles and data structure angles, and then establish corresponding models according to the observation status of the algorithm and the algorithm principle itself. Therefore, selecting an algorithm with large difference can maximize the advantages of different algorithms and make each differentiated model complement each other;

[0175] A plurality of initial prediction models of different types are selected in advance, and the models are trained by using the training set constructed in advance;

[0176] After the training, the error difference degree r of each initial prediction model is calculated by the Pearson correlation coefficient xy The calculation is as follows:

[0177]

[0178] Wherein: represents the average value of the samples in the training set; x i represents the i-th sample in the training set; m represents the total number of samples in the training set or the test set; y i represents the i-th sample in the test set; represents the average value of the samples in the test set;

[0179] A preset error difference degree threshold is set, and the initial prediction model with an error difference degree greater than the threshold is used as a sub-prediction model of the first layer prediction model.

[0180] Specifically, in this embodiment, the XGBoost algorithm model, the LSTM model, the random forest (RF), the gradient boosted decision tree (GBDT), the support vector machine (SVM), and the KNN model are selected in the first layer prediction model.

[0181] The XGBoost (extreme gradient boosting tree) model is mainly constructed by using the XGBClassifier function function in the XGBoost library for modeling. After the XGBoost model is established, the model needs to be adjusted. Some parameters are adjusted to make the model more effective. In order to speed up the convergence speed, a higher learning rate (learning_rate) is selected, and the GridSearchCV function in sklearn is introduced for grid search to determine the number of trees. Then, the learning rate and the number of trees are given to optimize the decision tree, and then the regularization optimization is performed, and finally the learning rate is reduced to determine the final parameters.

[0182] That is, the number of decision trees in the parameter optimization of the model is 260, the learning rate is 0.01, the regularization coefficient gamma is 1, the sampling rate of the training data is 0.7, the maximum depth of the tree is 6, the regularization parameter is 100, the minimum tree model leaf weight is 2, and the feature sampling rate of the sample data is 0.7.

[0183] The long short-term memory network (LSTM) is improved on the basis of a recurrent neural network (RNN) and is slightly different from a general RNN in structure. The neuron of the LSTM adds a "gate" structure mechanism (three gates of input, forgetting, and output) on the basis of an RNN unit, and the "gates" can capture long-term memory and short-term memory along time steps, solving the key problem of gradient disappearance and gradient explosion that a standard RNN cannot handle long-term dependent information;

[0184] The random forest and the gradient boosting decision tree respectively adopt the ensemble learning methods of Bagging and Boosting, have excellent learning ability and rigorous mathematical theory support, and have been widely applied in various fields.

[0185] The support vector machine has unique advantages in solving small sample, nonlinearity, and high-dimensional regression problems. The KNN also has good practical application effect because of its mature theory and efficient training.

[0186] As a preferred embodiment of the present embodiment, the training step of the brand influence trend prediction model is:

[0187] The brand-related public opinion information sentiment classification and hot topics are constructed into a training set, and the training set is divided into sub-training sets corresponding to the number of prediction models.

[0188] It is worth noting that the training set of the meta-learner is generated from the output of the base learner. If the training set of the base learner is directly used to generate the secondary training set, serious overfitting may occur. In order to prevent the data from being repeatedly learned by the double-layered learner and avoid the occurrence of the "overfitting" effect, it is necessary to reasonably divide the data use process. According to the selected six base learners, the original training data set needs to be divided into six sub-data sets according to the time dimension, and the data IDs of each block are ensured to be non-overlapping. For a single base learner, one data block is used as a validation set, and the remaining five data blocks are used as a training set. Each base learner can output a prediction result for its own test data set, and the six results can be combined into a new data set. The new generated data set is the same size as the original data set. Thus, the feature transformation of all data from the input feature to the output feature is realized. Moreover, because each base learner's predicted data block does not participate in the training of the learner, such a configuration makes all data only used once during model training, effectively preventing overfitting.

[0189] For any sub-prediction model in the first layer prediction model, any sub-training set is randomly selected as a test set, and the remaining training set is input into the prediction model for training, and then the sub-prediction model is evaluated through the corresponding test set and trained by adjusting the parameters of the sub-prediction model according to the evaluation result, and the prediction result of each sub-prediction model is obtained through the trained sub-prediction model.

[0190] The new training set is constructed through the prediction results of all sub-prediction models, and then the new training set is input into the second layer prediction model for training to obtain the trained brand influence trend prediction model.

[0191] Embodiment two:

[0192] The embodiment provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method according to any embodiment of the present application when executing the program.

[0193] Embodiment three:

[0194] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executable on a processor to implement the method according to any embodiment of the present application.

[0195] In the embodiments of the present application, "at least one" refers to one or more, and "a plurality of" refers to two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent the cases of A alone, A and B together, and B alone. Wherein A and B can be singular or plural. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. "At least one of the following" and the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, wherein a, b, and c can be single or multiple.

[0196] Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be realized by electronic hardware, computer software and a combination of electronic hardware and computer software. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0197] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the system, device and unit described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described here.

[0198] In several embodiments provided in the present application, any function, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts of the technical solutions that make contributions to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory; hereinafter referred to as: ROM), a random access memory (Random Access Memory; hereinafter referred to as: RAM), a magnetic disk or an optical disk, and various media that can store program codes. The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent flow transformation obtained by using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.< / unk> < / pad> < / unk> < / pad>

Claims

1. A method for dynamically monitoring brand influence based on big data driving, characterized in that, The method comprises the following steps: collecting webpage data of various network media and various forums through a network crawler and preprocessing the webpage data; constructing a media public opinion sentiment classification model, and classifying the public opinion information in the preprocessed webpage data of various network media through the media public opinion sentiment classification model; constructing a hot topic mining model, and mining hot topics in the preprocessed webpage data of various forums through the hot topic mining model, and constructing a hot topic tracking model, and tracking the hot topics through the hot topic tracking model; constructing a brand influence trend prediction model, which predicts the influence trend of a corresponding brand by classifying the public opinion information related to the brand and predicting hot topics; The hot topic tracking model tracks hot topics through the following steps: dividing media articles in various network media webpage data into different time windows in chronological order, and identifying hot topics in each time window; The topic intensity of each popular topic in any time window is calculated by the average of the distribution probability of the popular topic on each media article in the time window, and the topic intensity of the popular topic z is denoted as z The calculation formula is as follows: where: D i denotes the number of media articles contained in the time window i; d j denotes the jth media article in the time window i; denotes the media article d j belongs to the popular topic z; calculating the proportion of the same feature words between two hot topics through a Jaccard coefficient, then calculating the probability of two hot topic feature words appearing in the same media article through a word co-occurrence algorithm, and combining the two to construct a JW algorithm to calculate whether two hot topics are related, and then tracking the content of the related hot topics at the same time; The step of calculating the proportion of the same feature words between two hot topics through a Jaccard coefficient is as follows: The characteristic word vector set of a hot topic z1 The characteristic word vector set of a hot topic z2 Wherein n is the total number of characteristic words, and the proportion of the same characteristic words between two hot topics is calculated by the following formula: The calculation formula of the probability of two hot topic feature words appearing in the same media article through a word co-occurrence algorithm is as follows: wherein: f is the total number of time windows; w is the total number of articles in the corresponding time window; ||Segment|| represents the number of articles in the two adjacent time windows where the popular topic z1, z2 is located; represents the media articles containing the feature words in the two adjacent time windows simultaneously; The calculation formula of the JW algorithm is as follows: JW(z1,z2)=γJaccard(z1,z2)+(1-γ)Word co-occurrence(z1,z2) Where: γ represents the weight; A threshold is set, and when the value calculated by the JW algorithm for two hot topics is greater than the threshold, it is determined that the two hot topics are related.

2. The brand influence dynamic monitoring method based on big data driving according to claim 1, characterized in that, The preprocessing steps of the webpage data are as follows: Filtering webpage tags in webpage data to obtain text data, removing punctuation and spaces in the text data, then cutting the sentences in the text data into word items through a jieba segmentation algorithm, then filtering word items without specific meaning based on a pre-set stop word table, and finally converting all text data into word vector form. 3.The brand influence dynamic monitoring method based on big data driving according to claim 1, characterized in that, The media public opinion sentiment classification model is based on a TextCNN-GRU model, and the specific steps of sentiment classification are as follows: Based on the preprocessed media webpage data, a training set and a test set are constructed; input the training set into the media public opinion sentiment classification model to train the model, then evaluate the performance of the media public opinion sentiment classification model through the test set, if the performance evaluation result of the media public opinion sentiment classification model is less than the pre-set performance threshold, adjust the parameters of the media public opinion sentiment classification model and retrain through the training set, then evaluate the performance again, repeat the operation until the performance of the media public opinion sentiment classification model is greater than the pre-set performance threshold, then stop training, and obtain the trained media public opinion sentiment classification model; The media public opinion sentiment classification model trained classifies the public opinion information in the media webpage data.

4. The brand influence dynamic monitoring method based on big data driving according to claim 1, characterized in that, The hot topic mining model mines hot topics through the following steps: The preprocessed forum webpage data is stored in a document and forms a document set, and all word pairs in the document set are combined to generate all possible word pairs. The BTM topic model is obtained by modeling the word pairs through the BTM algorithm, and the initial number of topics of the BTM topic model is set. The topics are extracted from the word pairs through the BTM topic model, and the KL distance is used as an evaluation index of topic similarity to continuously adjust the number of topics. When the average similarity between the topics extracted by the BTM topic model is the smallest, the trained BTM topic model is obtained. After the topics of the document set are extracted through the trained BTM topic model, the hot topics are determined through the following steps: Setting the topic importance indicator TI j (t i ), represents the importance of the ith topic t i in the jth document d j , this indicator takes into account the following two factors: That is, topic t i In document d j The number of times it appears in topic t i The ratio of the number of times I appears in all documents M j (t i ) and theme t i In document d j The number of times it appears in document d j The ratio T of the number of occurrences of all other themes k. j (t i ), I j (t i ) and T j (t i The calculation method is shown in the following formula: Thus, the subject importance indicator T j (t i ) is calculated as follows: TI j (t i )=I j (t i )·T j (t i ) Topic t i The higher the topic importance score, the greater the importance of topic t. i The more information it contains, the more representative it is, and thus, the more popular it is. The topic t is then calculated using the following formula. i Importance within the entire document set: where: TI(t i ) represents the importance of topic t i in the entire document collection.

5. The brand influence dynamic monitoring method based on big data driving according to claim 1, characterized in that, The brand influence trend prediction model is based on a stacking ensemble learning model, which is composed of two layers of prediction models. The first layer prediction model is composed of several types of sub-prediction models, and the second layer prediction model is composed of an extreme gradient boosting model. The first layer prediction model is constructed by: Several different types of initial prediction models are selected in advance, and the models are trained through the pre-constructed training set. After the training, the error difference degree r of each initial prediction model is calculated by Pearson correlation coefficient xy The calculation is performed as shown in the following equation: where: represents the mean of the samples in the training set; x i represents the i-th sample in the training set; m represents the total number of samples in the training set or the test set; y i represents the i-th sample in the test set; represents the mean of the samples in the test set; An error difference threshold is preset, and the initial prediction model with an error difference greater than the threshold is used as a sub-prediction model of the first layer prediction model.

6. The brand influence dynamic monitoring method based on big data driving according to claim 5, characterized in that, The training steps of the brand influence trend prediction model are as follows: The public opinion information sentiment classification and related hot topics related to the brand are constructed as a training set, and the training set is divided into sub-training sets corresponding to the number of prediction models. For any sub-prediction model in the first layer prediction model, randomly select any sub-training set as a test set, and input the remaining training set into the prediction model for training. The sub-prediction model is trained by evaluating the sub-prediction model through the corresponding test set and adjusting the parameters of the sub-prediction model according to the evaluation result. The prediction result of each sub-prediction model is obtained by predicting the trained sub-prediction model. A new training set is constructed through the prediction results of all sub-prediction models, and the new training set is input into the second layer prediction model for training to obtain the trained brand influence trend prediction model.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method of any one of claims 1 to 6.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1 to 6.

Images