A brand data intelligent identification and diagnosis method and system based on big data

Abstract

The present invention discloses a brand data intelligent identification and diagnosis method and system based on big data, including obtaining brand data and brand history data, preprocessing the brand data; extracting multiple categories of features from the brand data to obtain feature data, clustering the feature data to obtain diagnostic data, and performing correlation analysis on the feature data to obtain evaluation data; constructing a calculation model, training and testing the model based on the brand history data, inputting the diagnostic data into the calculation model to obtain a diagnosis result, and inputting the evaluation data into the calculation model to obtain an evaluation result; combining the diagnosis result and the evaluation result for analysis, obtaining an analysis result and outputting it. This method can not only improve the accuracy of the brand data intelligent identification and diagnosis method based on big data, but also has good interpretability, and can be directly applied to the brand data intelligent identification and diagnosis system of big data.

Description

Technical Field

[0001] The present invention relates to the field of big data, and in particular to a method and system for intelligently identifying and diagnosing brand data based on big data. Background Art

[0002] The application of diagnostic technology in the field of big data is becoming more and more extensive, which can help brand managers to diagnose and evaluate corporate brands in a timely and efficient manner, and realize timely management of corporate brands. At present, the intelligent identification and diagnosis of brand data based on big data has the characteristics of huge amount of information, diverse data types, and high information density. There are many uncertain factors in the intelligent identification and diagnosis method of brand data, which leads to great uncertainty in the intelligent identification and diagnosis method of brand data. Although some intelligent identification and diagnosis methods of brand data for diagnostic systems have been invented, they still cannot effectively solve the uncertainty problem of the intelligent identification and diagnosis method of brand data. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for intelligent identification and diagnosis of brand data based on big data.

[0004] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0005] The present invention comprises the following steps:

[0006] A obtains brand data and brand history data, and preprocesses the brand data, wherein the brand data includes text data and numerical data;

[0007] B. constructing a feature extraction model, performing feature extraction of the first category on the text data to obtain first feature data; performing feature extraction of the second category on the numerical data to obtain second feature data; performing cluster analysis on the first feature data and the second feature data to obtain diagnostic data, and performing correlation analysis on the first feature data and the second feature data to obtain evaluation data;

[0008] C builds a calculation model, divides the brand history data into training data and test data according to the ratio of n:m, trains the model with the training data, tests the model with the test data, inputs the diagnosis data into the brand diagnosis model to obtain a diagnosis result, and inputs the evaluation data into the calculation model to obtain an evaluation result;

[0009] D combines the diagnosis result and the evaluation result for analysis and outputs the analysis result.

[0010] Furthermore, the preprocessing in step A includes performing text cleaning, stop word removal, punctuation removal, noise removal and word form restoration on the text data; and performing data cleaning, missing value processing, outlier removal and error value removal on the numerical data.

[0011] Furthermore, the method for constructing the feature extraction model includes:

[0012] The feature extraction model includes a sentiment tendency feature extraction module, a keyword feature extraction module, a topic feature extraction module, a time domain feature extraction module and a statistical feature extraction module; the sentiment tendency feature extraction module is based on a word embedding model and a HowNet sentiment dictionary, and uses a Senta sentiment analysis system to extract the sentiment tendency of the text data; the keyword feature extraction module extracts keywords from the text data according to a TF-IDF algorithm; the topic feature extraction module extracts the topic distribution of the text data based on an LDA topic model and uses a Gibbs sampling technique; the time domain feature extraction module extracts the time domain features of the numerical data according to a sliding window algorithm; the statistical feature extraction module extracts the statistical features of the numerical data according to a statistical analysis technique.

[0013] Further, the method of performing feature extraction of the first category on the text data to obtain first feature data; and performing feature extraction of the second category on the numerical data to obtain second feature data includes:

[0014] The first category includes keyword feature extraction, topic feature extraction and sentiment tendency feature extraction; the second category includes time domain feature extraction and statistical feature extraction; keyword feature extraction is performed on the text data to obtain keyword features, topic feature extraction is performed on the text data to obtain topic features, and sentiment tendency feature extraction is performed on the text data to obtain sentiment features, and the keyword features, the topic features and the sentiment features are first feature data; time domain feature extraction is performed on the numerical data to obtain time domain features, and statistical feature extraction is performed on the numerical data to obtain statistical features, and the time domain features and the statistical features are second feature data.

[0015] Furthermore, the method of performing cluster analysis on the first feature data and the second feature data to obtain diagnostic data includes:

[0016] The key feature data in the first feature data and the second feature data is recorded as x i and x j , calculate x i and x j The similarity between:

[0017]

[0018] where s ij For x i and x j The similarity between ij For xi and x j The Euclidean distance between them, k is x i and x j The dimension is k = 1, 2, ..., d, and the similarity matrix S = (s ij ), sort all similarity values ​​in ascending order, use K-means algorithm to classify all similarity values ​​into two categories, and take the category with larger mean as the intra-class similarity set s in , calculate the truncated similarity:

[0019]

[0020] where s c is the cutoff similarity, |s in | is the intra-class similarity set s in The number of elements in , calculate the local density:

[0021]

[0022] where ρ i is the key feature data x i The local density of , calculate the relative similarity:

[0023]

[0024] where δ i is the key feature data x i The relative similarity of argmax is a mathematical operation, which means selecting the key feature data with the largest local density from all key feature data, ρ n is the local density of the nth key feature data, ρ j is the key feature data x j The local density, x n For one of the key feature data, calculate the cluster center weight:

[0025]

[0026] where γ i is the key feature data x i The cluster center weights, * is a mathematical operation, which means that after normalization, the cluster center weights are arranged in descending order to obtain the sequence γ1>γ2,......,γ k-1 >γ k >γ k+1 ,......,γ N , for each point γ k Linear regression analysis is performed on the left and right sides of (2≤k≤N-1) to obtain two γ value prediction sequences: and {γk+ '1,γ k+ '2,...,γ N '}, calculate the change point γ t :

[0027]

[0028] Where t is the change point γ t The index of E is argmin, which is a mathematical operation. k Value, E k For γ i The sum of squared errors, 2≤k≤N-1, γ i is the key feature data x i The cluster center weight, For γ i The cluster center prediction weight on the left, γ i 'for γ i The cluster center prediction weight on the right; select the change point and its previous points as the initial cluster center, and according to the allocation strategy of the DPC algorithm, assign the non-cluster center point to the cluster with the highest density point with the greatest similarity, and get the initial cluster P = {P1, P2, ..., P cp}, define the key feature data x that meets the following conditions n For outliers:

[0029]

[0030] where μ(·) and σ(·) represent the mean and standard deviation of the data, and the set of outliers is denoted by x out , calculate the similarity radius:

[0031]

[0032] Where r is the similarity radius, X is the key feature data set, for each key feature data, the key feature data with a similarity greater than or equal to r are called the neighbors of the key feature data, and the K-means algorithm is used to sort the neighbors of the key feature data in descending order and cut them into two ends. The cutoff point is the critical number of neighbors. The key feature data with a neighbor number greater than the critical number of neighbors is the core data, and the set is recorded as x core ; For any two initial clusters P i and P j , calculate the maximum similarity of the two initial clusters, and record the key feature data with the maximum similarity between the two initial clusters as and Right now:

[0033]

[0034] in for and The similarity between the two initial clusters P i and P j The maximum similarity, s uv For the two initial clusters P i and P j The similarity between any two key feature data, x u is the initial cluster P i One of the key feature data, x v is the initial cluster P j One of the key feature data in, if any two initial clusters P i and P j satisfy:

[0035]

[0036] Then merge the two clusters and update the clustering result. Repeat the above steps until any two clusters do not satisfy equation (1) and obtain the final cluster C = {C1, C2, ..., C nc}, and output the cluster results as diagnostic data.

[0037] Furthermore, the method of performing correlation analysis on the first feature data and the second feature data to obtain evaluation data includes:

[0038] The key feature data in the key feature data is recorded as x a and x b , according to the KMedian algorithm, x a and x b Divide into different data clusters, get a grid division value of m rows and n columns, and calculate x based on the obtained grid division value a and x b The joint probability distribution and marginal probability distribution between them, and then we get x a and x b The mutual information is:

[0039]

[0040] Where I(x a ;x b ) is x a and x b The mutual information of a ;x b ) is x a and x b The joint probability distribution of a ) is x a The marginal probability distribution of p(xb ) is x b The marginal probability distribution of ; Based on the mutual information, calculate x a and x b The correlation coefficient between:

[0041]

[0042] Where MIC(x a ;x b ) is x a and x b The correlation coefficient between them, m is the number of grid columns, n is the number of grid rows, N is the total number of key feature data, I(x a ;x b ) is x a and x b The mutual information between them is obtained, a correlation threshold a=0.5 is set, a correlation coefficient between every two key feature data in the key feature data is calculated, and key feature data with a correlation coefficient greater than the correlation threshold is allocated to the evaluation data to obtain the evaluation data.

[0043] Furthermore, a calculation model is constructed, and the brand history data is divided into training data and test data according to the ratio of n:m. The method of using the training data to train the model and using the test data to test the model includes:

[0044] The brand history data includes brand index data and brand feature data. The brand index data in the training data is used as the dependent variable and the brand feature data is used as the independent variable to construct a multivariate linear regression model:

[0045] Y=βX+ε

[0046] Where Y is the brand index data, Y = {Y1, Y2, Y3}, X is the brand feature data, β is the regression coefficient, β={β0,β1,...,β n}, ε is the random error, ε={ε0,ε1,...,ε n}; Construct the residual sum of squares formula:

[0047]

[0048] in is the residual sum of squares, is the predicted regression coefficient, τ is a mathematical expression, which means transposition. The predicted regression coefficient in the residual sum of squares is derived and set equal to zero to obtain the predicted regression coefficient value:

[0049]

[0050] According to the calculated predicted regression coefficient value, the trained multivariate linear regression model is obtained:

[0051]

[0052] in is the predicted brand indicator data, X is the brand feature data, is the calculated regression coefficient, ε is random error, ε={ε0,ε1,...,ε n}; Use the test data to test the prediction effect of the model:

[0053]

[0054] Where R 2 is the coefficient of determination, R 2 The closer the value is to 1, the better the prediction effect is. Y is the brand index data in the test data, Y = {Y1, Y2, Y3}, is the predicted data obtained through model prediction. is the mean of the brand indicator data in the test data,

[0055] Furthermore, the method of combining the diagnosis result and the evaluation result for analysis and outputting the analysis result includes:

[0056] Invite M experts to score, and combine the data in the diagnosis results and the evaluation results into a set U = {u1, u2, ..., u n}, each expert compares and scores the data in set U in pairs, and the rth expert gives the uth i data and uth j The data is scored Calculate the rth expert score Deviation from the mean:

[0057]

[0058] in Score the rth expert Deviation from the mean, is the mean, i,j=1,2,...,n; r=1,2,...,M, and the rth expert score is calculated based on the deviation Weight:

[0059]

[0060] Where λ(r) is the score of the rth expert The weight of the score is normalized as follows:

[0061]

[0062] where λ * (r) is the rth expert score after normalization The weights are r=1,2,...,M. The scores between each two data in set U are calculated according to the modified weights to obtain the comparison judgment matrix:

[0063] A=[a ij ] n×n

[0064] Where A is the comparison judgment matrix, a ij For the score between each two data in set U, according to the comparison judgment matrix, calculate the maximum characteristic root λ of the comparison judgment matrix max :

[0065]

[0066] Among them, M i To compare the product of each row of the matrix, M j To compare the product of each column element of the matrix, n is the number of rows and columns, according to the maximum characteristic root λ max Find the corresponding eigenvector:

[0067] AW ′ =λ max W ′

[0068] Where W ′ is the largest characteristic root λ max Corresponding feature vector, the feature vector is processed logarithmically to obtain the weight vector W, where W = logW ′ The weight vector is the weight distribution of the importance of each data in the diagnosis result and the evaluation result to the brand score, and the brand score is calculated:

[0069]

[0070] Where C is the brand score, u i For the set U = {u1,u2,...,u n}, the i-th data, w i is the weight vector W in u i The corresponding weight distribution, n is the total number of data in the set U. A radar chart is drawn according to the diagnosis result, a line chart is drawn according to the evaluation result, and the brand score, radar chart and line chart are output as analysis results.

[0071] The beneficial effects of the present invention are:

[0072] The present invention is a brand data intelligent identification and diagnosis method and system based on big data. Compared with the prior art, the present invention has the following technical effects:

[0073] The present invention can improve the accuracy of brand data diagnosis through preprocessing, feature extraction, cluster analysis, correlation analysis and calculation model construction steps, thereby improving the precision of brand data diagnosis. By making brand data diagnosis intelligent, it can greatly save resources and manpower costs and improve work efficiency. It can realize automatic identification and diagnosis of brand data and perform feature extraction and calculation of brand data in real time, which is of great significance to brand data diagnosis and can adapt to brand data intelligent diagnosis systems for different products and has a certain degree of universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The present invention is a flowchart of the steps of a method and system for intelligent identification and diagnosis of brand data based on big data. DETAILED DESCRIPTION

[0075] The present invention is further described below by means of specific embodiments. The illustrative embodiments and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0076] The present invention provides a method and system for intelligently identifying and diagnosing brand data based on big data, comprising the following steps:

[0077] like Figure 1 As shown, in this embodiment, the following steps are included:

[0078] A obtains brand data and brand history data, and preprocesses the brand data, wherein the brand data includes text data and numerical data;

[0079] In actual evaluation, taking agricultural product brands as an example, the brand data of a certain agricultural product is obtained as follows:

[0080]

[0081]

[0082]

[0083] The brand historical data of the agricultural product are: brand awareness, brand reputation, brand loyalty, market trends, consumer behavior and fierce competition situation as brand indicator data; emotional tendency characteristics, theme characteristics, keyword characteristics, time domain characteristics and statistical characteristics as brand characteristic data.

[0084] B. constructing a feature extraction model, performing feature extraction of the first category on the text data to obtain first feature data; performing feature extraction of the second category on the numerical data to obtain second feature data; performing cluster analysis on the first feature data and the second feature data to obtain diagnostic data, and performing correlation analysis on the first feature data and the second feature data to obtain evaluation data;

[0085] In the actual evaluation, the first feature data is: ① The keyword features are "very delicious", "not bad", "not delicious", "diarrhea", "especially love to eat" and "never buy again"; ② The theme features are taste, quality and continued support; ③ The emotional tendency features are positive values: 8, neutral values: 4, and negative values: 6; The second feature data is: The time domain features are: ⑴ The number of consumers who repeat purchases is 290; ⑵ The number of consumers at the end of the quarter is 135, the number of newly acquired consumers is 55, and the number of consumers at the beginning of the quarter is 110; ⑶ The customer retention rate is 73%; ⑷ The repeat purchase rate is 54%; The statistical characteristics are: ⑸ The number of consumers who correctly identify the brand is 1; ⑹The total number of consumers participating in the survey is 4; ⑺The social media interaction score is 120; ⑻The number of recommenders is 3, and the number of detractors is 1; ⑼The satisfaction score of all respondents is 23; ⑽The brand image score is 20; ⑾The number of advertising exposures is 8,100 times, and the brand search volume is 1,370; ⑿The total number of consumers is 540; ⒀The percentage change in price is an increase of 20%, and the percentage change in demand is an increase of 10%; ⒁The recognition rate is 25%; ⒂The exposure is 2427.5; ⒃The market share is 10%; ⒄The net recommendation value is 50%; ⒅The satisfaction score is 5.75; ⒆The brand image share is 0.47; ⒇The price sensitivity is 0.5;

[0086] The diagnostic data are as follows: Category 1 includes ①, ②, ③, ⑸, ⑺, ⑾, and ⒂; Category 2 includes ⑹, ⑻, ⑼, ⑽, ⒅, and ⒆; Category 3 includes ⑴, ⑿, ⑵, ⒀, ⒁, ⒃, ⒄, ⑷, ⑶, and ⒇;

[0087] The evaluation data are: (1), (2), (3), (4), (5), (6), (7), and (8);

[0088] C builds a calculation model, divides the brand history data into training data and test data according to the ratio of n:m, trains the model with the training data, tests the model with the test data, inputs the diagnosis data into the brand diagnosis model to obtain a diagnosis result, and inputs the evaluation data into the calculation model to obtain an evaluation result;

[0089] In the actual evaluation, the diagnosis results are: brand awareness is 54.48, brand reputation is 2.33 and brand loyalty is 1.98; the evaluation results are: market trends change rapidly, consumer behavior is changeable and the competition situation is fierce.

[0090] D combines the diagnosis result and the evaluation result for analysis and outputs the analysis result.

[0091] In the actual evaluation, the brand score was 568.77. The values ​​of brand awareness, brand reputation and brand loyalty were plotted as a radar chart, and the market trends, consumer behavior and competition situation were plotted as a line chart. The brand score, radar chart and line chart were output as the analysis results.

[0092] In this embodiment, the preprocessing in step A includes performing text cleaning, stop word removal, punctuation removal, noise removal and word form restoration on the text data; and performing data cleaning, missing value processing, outlier removal and error value removal on the numerical data.

[0093] In this embodiment, the method for constructing the feature extraction model includes:

[0094] The feature extraction model includes a sentiment tendency feature extraction module, a keyword feature extraction module, a topic feature extraction module, a time domain feature extraction module and a statistical feature extraction module; the sentiment tendency feature extraction module is based on a word embedding model and a HowNet sentiment dictionary, and uses a Senta sentiment analysis system to extract the sentiment tendency of the text data; the keyword feature extraction module extracts keywords from the text data according to a TF-IDF algorithm; the topic feature extraction module extracts the topic distribution of the text data based on an LDA topic model and uses a Gibbs sampling technique; the time domain feature extraction module extracts the time domain features of the numerical data according to a sliding window algorithm; the statistical feature extraction module extracts the statistical features of the numerical data according to a statistical analysis technique.

[0095] In this embodiment, the method of performing a first-category feature extraction on the text data to obtain first feature data and performing a second-category feature extraction on the numerical data to obtain second feature data includes:

[0096] The first category includes keyword feature extraction, topic feature extraction and sentiment tendency feature extraction; the second category includes time domain feature extraction and statistical feature extraction; keyword feature extraction is performed on the text data to obtain keyword features, topic feature extraction is performed on the text data to obtain topic features, and sentiment tendency feature extraction is performed on the text data to obtain sentiment features, and the keyword features, the topic features and the sentiment features are first feature data; time domain feature extraction is performed on the numerical data to obtain time domain features, and statistical feature extraction is performed on the numerical data to obtain statistical features, and the time domain features and the statistical features are second feature data.

[0097] In this embodiment, the method for performing cluster analysis on the first feature data and the second feature data to obtain diagnostic data includes:

[0098] The key feature data in the first feature data and the second feature data is recorded as x i and x j , calculate x i and x j The similarity between:

[0099]

[0100] where s ij For x i and x j The similarity between ij For x i and x j The Euclidean distance between them, k is x i and x j The dimension is k = 1, 2, ..., d, and the similarity matrix S = (s ij ), sort all similarity values ​​in ascending order, use K-means algorithm to classify all similarity values ​​into two categories, and take the category with larger mean as the intra-class similarity set s in , calculate the truncated similarity:

[0101]

[0102] where s c is the cutoff similarity, |s in | is the intra-class similarity set s in The number of elements in , calculate the local density:

[0103]

[0104] where ρ i is the key feature data x i The local density of , calculate the relative similarity:

[0105]

[0106] where δ i is the key feature data x i The relative similarity of argmax is a mathematical operation, which means selecting the key feature data with the largest local density from all key feature data, ρ n is the local density of the nth key feature data, ρ j is the key feature data x j The local density, x nFor one of the key feature data, calculate the cluster center weight:

[0107]

[0108] where γ i is the key feature data x i The cluster center weights, * is a mathematical operation, which means that after normalization, the cluster center weights are arranged in descending order to obtain the sequence γ1>γ2,......,γ k-1 >γ k >γ k+1 ,......,γ N , for each point γ k Linear regression analysis is performed on the left and right sides of (2≤k≤N-1) to obtain two γ value prediction sequences: and Calculate the change point γ t :

[0109]

[0110] Where t is the change point γ t The index of E is argmin, which is a mathematical operation. k Value, E k For γ i The sum of squared errors, 2≤k≤N-1, γ i is the key feature data x i The cluster center weight, For γ i The cluster center prediction weight on the left, γ i 'for γ i The cluster center prediction weight on the right; select the change point and its previous points as the initial cluster center, and according to the allocation strategy of the DPC algorithm, assign the non-cluster center point to the cluster with the highest density point with the greatest similarity, and get the initial cluster P = {P1, P2, ..., P cp}, define the key feature data x that meets the following conditions n For outliers:

[0111]

[0112] where μ(·) and σ(·) represent the mean and standard deviation of the data, and the set of outliers is denoted by x out , calculate the similarity radius:

[0113]

[0114] Where r is the similarity radius, X is the key feature data set, for each key feature data, the key feature data with a similarity greater than or equal to r are called the neighbors of the key feature data, and the K-means algorithm is used to sort the neighbors of the key feature data in descending order and cut them into two ends. The cutoff point is the critical number of neighbors. The key feature data with a neighbor number greater than the critical number of neighbors is the core data, and the set is recorded as x core ; For any two initial clusters P i and P j , calculate the maximum similarity of the two initial clusters, and record the key feature data with the maximum similarity between the two initial clusters as and Right now:

[0115]

[0116] in for and The similarity between the two initial clusters P i and P j The maximum similarity, s uv For the two initial clusters P i and P j The similarity between any two key feature data, x u is the initial cluster P i One of the key feature data, x v is the initial cluster P j One of the key feature data in, if any two initial clusters P i and P j satisfy:

[0117]

[0118] Then merge the two clusters and update the clustering result. Repeat the above steps until any two clusters do not satisfy equation (1) and obtain the final cluster C = {C1, C2, ..., C nc}, and output the cluster results as diagnostic data.

[0119] In this embodiment, the method of performing correlation analysis on the first feature data and the second feature data to obtain evaluation data includes:

[0120] The key feature data in the key feature data is recorded as x a and x b , according to the KMedian algorithm, x a and x b Divide into different data clusters, get a grid division value of m rows and n columns, and calculate x based on the obtained grid division valuea and x b The joint probability distribution and marginal probability distribution between them, and then we get x a and x b The mutual information is:

[0121]

[0122] Where I(x a ;x b ) is x a and x b The mutual information of a ;x b ) is x a and x b The joint probability distribution of a ) is x a The marginal probability distribution, p(x b ) is x b The marginal probability distribution of ; Based on the mutual information, calculate x a and x b The correlation coefficient between:

[0123]

[0124] Where MIC(x a ;x b ) is x a and x b The correlation coefficient between them, m is the number of grid columns, n is the number of grid rows, N is the total number of key feature data, I(x a ;x b ) is x a and x b The mutual information between them is obtained, a correlation threshold a=0.5 is set, a correlation coefficient between every two key feature data in the key feature data is calculated, and key feature data with a correlation coefficient greater than the correlation threshold is allocated to the evaluation data to obtain the evaluation data.

[0125] In this embodiment, a calculation model is constructed, brand history data is divided into training data and test data according to a ratio of n:m, and a method of using the training data to train the model and using the test data to test the model includes:

[0126] The brand history data includes brand index data and brand feature data. The brand index data in the training data is used as the dependent variable and the brand feature data is used as the independent variable to construct a multivariate linear regression model:

[0127] Y=βX+ε

[0128] Where Y is the brand index data, Y = {Y1, Y2, Y3}, X is the brand feature data, β is the regression coefficient, β={β0,β1,...,β n}, ε is the random error, ε = {ε0, ε1,...,ε n}; Construct the residual sum of squares formula:

[0129]

[0130] in is the residual sum of squares, is the predicted regression coefficient, τ is a mathematical expression, which means transposition. The predicted regression coefficient in the residual sum of squares is derived and set equal to zero to obtain the predicted regression coefficient value:

[0131]

[0132] According to the calculated predicted regression coefficient value, the trained multivariate linear regression model is obtained:

[0133]

[0134] in is the predicted brand indicator data, X is the brand feature data, is the calculated regression coefficient, ε is random error, ε={ε0,ε1,...,ε n}; Use the test data to test the prediction effect of the model:

[0135]

[0136] Where R 2 is the coefficient of determination, R 2 The closer the value is to 1, the better the prediction effect is. Y is the brand index data in the test data, Y = {Y1, Y2, Y3}, is the predicted data obtained through model prediction. is the mean of the brand indicator data in the test data,

[0137] In this embodiment, the method of combining the diagnosis result and the evaluation result for analysis and outputting the analysis result includes:

[0138] Invite M experts to score, and combine the data in the diagnosis results and the evaluation results into a set U = {u1, u2, ..., u n}, each expert compares and scores the data in set U in pairs, and the rth expert gives the uth i data and uth j The data is scored Calculate the rth expert score Deviation from the mean:

[0139]

[0140] in Score the rth expert Deviation from the mean, is the mean, i,j=1,2,...,n; r=1,2,...,M, and the rth expert score is calculated based on the deviation Weight:

[0141]

[0142] Where λ(r) is the score of the rth expert The weight of the score is normalized as follows:

[0143]

[0144] where λ * (r) is the rth expert score after normalization The weights are r=1,2,...,M. The scores between each two data in set U are calculated according to the modified weights to obtain the comparison judgment matrix:

[0145] A=[a ij ] n×n

[0146] Where A is the comparison judgment matrix, a ij For the score between each two data in set U, according to the comparison judgment matrix, calculate the maximum characteristic root λ of the comparison judgment matrix max :

[0147]

[0148] Among them, M i To compare the product of each row of the matrix, M j To compare the product of each column element of the matrix, n is the number of rows and columns, according to the maximum characteristic root λ max Find the corresponding eigenvector:

[0149] AW ′ =λ max W ′

[0150] Where W ′ is the largest characteristic root λ max Corresponding feature vector, the feature vector is processed logarithmically to obtain the weight vector W, where W = logW ′ The weight vector is the weight distribution of the importance of each data in the diagnosis result and the evaluation result to the brand score, and the brand score is calculated:

[0151]

[0152] Where C is the brand score, u i For the set U = {u1,u2,...,u n}, the i-th data, w i is the weight vector W in u i The corresponding weight distribution, n is the total number of data in the set U. A radar chart is drawn according to the diagnosis result, a line chart is drawn according to the evaluation result, and the brand score, radar chart and line chart are output as analysis results.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A brand data intelligent identification and diagnosis method based on big data, characterized in that: The following steps are involved: A obtains brand data and brand history data, and preprocesses the brand data, wherein the brand data includes text data and numerical data; B. constructing a feature extraction model to extract features of the first category from the text data to obtain first feature data; Performing feature extraction of a second category on the numerical data to obtain second feature data; performing cluster analysis on the first feature data and the second feature data to obtain diagnostic data, and performing correlation analysis on the first feature data and the second feature data to obtain evaluation data; C builds a calculation model, divides the brand history data into training data and test data according to the ratio of n:m, trains the model with the training data, tests the model with the test data, inputs the diagnosis data into the brand diagnosis model to obtain a diagnosis result, and inputs the evaluation data into the calculation model to obtain an evaluation result; D. combining the diagnosis result and the evaluation result for analysis, and outputting the analysis result; The method of combining the diagnosis result and the evaluation result for analysis and outputting the analysis result comprises: Invite M experts to score, and combine the data in the diagnosis results and the evaluation results into a set U = {u1, u2, ..., u n }, each expert compares and scores the data in set U in pairs, and the rth expert gives the uth i data and uth j The data is scored Calculate the rth expert score Deviation from the mean: in Score the rth expert Deviation from the mean, is the mean, i,j=1,2,...,n; r=1,2,...,M, and the rth expert score is calculated based on the deviation Weight: Where λ(r) is the score of the rth expert The weight of the score is normalized as follows: where λ * (r) is the rth expert score after normalization The weights are r=1,2,...,M. The scores between each two data in set U are calculated according to the modified weights to obtain the comparison judgment matrix: A=[a ij ] n×n Where A is the comparison judgment matrix, a ij For the score between each two data in set U, according to the comparison judgment matrix, calculate the maximum characteristic root λ of the comparison judgment matrix max : Among them, M i To compare the product of each row of the matrix, M j To compare the product of each column element of the matrix, n is the number of rows and columns, according to the maximum characteristic root λ max Find the corresponding eigenvector: AW′=λ max W′ Where W′ is the largest characteristic root λ max Corresponding feature vectors are processed logarithmically to obtain weight vectors W, where W = logW'; the weight vectors are weight distributions of the importance of each data in the diagnosis results and the evaluation results to the brand score, and the brand score is calculated: Where C is the brand score, u i For the set U = {u1,u2,...,u n }, the i-th data, w i is the weight vector W in u i Corresponding weight distribution, n is the total number of data in set U, a radar chart is drawn according to the diagnosis result, a line chart is drawn according to the evaluation result, and the brand score, radar chart and line chart are output as analysis results.

2. The method for intelligent brand data identification and diagnosis based on big data according to claim 1 is characterized in that: The preprocessing in step A includes performing text cleaning, stop word removal, punctuation removal, noise removal and word form restoration on the text data; and performing data cleaning, missing value processing, outlier removal and error value removal on the numerical data.

3. The method for intelligent brand data identification and diagnosis based on big data according to claim 1 is characterized in that: The method for constructing the feature extraction model comprises: The feature extraction model includes a sentiment tendency feature extraction module, a keyword feature extraction module, a topic feature extraction module, a time domain feature extraction module and a statistical feature extraction module; the sentiment tendency feature extraction module is based on a word embedding model and a HowNet sentiment dictionary, and uses a Senta sentiment analysis system to extract the sentiment tendency of the text data; the keyword feature extraction module extracts keywords from the text data according to a TF-IDF algorithm; the topic feature extraction module extracts the topic distribution of the text data based on an LDA topic model and uses a Gibbs sampling technique; the time domain feature extraction module extracts the time domain features of the numerical data according to a sliding window algorithm; the statistical feature extraction module extracts the statistical features of the numerical data according to a statistical analysis technique.

4. The method for intelligent brand data identification and diagnosis based on big data according to claim 1 is characterized in that: Extracting features of a first category from the text data to obtain first feature data; The method of extracting the second category of features from the numerical data to obtain second feature data includes: The first category includes keyword feature extraction, topic feature extraction and sentiment feature extraction; the second category includes time domain feature extraction and statistical feature extraction; Keyword features are extracted from the text data to obtain keyword features, topic features are extracted from the text data to obtain topic features, and sentiment tendency features are extracted from the text data to obtain sentiment features, and the keyword features, the topic features and the sentiment features are first feature data; time domain features are extracted from the numerical data to obtain time domain features, and statistical features are extracted from the numerical data to obtain statistical features, and the time domain features and the statistical features are second feature data.

5. The method for intelligent brand data identification and diagnosis based on big data according to claim 1 is characterized in that: The method for performing cluster analysis on the first feature data and the second feature data to obtain diagnostic data includes: The key feature data in the first feature data and the second feature data is recorded as x i and x j , calculate x i and x j The similarity between: where s ij For x i and x j The similarity between ij For x i and x j The Euclidean distance between them, k is x i and x j The dimension is k = 1, 2, ..., d, and the similarity matrix S = (s ij ), sort all similarity values ​​in ascending order, use K-means algorithm to classify all similarity values ​​into two categories, and take the category with larger mean as the intra-class similarity set s in , calculate the truncated similarity: where s c is the cutoff similarity, |s in | is the intra-class similarity set s in The number of elements in , calculate the local density: where ρ i is the key feature data x i The local density of , calculate the relative similarity: where δ i is the key feature data x i The relative similarity of argmax is a mathematical operation, which means selecting the key feature data with the largest local density from all key feature data, ρ n is the local density of the nth key feature data, ρ j is the key feature data x j The local density, x n For one of the key feature data, calculate the cluster center weight: where γ i is the key feature data x i The cluster center weights, * is a mathematical operation, which means that after normalization, the cluster center weights are arranged in descending order to obtain the sequence γ1>γ2,......,γ k-1 >γ k >γ k+1 ,......,γ N , for each point γ k Linear regression analysis is performed on the left and right sides of , and two γ value prediction sequences are obtained: and {γ' k+1 ,γ' k+2 ,...,γ' N }, calculate the change point γ t : Where t is the change point γ t The index of E is argmin, which is a mathematical operation. k Value, E k For γ i The sum of squared errors, 2≤k≤N-1, γ i is the key feature data x i The cluster center weight, For γ i The cluster center prediction weight on the left, γ' i For γ i The cluster center prediction weight on the right; select the change point and its previous points as the initial cluster center, and according to the allocation strategy of the DPC algorithm, assign the non-cluster center point to the cluster with the highest density point with the greatest similarity, and get the initial cluster P = {P1, P2, ..., P cp }, define the key feature data x that meets the following conditions n For outliers: Where μ(·) and σ(·) represent the mean and standard deviation of the data, and the set of outliers is denoted as X out , calculate the similarity radius: Where r is the similarity radius, X is the key feature data set, for each key feature data, the key feature data with a similarity greater than or equal to r are called the neighbors of the key feature data, and the K-means algorithm is used to sort the neighbors of the key feature data in descending order and cut them into two sections. The cutoff point is the critical number of neighbors. The key feature data with a neighbor number greater than the critical number of neighbors is the core data, and the set is recorded as X core ; For any two initial clusters P i and P j , calculate the maximum similarity of the two initial clusters, and record the key feature data with the maximum similarity between the two initial clusters as and Right now: in for and The similarity between the two initial clusters P i and P j The maximum similarity, s uv For the two initial clusters P i and P j The similarity between any two key feature data, x u is the initial cluster P i One of the key feature data, x v is the initial cluster P j One of the key feature data in, if any two initial clusters P i and P j satisfy: Then merge the two clusters and update the clustering result. Repeat the above steps until any two clusters do not satisfy equation (1) and obtain the final cluster C = {C1, C2, ..., C nc }, and output the cluster results as diagnostic data.

6. The method for intelligent brand data identification and diagnosis based on big data according to claim 1 is characterized in that: The method of performing correlation analysis on the first feature data and the second feature data to obtain evaluation data includes: The key feature data in the key feature data is recorded as x a and x b , according to the KMedian algorithm, x a and x b Divide into different data clusters, get a grid division value of m rows and n columns, and calculate x based on the obtained grid division value a and x b The joint probability distribution and marginal probability distribution between them, and then we get x a and x b The mutual information is: Where I(x a ;x b ) is x a and x b The mutual information of a ;x b ) is x a and x b The joint probability distribution of a ) is x a The marginal probability distribution, p(x b ) is x b The marginal probability distribution of ; Based on the mutual information, calculate x a and x b The correlation coefficient between: Where MIC(x a ;x b ) is x a and x b The correlation coefficient between them, m is the number of grid columns, n is the number of grid rows, N is the total number of key feature data, I(x a ;x b ) is x a and x b The mutual information between them is obtained, a correlation threshold E=0.5 is set, a correlation coefficient between every two key feature data in the key feature data is calculated, and the key feature data whose correlation coefficient is greater than the correlation threshold is allocated to the evaluation data to obtain the evaluation data.

7. The method for intelligent brand data identification and diagnosis based on big data according to claim 1 is characterized in that: Construct a computational model, divide the brand history data into training data and test data according to the ratio of n:m, use the training data to train the model, and use the test data to test the model. The method includes: The brand history data includes brand index data and brand feature data. The brand index data in the training data is used as the dependent variable and the brand feature data is used as the independent variable to construct a multivariate linear regression model: Y=βX+ε Where Y is the brand index data, Y = {Y1, Y2, Y3}, X is the brand feature data, β is the regression coefficient, β={β0,β1,...,β n }, ε is the random error, ε={ε0,ε1,...,ε n }; Construct the residual sum of squares formula: in is the residual sum of squares, is the predicted regression coefficient, τ is a mathematical symbol, indicating transposition, the predicted regression coefficient in the residual sum of squares is differentiated and set equal to zero to obtain the predicted regression coefficient value: According to the calculated predicted regression coefficient value, the trained multivariate linear regression model is obtained: in is the predicted brand indicator data, X is the brand feature data, is the calculated regression coefficient, ε is random error, ε={ε0,ε1,...,ε n }; Use the test data to test the prediction effect of the model: Where R 2 is the coefficient of determination, R 2 The closer the value is to 1, the better the prediction effect is. Y is the brand index data in the test data, Y = {Y1, Y2, Y3}, is the predicted data obtained through model prediction. is the mean of the brand indicator data in the test data,

Images