Big data information background real-time accurate analysis and processing system

Abstract

The invention provides a big data information background real-time accurate analysis processing system, which relates to the field of data analysis, and adopts a natural language processing technology based on deep learning to pre-process each evaluation data in a set of evaluation data of an object product, introduces reconstruction processing and semantic coding of a product public opinion question template problem, and improves the quality of the product public opinion question template problem. And performing product emotional tendency reasoning on each piece of reconstructed evaluation data, so as to automatically evaluate whether the product public opinion is positive or negative according to semantic significant aggregation representation between each piece of product emotional tendency semantic reasoning coding characteristics obtained by reasoning. The method can better adapt to changes of user expression habits and language environments, and can better understand semantic information and context dependency relationships in texts, so that the accuracy and generalization ability of product sentiment analysis are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and more specifically, to a big data information background real-time precision analysis and processing system. Background Art

[0002] With the rapid development of the Internet and the popularity of social media, the amount of data generated by users has grown exponentially. This data is not only huge in quantity, but also covers a wide range of fields, including posts and comments on social media and user reviews on e-commerce platforms, each of which carries the user's true feelings and opinions. Therefore, using this data for sentiment analysis can help companies better understand and meet user needs, thereby improving the market competitiveness of products and user experience.

[0003] However, traditional methods for sentiment analysis of products often build models in one go, which makes it difficult to dynamically adjust them based on new data. Over time, changes in user expression habits and language environments may cause the model to become invalid. For example, on social media, new popular words and expressions continue to emerge, while traditional models are usually static and cannot be updated and adapted to these changes in a timely manner. This lack of dynamic adjustment capability causes the model to become inaccurate when faced with new data. In addition, traditional methods often rely on keyword matching or simple statistical methods, which cannot effectively capture semantic information and contextual dependencies in text. Keyword matching can only identify isolated words, but cannot understand the meaning of words in a specific context, resulting in inaccurate sentiment analysis results that cannot fully reflect the true evaluation of the product.

[0004] Therefore, a big data information background real-time and precise analysis and processing system is needed to solve the above problems. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a big data information background real-time accurate analysis and processing system.

[0006] According to one aspect of the present application, a big data information background real-time precision analysis and processing system is provided, which includes:

[0007] An evaluation data acquisition processing module, used to collect a set of evaluation data of the target product and perform preprocessing to obtain a set of preprocessed evaluation data;

[0008] An evaluation data reconstruction module, used for adding a product public opinion question template question to the end of each preprocessed evaluation data in the set of preprocessed evaluation data to obtain a set of reconstructed evaluation data;

[0009] A product sentiment tendency reasoning module, used for performing semantic coding on the set of reconstructed evaluation data to obtain a set of semantic coding vectors of the reconstructed evaluation data, and then performing product sentiment tendency reasoning to obtain a set of product sentiment tendency semantic reasoning coding vectors;

[0010] A product sentiment tendency aggregation module, used for performing product sentiment tendency semantic feature aggregation on the set of product sentiment tendency semantic reasoning coding vectors to obtain a significant aggregation representation of product sentiment tendency;

[0011] The product sentiment tendency aggregation module includes: a product static energy factor calculation unit, which is used to calculate the static energy factor of each product sentiment tendency semantic reasoning coding vector in the set of product sentiment tendency semantic reasoning coding vectors to obtain a set of product sentiment tendency semantic static energy factors; an energy distribution significant aggregation unit, which is used to perform significant aggregation based on energy distribution on the set of product sentiment tendency semantic reasoning coding vectors based on the set of product sentiment tendency semantic static energy factors to obtain a significant aggregation representation of the product sentiment tendency;

[0012] The analysis result generation module is used to obtain the analysis result based on the significant aggregation representation of the product sentiment tendency.

[0013] The present application has at least the following technical effects: Compared with the prior art, the big data information background real-time precise analysis and processing system provided by the present application adopts natural language processing technology based on deep learning to pre-process each evaluation data in the set of evaluation data of the benchmark product, introduces reconstruction processing and semantic encoding of product public opinion question template questions, and performs product sentiment tendency inference on each reconstructed evaluation data, thereby automatically evaluating whether the product public opinion is positive or negative based on the semantically significant aggregation representation between the semantic reasoning coding features of each product sentiment tendency obtained by reasoning, and can automatically and gradually adjust with the influx of new data to better adapt to changes in user expression habits and language environment, and at the same time can better understand the semantic information and contextual dependencies in the text, thereby significantly improving the accuracy and generalization ability of product sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0015] Figure 1This is a system block diagram of a big data information background real-time precision analysis and processing system according to an embodiment of the present application.

[0016] Figure 2 The present invention is a data flow diagram of a big data information background real-time precision analysis and processing system according to an embodiment of the present application.

[0017] Figure 3 It is a block diagram of the product sentiment tendency reasoning module in the big data information background real-time and precise analysis and processing system according to the embodiment of the present application.

[0018] Figure 4 It is a block diagram of a product sentiment tendency aggregation module in a big data information background real-time precision analysis and processing system according to an embodiment of the present application.

[0019] Figure 5 This is a block diagram of a product static energy factor calculation unit in a big data information background real-time precision analysis and processing system according to an embodiment of the present application.

[0020] Figure 6 It is a block diagram of the energy distribution significant aggregation unit in the big data information background real-time precise analysis and processing system according to the embodiment of the present application. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] With the rapid development of the Internet and the widespread penetration of social media, the amount of data generated by users is growing at an alarming rate. Not only is the amount of this data huge, but it also covers many areas, such as posts and comments on social media, and user feedback on e-commerce platforms. This data contains the real emotions and opinions of users, providing companies with a valuable opportunity to gain insight into user needs through sentiment analysis, thereby improving the market competitiveness of products and optimizing user experience.

[0023] However, traditional sentiment analysis methods often have limitations. They are usually built based on one-time models and lack the ability to adapt to new data and dynamically adjust. Over time, users' language habits and expressions continue to evolve, such as the endless emergence of new words and expressions on social media, and traditional models often cannot capture these changes in a timely manner, causing the accuracy and effectiveness of the models to decrease over time. In addition, traditional methods often rely on keyword matching or simple statistical analysis, which cannot deeply understand the semantic content and contextual dependencies of the text. Keyword matching can only identify isolated words, but cannot grasp the specific meaning of these words in a specific context, which limits the accuracy of sentiment analysis and makes it unable to fully reflect the user's true evaluation of the product.

[0024] In response to the above technical problems, the technical concept of the present application is to collect a set of evaluation data of the target product, and use natural language processing technology based on deep learning to pre-process each evaluation data, introduce reconstruction processing and semantic encoding of product public opinion question template questions, and perform product sentiment tendency inference on each reconstructed evaluation data, so as to automatically evaluate whether the product public opinion is positive or negative based on the semantically significant aggregation representation between the semantic reasoning coding features of each product sentiment tendency obtained by reasoning. Compared with traditional keyword matching or simple statistical methods, it can automatically and gradually adjust with the influx of new data to better adapt to changes in user expression habits and language environment. At the same time, it can better understand the semantic information and context dependencies in the text, thereby significantly improving the accuracy and generalization ability of product sentiment analysis.

[0025] Figure 1 This is a system block diagram of a big data information background real-time precision analysis and processing system according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the big data information background real-time accurate analysis and processing system according to the embodiment of the present application. Figure 1 and Figure 2As shown, in the big data information background real-time precision analysis and processing system 100, it includes: an evaluation data acquisition processing module 110, which is used to collect a set of evaluation data of the target product and preprocess it to obtain a set of preprocessed evaluation data; an evaluation data reconstruction module 120, which is used to add product public opinion question template questions at the end of each preprocessed evaluation data in the set of preprocessed evaluation data to obtain a set of reconstructed evaluation data; a product sentiment tendency inference module 130, which is used to semantically encode the set of reconstructed evaluation data to obtain a set of reconstructed evaluation data semantic encoding vectors and then perform product sentiment tendency inference to obtain a set of product sentiment tendency semantic inference encoding vectors; a product sentiment tendency aggregation module 140, which is used to aggregate product sentiment tendency semantic features on the set of product sentiment tendency semantic inference encoding vectors to obtain a significantly aggregated representation of product sentiment tendencies; an analysis result generation module 150, which is used to obtain an analysis result based on the significantly aggregated representation of product sentiment tendencies.

[0026] In the embodiment of the present application, the evaluation data acquisition processing module 110 is used to collect the set of evaluation data of the target product and preprocess it to obtain the set of preprocessed evaluation data. Specifically, in the embodiment of the present application, the evaluation data acquisition processing module is used to: collect the set of evaluation data of the target product; preprocess each evaluation data in the set of evaluation data to obtain the set of preprocessed evaluation data.

[0027] Specifically, each evaluation data in the set of evaluation data of the target product usually contains the user's specific feelings, experience and opinion information about the product, which is the core data for judging the direction of product public opinion. By collecting the evaluation data of different users on the target product and performing emotional semantic analysis on it, it can help enterprises grasp user emotions more accurately, and then provide support for enterprise decision-making. Here, the evaluation data of the target product is obtained from the database. As the user evaluation data changes continuously, the evaluation data in the database also changes accordingly. Accordingly, considering that each evaluation data in the set of evaluation data may contain content unrelated to the product, such as advertisements, links or meaningless characters, this information will interfere with sentiment analysis, and there may also be spelling errors or typing errors in the evaluation data when the user enters the evaluation, which easily leads to the model being unable to correctly understand the text. Based on this, in order to improve the quality of the data so that the subsequent sentiment analysis model can more accurately identify the user's true intentions and emotional tendencies, in the technical solution of the present application, each evaluation data in the set of evaluation data is preprocessed to obtain a set of preprocessed evaluation data. In particular, in a specific implementable embodiment of the present application, the step of preprocessing each evaluation data in a set of evaluation data may include: a. using regular expressions to delete links and advertising content as well as special characters and punctuation marks in the evaluation data; b. using spelling correction tools to check and correct the rating data; c. unifying the format of the evaluation data and converting all of them into text form.

[0028] In an embodiment of the present application, the evaluation data reconstruction module 120 is used to add a product public opinion question template question at the end of each preprocessed evaluation data in the set of preprocessed evaluation data to obtain a set of reconstructed evaluation data. Specifically, the set of preprocessed evaluation data expresses the evaluation and feelings of different users on the product. In order to guide and structure the user evaluation data and help the model better understand the context in the comments to more accurately capture the user's emotional tendency, in the technical solution of the present application, a product public opinion question template question is added to the end of each preprocessed evaluation data in the set of preprocessed evaluation data to obtain a set of reconstructed evaluation data. In particular, in a specific implementation method of the embodiment of the present application, the following product public opinion question template questions can be added to the end of the preprocessed evaluation data: "Are you satisfied with the overall quality of the product?", "Which aspects of the product do you think need to be improved?", etc. By adding the question template question, additional contextual information is provided, which helps the model to better understand the background of the user's evaluation, and thus can more accurately capture the user's emotional tendency towards the product.

[0029] In the embodiment of the present application, the product sentiment tendency reasoning module 130 is used to perform semantic coding on the set of reconstructed evaluation data to obtain a set of reconstructed evaluation data semantic coding vectors, and then perform product sentiment tendency reasoning to obtain a set of product sentiment tendency semantic reasoning coding vectors. Specifically, Figure 3 FIG. 1 is a block diagram of a product sentiment tendency reasoning module in a big data information background real-time precision analysis and processing system according to an embodiment of the present application. Figure 3 As shown, the product sentiment tendency reasoning module 130 includes: a reconstructed evaluation data semantic encoding unit 131, which is used to semantically encode each reconstructed evaluation data in the set of reconstructed evaluation data to obtain a set of semantic encoding vectors of the reconstructed evaluation data; a product sentiment tendency semantic feature encoding unit 132, which is used to input each reconstructed evaluation data semantic encoding vector in the set of reconstructed evaluation data semantic encoding vectors into a product sentiment tendency reasoner based on an RNN model to obtain a set of product sentiment tendency semantic reasoning encoding vectors.

[0030] Specifically, in order to further understand and process the semantic meaning between the contexts of each reconstructed evaluation data in the set of reconstructed evaluation data, so as to more accurately judge whether the user's attitude towards the product is positive or negative, in the technical solution of the present application, each reconstructed evaluation data in the set of reconstructed evaluation data is semantically encoded to capture and extract key implicit evaluation semantic information, and obtain a set of reconstructed evaluation data semantic encoding vectors. Subsequently, considering that each reconstructed evaluation data semantic encoding vector in the set of reconstructed evaluation data semantic encoding vectors contains user evaluation semantic information and hidden emotional tendencies in different evaluation data. The RNN model captures the sequential dependency between words in the text, so as to use the internal memory mechanism (such as hidden state) to remember the previous vocabulary information, and use it when processing the current vocabulary, so as to more accurately understand the semantics of the text. Therefore, in order to more accurately capture and quantify the emotional tendencies in user evaluations, in the technical solution of the present application, each reconstructed evaluation data semantic encoding vector in the set of reconstructed evaluation data semantic encoding vectors is input into a product emotional tendency reasoner based on the RNN model to obtain a set of product emotional tendency semantic reasoning encoding vectors. That is, the RNN model can infer the emotional tendency implied in each evaluation data through the semantic information of the evaluation data, so as to better understand the complex emotional expression of the user in each evaluation.

[0031] In the embodiment of the present application, the product sentiment tendency aggregation module 140 is used to aggregate the product sentiment tendency semantic features of the set of product sentiment tendency semantic reasoning encoding vectors to obtain a significant aggregation representation of the product sentiment tendency. Specifically, Figure 4FIG. 1 is a block diagram of a product sentiment tendency aggregation module in a big data information background real-time precision analysis and processing system according to an embodiment of the present application. Figure 4 As shown, the product sentiment tendency aggregation module 140 includes: a product static energy factor calculation unit 141, which is used to calculate the static energy factor of each product sentiment tendency semantic reasoning coding vector in the set of product sentiment tendency semantic reasoning coding vectors to obtain a set of product sentiment tendency semantic static energy factors; an energy distribution significant aggregation unit 142, which is used to perform significant aggregation based on energy distribution on the set of product sentiment tendency semantic reasoning coding vectors based on the set of product sentiment tendency semantic static energy factors to obtain the product sentiment tendency significant aggregation representation.

[0032] Accordingly, considering that the set of product sentiment tendency semantic reasoning coding vectors expresses the semantic feature representation of each user evaluation at the sentiment analysis level, the vector carries a part of the semantic information about the evaluation text and also implies the characteristics of the user's positive or negative sentiment tendency towards the product. In order to further refine and aggregate these sentiment features, thereby obtaining a more significant and concentrated sentiment representation, thereby improving the accuracy and interpretability of product public opinion sentiment analysis, in the technical solution of the present application, the set of product sentiment tendency semantic reasoning coding vectors is aggregated with product sentiment tendency semantic features to obtain a product sentiment tendency significant aggregation representation vector as a product sentiment tendency significant aggregation representation.

[0033] Specifically, first, the static energy factor of each product sentiment tendency semantic reasoning coding vector in the set of product sentiment tendency semantic reasoning coding vectors is calculated to obtain a set of product sentiment tendency semantic static energy factors. Specifically, by calculating the static energy factor, the importance of each sentiment reasoning coding vector in sentiment analysis can be quantified, which helps to identify which feature vectors have a greater impact on the overall sentiment tendency and provides decision support for the subsequent selection of initial clustering centers.

[0034] Then, based on the set of the product sentiment tendency semantic static energy factors, the set of the product sentiment tendency semantic reasoning coding vectors is significantly aggregated based on energy distribution to obtain the significant aggregation representation of the product sentiment tendency, which can find out the features that occupy an important position in the sentiment distribution, so as to better reflect the overall sentiment tendency. Specifically, the product sentiment tendency semantic reasoning coding vector corresponding to the largest product sentiment tendency semantic static energy factor is selected from the set of the product sentiment tendency semantic static energy factors as the initial center vector of the product sentiment tendency semantic reasoning clustering. In this way, a most representative initial center point can be determined, so that the final clustering result is more consistent with the actual sentiment distribution, and the ability to capture and understand real features is enhanced. Then, the dynamic aggregation energy factor of each reasoning coding vector in the set of the product sentiment tendency semantic reasoning coding vector is calculated to obtain a set of dynamic aggregation energy factors of product sentiment tendency semantic reasoning. In particular, the dynamic aggregation energy factor is calculated based on the static energy factors of the initial center vector and each product sentiment tendency semantic reasoning coding vector, and the spatial span between the initial center vector and each product sentiment tendency semantic reasoning coding vector. In this way, the obtained dynamic aggregation energy factor not only takes into account the proximity of the position, but also reflects the difference in the importance of the intrinsic attributes of each point, so as to more accurately reflect the interaction between different samples. Then, in order to better focus on the key features that best represent the emotional tendency, each dynamic aggregation energy factor is input into the gated mask unit for adaptive adjustment and attention of the weight, so as to strengthen the attention to the significant features and reduce the influence of irrelevant information, so as to obtain a set of dynamic aggregation weight factors of product emotional tendency semantic reasoning. Finally, based on the mixed weights in the set of weight factors, the set of product emotional tendency semantic reasoning encoding vectors is weighted and aggregated to generate the final product emotional tendency significant aggregation representation vector. In this way, the importance of each emotional reasoning encoding vector can be integrated to generate a more accurate and representative comprehensive emotional representation, which can provide assistance for subsequent product emotional analysis.

[0035] More specifically, Figure 5 FIG. 1 is a block diagram of a product static energy factor calculation unit in a big data information background real-time precision analysis and processing system according to an embodiment of the present application. Figure 5As shown, the product static energy factor calculation unit 141 includes: a product emotion tendency semantic reasoning mean variance calculation subunit 1411, which is used to calculate the mean and variance of the product emotion tendency semantic reasoning coding vector to obtain the product emotion tendency semantic reasoning mean and product emotion tendency semantic reasoning variance; a product emotion tendency semantic reasoning difference modulation vector generation subunit 1412, which is used to subtract the product emotion tendency semantic reasoning coding vector from the product emotion tendency semantic reasoning mean by position, and calculate the fourth power of each position of the subtracted feature vector to obtain the product emotion tendency semantic reasoning difference modulation vector; a product emotion tendency semantic reasoning expected value calculation subunit 1413, which is used to calculate the expected value of the product emotion tendency semantic reasoning difference modulation vector to obtain the product emotion tendency semantic reasoning expected value; a product emotion tendency semantic static energy factor generation subunit 1414, which is used to divide the product emotion tendency semantic reasoning expected value by the square of the product emotion tendency semantic reasoning variance, and input the divided value into a sigmoid function to obtain the product emotion tendency semantic static energy factor.

[0036] More specifically, Figure 6 FIG. 1 is a block diagram of an energy distribution significant aggregation unit in a big data information background real-time accurate analysis and processing system according to an embodiment of the present application. Figure 6As shown, the energy distribution significant aggregation unit 142 includes: a product emotion tendency semantic reasoning clustering initial center vector determination subunit 1421, which is used to select the product emotion tendency semantic reasoning coding vector corresponding to the largest product emotion tendency semantic static energy factor in the set of product emotion tendency semantic static energy factors as the product emotion tendency semantic reasoning clustering initial center vector; a product emotion tendency dynamic aggregation energy factor calculation subunit 1422, which is used to calculate the product emotion tendency dynamic aggregation energy factor based on the spatial span between each product emotion tendency semantic reasoning coding vector in the set of product emotion tendency semantic reasoning coding vector and the product emotion tendency semantic reasoning clustering initial center vector, as well as the static energy factor of each product emotion tendency semantic reasoning coding vector and the product emotion tendency semantic reasoning clustering initial center vector. The static energy factor of the quantity is used to calculate the dynamic aggregation energy factor of each product emotion tendency semantic reasoning coding vector in the set of the product emotion tendency semantic reasoning coding vector to obtain a set of product emotion tendency semantic reasoning dynamic aggregation energy factors; the dynamic aggregation energy factor gating mask subunit 1423 is used to input the set of product emotion tendency semantic reasoning dynamic aggregation energy factors into the gating mask unit to obtain a set of product emotion tendency semantic reasoning dynamic aggregation weight factors; the product emotion tendency significant aggregation representation generation subunit 1424 is used to calculate the weighted sum of the set of product emotion tendency semantic reasoning coding vectors with the set of product emotion tendency semantic reasoning dynamic aggregation weight factors to obtain the product emotion tendency semantic reasoning coding vector as the product emotion tendency significant aggregation representation.

[0037] More specifically, in an embodiment of the present application, the product sentiment tendency dynamic aggregation energy factor calculation subunit is used to: multiply the static energy factor of the product sentiment tendency semantic reasoning coding vector and the static energy factor of the product sentiment tendency semantic reasoning clustering initial center vector by a first weighting parameter to obtain a first product sentiment tendency semantic reasoning dynamic aggregation energy factor; multiply the square of the spatial span between the product sentiment tendency semantic reasoning coding vector and the product sentiment tendency semantic reasoning clustering initial center vector by a second weighting parameter to obtain a second product sentiment tendency semantic reasoning dynamic aggregation energy factor; divide the first product sentiment tendency semantic reasoning dynamic aggregation energy factor by the second product sentiment tendency semantic reasoning dynamic aggregation energy factor to obtain the product sentiment tendency semantic reasoning dynamic aggregation energy factor.

[0038] More specifically, in an embodiment of the present application, the dynamic aggregation energy factor gating mask subunit includes: a product emotion dynamic aggregation energy factor normalization secondary subunit, which is used to normalize the set of product emotion tendency semantic reasoning dynamic aggregation energy factors to obtain a set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors; a normalized product emotion dynamic aggregation energy factor masking secondary subunit, which is used to mask the set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors to obtain a set of product emotion tendency semantic reasoning dynamic aggregation weight factors.

[0039] More specifically, in an embodiment of the present application, the product emotion dynamic aggregation energy factor normalization secondary sub-unit is used to: use the sigmoid function to normalize the set of product emotion tendency semantic reasoning dynamic aggregation energy factors to obtain the set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors.

[0040] More specifically, in an embodiment of the present application, the normalized product emotion dynamic aggregation energy factor mask secondary sub-unit is used to: in response to each normalized product emotion tendency semantic reasoning dynamic aggregation energy factor in the set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors being greater than a predetermined threshold, set the normalized product emotion tendency semantic reasoning dynamic aggregation energy factor to the original value, and set the rest to zero, so as to obtain a set of product emotion tendency semantic reasoning dynamic aggregation weight factors.

[0041] In the embodiment of the present application, specifically, the product sentiment tendency aggregation module is used to: perform product sentiment tendency semantic feature aggregation processing on the set of product sentiment tendency semantic reasoning coding vectors according to the following formula; wherein the formula is:

[0042] X={x1,x2,...,x i ,...,x n}

[0043]

[0044] x c =x m

[0045]

[0046] w si =mask(w i )

[0047]

[0048] Where X is the set of semantic reasoning encoding vectors of the product sentiment tendency, x1, x2, ..., x i ,...,x n are respectively the 1st, 2nd, ..., ith, ..., nth product sentiment tendency semantic reasoning coding vectors in the set of product sentiment tendency semantic reasoning coding vectors, is the eigenvalue of each position in the semantic reasoning encoding vector of the sentiment tendency of the i-th product, E(·) is the expected value, μ i and σ i 2 X i The mean and variance of , sigmoid(·) is the sigmoid function, is x i The corresponding product sentiment tendency semantic static energy factor, argmax j (·) is the j value corresponding to the maximum value, m is the maximum matching value, x c is the initial center vector of the semantic reasoning clustering of product sentiment tendency, is x m The corresponding product sentiment tendency semantic static energy factor, Count(x i →x m ) represents x i With x m The spatial span between them, γ and δ are weighting parameters, For X i The corresponding product sentiment tendency semantic reasoning dynamic aggregation energy factor, W i Yes X i The corresponding normalized product sentiment tendency semantic reasoning dynamic aggregation energy factor, mask(·) is the masking process, θ is the predetermined threshold, w si is x i The corresponding product sentiment tendency semantic reasoning dynamic aggregation weight factor, n is the number of vectors in the set of product sentiment tendency semantic reasoning encoding vectors, x < It is the semantic reasoning encoding vector of the sentiment tendency of the product.

[0049] In an embodiment of the present application, the analysis result generation module 150 is used to obtain an analysis result based on the significant aggregation representation of the product sentiment tendency. Specifically, in an embodiment of the present application, the analysis result generation module is used to: input the significant aggregation representation vector of the product sentiment tendency into the real-time analysis module of product public opinion based on the classifier to obtain the analysis result, and the analysis result is used to indicate that the product public opinion is positive or the product public opinion is negative. Specifically, the significant aggregation representation of the product sentiment tendency obtained by significant aggregation of the set of product sentiment tendency semantic reasoning encoding vectors is classified and processed to automatically evaluate whether the product public opinion is positive or negative. Compared with traditional keyword matching or simple statistical methods, it can be automatically adjusted step by step with the influx of new data to better adapt to changes in user expression habits and language environment, and at the same time, it can better understand the semantic information and context dependencies in the text, thereby significantly improving the accuracy and generalization ability of product sentiment analysis.

[0050] In particular, the present application takes into account that each product sentiment tendency semantic reasoning coding vector in the set of product sentiment tendency semantic reasoning coding vectors respectively represents the product sentiment tendency semantic reasoning coding feature representation obtained by text semantic feature encoding and text semantic feature decoding of each preprocessed evaluation data in the set of preprocessed evaluation data. When the set of product sentiment tendency semantic reasoning coding vectors is subjected to product sentiment tendency semantic feature aggregation, the differences in text semantic encoding features and text semantic decoding of each preprocessed evaluation data in the set of preprocessed evaluation data will make the product sentiment tendency significantly aggregated representation vector sentiment semantic fusion representation have a complex aggregation spatial structure, thereby expecting to improve its classification regression convergence and generalization effects under the complex aggregation spatial structure.

[0051] Preferably, in an example of the present application, the product sentiment tendency significant aggregation representation vector is passed through a classifier-based product public opinion real-time analysis module to obtain an analysis result, including: calculating the sum of the absolute values ​​of each eigenvalue of the product sentiment tendency significant aggregation representation vector to obtain a first product sentiment tendency significant aggregation representation space structure value, and calculating the square root of the sum of the squares of each eigenvalue of the product sentiment tendency significant aggregation representation vector to obtain a second product sentiment tendency significant aggregation representation space structure value; multiplying each eigenvalue of the product sentiment tendency significant aggregation representation vector with the first product sentiment tendency significant aggregation representation space structure value and the second product sentiment tendency significant aggregation representation space structure value to obtain a first product sentiment tendency significant aggregation representation structure reference value and a second product sentiment tendency significant aggregation representation structure reference value corresponding to each eigenvalue; multiplying each eigenvalue of the product sentiment tendency significant aggregation representation vector with the length of the product sentiment tendency significant aggregation representation vector and the square root of the length to obtain a first product sentiment tendency significant aggregation representation structure reference value and a second product sentiment tendency significant aggregation representation structure reference value corresponding to each eigenvalue The corresponding first product sentiment tendency significant aggregation representation scale transformation value and the second product sentiment tendency significant aggregation representation scale transformation value; the first product sentiment tendency significant aggregation representation structure reference value is divided by the difference between the first product sentiment tendency significant aggregation representation space structure value and the first product sentiment tendency significant aggregation representation scale transformation value to obtain the first product sentiment tendency significant aggregation representation transformation adjustment value; the second product sentiment tendency significant aggregation representation structure reference value is divided by the difference between the second product sentiment tendency significant aggregation representation space structure value and the second product sentiment tendency significant aggregation representation scale transformation value to obtain the second product sentiment tendency significant aggregation representation transformation adjustment value; the weighted sum of the first product sentiment tendency significant aggregation representation transformation adjustment value and the second product sentiment tendency significant aggregation representation transformation adjustment value is calculated to obtain each eigenvalue of the optimized product sentiment tendency significant aggregation representation vector; the optimized product sentiment tendency significant aggregation representation vector is passed through the classifier-based product public opinion real-time analysis module to obtain the analysis result.

[0052] The optimized representation of the significant aggregation representation vector of the product sentiment tendency is:

[0053]

[0054] v 1i =(α×v i ) / (α-L×v i )

[0055]

[0056] v i ∈V∈R 1×L

[0057] v 1i ∈V1∈R 1×L

[0058] v 2i ∈V2∈R 1×L

[0059] Where V is the significant aggregate representation vector of the product sentiment tendency, L is the number of eigenvalues ​​in the significant aggregate representation vector of the product sentiment tendency, R is the real number domain, v i is the eigenvalue of the i-th position in the significant aggregation representation vector of the product sentiment tendency, α is the spatial structure value of the significant aggregation representation of the first product sentiment tendency, β is the spatial structure value of the significant aggregation representation of the second product sentiment tendency, v 1i represents the significant aggregation of the sentiment tendency of the first product, represents the significant aggregation of the sentiment tendency of the first product at the i-th position in the transformation adjustment vector, represents the transformation adjustment value, v 2i is the second product sentiment tendency significant aggregation representation transformation adjustment value at the ith position in the second product sentiment tendency significant aggregation representation transformation adjustment vector, V1 is the first product sentiment tendency significant aggregation representation transformation adjustment vector, Represents point-by-point addition, ω represents weighted hyperparameter, ⊙ represents point-by-point multiplication, V2 represents the second product sentiment tendency significant aggregation representation transformation adjustment vector, and V′ represents the optimized product sentiment tendency significant aggregation representation vector.

[0060] Specifically, in the preferred example, for the spatial structure information of the feature set of the product sentiment tendency significant aggregation representation vector in the high-dimensional space, the scale-based frame transformation of each eigenvalue of the product sentiment tendency significant aggregation representation vector is performed by taking the class norm space structured representation of the product sentiment tendency significant aggregation representation vector as the reference window, and the spatial structure-based frame attention weight adjustment of each eigenvalue of the product sentiment tendency significant aggregation representation vector is realized to ensure the invariance of the spatial transformation (translation, scaling and rotation) of the product sentiment tendency significant aggregation representation vector under the interaction of feature space, thereby improving the convergence and generalization effect of classification regression of the feature set of the product sentiment tendency significant aggregation representation vector under the complex spatial structure representation, and improving the accuracy of the analysis result obtained by the classifier-based product public opinion real-time analysis module. Compared with traditional keyword matching or simple statistical methods, it can be automatically and gradually adjusted with the influx of new data to better adapt to changes in user expression habits and language environment, and at the same time, it can better understand the semantic information and context dependencies in the text, thereby significantly improving the accuracy and generalization ability of product sentiment analysis.

[0061] To summarize, a big data information background real-time precise analysis and processing system 100 based on the embodiment of the present application is explained, which adopts natural language processing technology based on deep learning to pre-process each evaluation data in the set of evaluation data of the benchmark product, introduces reconstruction processing and semantic encoding of product public opinion question template questions, and performs product sentiment tendency inference on each reconstructed evaluation data, so as to automatically evaluate whether the product public opinion is positive or negative based on the semantically significant aggregation representation between the semantic reasoning coding features of each product sentiment tendency obtained by reasoning, and can automatically and gradually adjust with the influx of new data to better adapt to changes in user expression habits and language environment, and at the same time can better understand the semantic information and contextual dependencies in the text, thereby significantly improving the accuracy and generalization ability of product sentiment analysis.

[0062] As described above, the big data information background real-time precision analysis and processing system 100 according to the embodiment of the present application can be implemented in various terminal devices, such as a server for big data information background real-time precision analysis and processing. In one example, the big data information background real-time precision analysis and processing system 100 according to the embodiment of the present application can be integrated into the terminal device as a software module and / or a hardware module. For example, the big data information background real-time precision analysis and processing system 100 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the big data information background real-time precision analysis and processing system 100 can also be one of the many hardware modules of the terminal device.

[0063] Alternatively, in another example, the big data information background real-time precision analysis and processing system 100 and the terminal device may also be separate devices, and the big data information background real-time precision analysis and processing system 100 may be connected to the terminal device via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

Claims

1. A real-time and accurate analysis and processing system for big data information background, characterized by: include: An evaluation data acquisition processing module, used to collect a set of evaluation data of the target product and perform preprocessing to obtain a set of preprocessed evaluation data; An evaluation data reconstruction module, used for adding a product public opinion question template question to the end of each preprocessed evaluation data in the set of preprocessed evaluation data to obtain a set of reconstructed evaluation data; A product sentiment tendency reasoning module, used for performing semantic coding on the set of reconstructed evaluation data to obtain a set of semantic coding vectors of the reconstructed evaluation data, and then performing product sentiment tendency reasoning to obtain a set of product sentiment tendency semantic reasoning coding vectors; A product sentiment tendency aggregation module, used for performing product sentiment tendency semantic feature aggregation on the set of product sentiment tendency semantic reasoning coding vectors to obtain a significant aggregation representation of product sentiment tendency; The product sentiment tendency aggregation module includes: a product static energy factor calculation unit, which is used to calculate the static energy factor of each product sentiment tendency semantic reasoning coding vector in the set of product sentiment tendency semantic reasoning coding vectors to obtain a set of product sentiment tendency semantic static energy factors; an energy distribution significant aggregation unit, which is used to perform significant aggregation based on energy distribution on the set of product sentiment tendency semantic reasoning coding vectors based on the set of product sentiment tendency semantic static energy factors to obtain a significant aggregation representation of the product sentiment tendency; The analysis result generation module is used to obtain the analysis result based on the significant aggregation representation of the product sentiment tendency.

2. The big data information background real-time accurate analysis and processing system according to claim 1 is characterized in that: The evaluation data acquisition processing module is used to: Collecting a set of evaluation data of the target product; Each evaluation data in the set of evaluation data is preprocessed to obtain the set of preprocessed evaluation data.

3. The big data information background real-time accurate analysis and processing system according to claim 2 is characterized in that: The product sentiment tendency reasoning module includes: A reconstructed evaluation data semantic coding unit, used for performing semantic coding on each reconstructed evaluation data in the set of reconstructed evaluation data to obtain a set of semantic coding vectors of the reconstructed evaluation data; The product sentiment tendency semantic feature encoding unit is used to input each reconstructed evaluation data semantic encoding vector in the set of reconstructed evaluation data semantic encoding vectors into the product sentiment tendency reasoner based on the RNN model to obtain the set of product sentiment tendency semantic reasoning encoding vectors.

4. The big data information background real-time accurate analysis and processing system according to claim 3 is characterized in that: The product static energy factor calculation unit comprises: A product sentiment tendency semantic reasoning mean variance calculation subunit, used to calculate the mean and variance of the product sentiment tendency semantic reasoning encoding vector to obtain the product sentiment tendency semantic reasoning mean and the product sentiment tendency semantic reasoning variance; A product sentiment tendency semantic reasoning difference modulation vector generating subunit, used for subtracting the product sentiment tendency semantic reasoning coding vector from the product sentiment tendency semantic reasoning mean by position, and calculating the fourth power of each position of the subtracted feature vector to obtain the product sentiment tendency semantic reasoning difference modulation vector; A product sentiment tendency semantic reasoning expected value calculation subunit, used to calculate the expected value of the product sentiment tendency semantic reasoning difference modulation vector to obtain the product sentiment tendency semantic reasoning expected value; The product sentiment tendency semantic static energy factor generating subunit is used to divide the expected value of the product sentiment tendency semantic reasoning by the square of the product sentiment tendency semantic reasoning variance, and input the value obtained by the division into the sigmoid function to obtain the product sentiment tendency semantic static energy factor.

5. The big data information background real-time accurate analysis and processing system according to claim 4 is characterized in that: The energy distribution significant polymerization unit comprises: A product sentiment tendency semantic reasoning clustering initial center vector determination subunit, used to select the product sentiment tendency semantic reasoning encoding vector corresponding to the largest product sentiment tendency semantic static energy factor in the set of product sentiment tendency semantic static energy factors as the product sentiment tendency semantic reasoning clustering initial center vector; A product sentiment tendency dynamic aggregation energy factor calculation subunit is used to calculate the dynamic aggregation energy factor of each product sentiment tendency semantic reasoning coding vector in the set of product sentiment tendency semantic reasoning coding vectors based on the spatial span between each product sentiment tendency semantic reasoning coding vector in the set of product sentiment tendency semantic reasoning coding vectors and the initial center vector of the product sentiment tendency semantic reasoning cluster, as well as the static energy factor of each product sentiment tendency semantic reasoning coding vector and the static energy factor of the initial center vector of the product sentiment tendency semantic reasoning cluster, so as to obtain a set of product sentiment tendency semantic reasoning dynamic aggregation energy factors; A dynamic aggregation energy factor gated mask subunit, used for inputting the set of dynamic aggregation energy factors of product sentiment tendency semantic reasoning into a gated mask unit to obtain a set of dynamic aggregation weight factors of product sentiment tendency semantic reasoning; The product sentiment tendency significant aggregate representation generating subunit is used to calculate the weighted sum of the set of product sentiment tendency semantic reasoning encoding vectors with the set of product sentiment tendency semantic reasoning dynamic aggregation weight factors to obtain the product sentiment tendency semantic reasoning encoding vector as the product sentiment tendency significant aggregate representation.

6. The big data information background real-time accurate analysis and processing system according to claim 5 is characterized in that: The product sentiment tendency dynamic aggregation energy factor calculation subunit is used to: Multiplying the static energy factor of the product sentiment tendency semantic reasoning encoding vector and the static energy factor of the product sentiment tendency semantic reasoning clustering initial center vector by the first weighting parameter to obtain a first product sentiment tendency semantic reasoning dynamic aggregation energy factor; Multiplying the square of the spatial span between the product sentiment tendency semantic reasoning encoding vector and the product sentiment tendency semantic reasoning clustering initial center vector by a second weighting parameter to obtain a second product sentiment tendency semantic reasoning dynamic aggregation energy factor; The first product sentiment tendency semantic reasoning dynamic aggregation energy factor is divided by the second product sentiment tendency semantic reasoning dynamic aggregation energy factor to obtain the product sentiment tendency semantic reasoning dynamic aggregation energy factor.

7. The big data information background real-time accurate analysis and processing system according to claim 6 is characterized in that: The dynamic aggregation energy factor gated mask subunit comprises: A product emotion dynamic aggregation energy factor normalization secondary subunit is used to normalize the set of product emotion tendency semantic reasoning dynamic aggregation energy factors to obtain a set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors; The normalized product emotion dynamic aggregation energy factor masking secondary sub-unit is used to mask the set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors to obtain the set of product emotion tendency semantic reasoning dynamic aggregation weight factors.

8. The big data information background real-time accurate analysis and processing system according to claim 7 is characterized in that: The product emotion dynamic aggregation energy factor normalization secondary subunit is used to: use the sigmoid function to normalize the set of product emotion tendency semantic reasoning dynamic aggregation energy factors to obtain the set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors.

9. The big data information background real-time accurate analysis and processing system according to claim 8 is characterized in that: The normalized product emotion dynamic aggregation energy factor mask secondary sub-unit is used to: in response to each normalized product emotion tendency semantic reasoning dynamic aggregation energy factor in the set of normalized product emotion tendency semantic reasoning dynamic aggregation energy factors being greater than a predetermined threshold, set the normalized product emotion tendency semantic reasoning dynamic aggregation energy factor to the original value, and set the rest to zero, so as to obtain a set of product emotion tendency semantic reasoning dynamic aggregation weight factors.

10. The big data information background real-time accurate analysis and processing system according to claim 9 is characterized in that: The analysis result generation module is used to: input the product sentiment tendency significant aggregation representation vector into the classifier-based product public opinion real-time analysis module to obtain the analysis result, and the analysis result is used to indicate that the product public opinion is positive or negative.

Images