Customer demand identification method and system based on online comment data

Abstract

The invention discloses a customer demand identification method and system based on online comment data, and relates to the technical field of data mining and natural language processing, and the method comprises the steps: obtaining the online comment data, carrying out the standardization processing, and carrying out the denoising processing of an online comment data set after the standardization processing; extracting features of the de-noised online comment data, and constructing a support vector machine (SVM) model to perform customer demand identification; and constructing a visual interface to display a customer demand identification result in real time, carrying out secure storage on the online comment data, and implementing access control. According to the method, the online comment data is acquired and standardized, the standardized online comment data set is de-noised, the de-noised online comment data features are extracted, and the SVM model is constructed for customer demand identification, so that the accuracy and efficiency of data processing are improved; the understanding ability of user demands is enhanced, and the accuracy and robustness of customer demand identification are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining and natural language processing, and in particular to a method and system for identifying customer needs based on online review data. Background Art

[0002] In today's digital age, online review data has gradually become an important resource for understanding customer needs and behaviors. With the rapid development of Internet technology, the popularity of e-commerce platforms and social media has enabled users to express their satisfaction, suggestions and dissatisfaction with products and services through online reviews. These review data are directly derived from users' real experience and have high value. Faced with massive and unstructured review data, traditional customer demand identification methods seem to be unable to cope with it. In order to effectively mine valuable customer needs from these reviews, researchers have proposed a variety of analysis methods based on natural language processing (NLP), machine learning (ML) and big data technology. These methods can automatically process and analyze review data, identify hidden patterns and trends, and thus help companies better understand market demand and optimize products and services.

[0003] Existing technologies still have many shortcomings in identifying customer needs based on online review data. Review data usually contains a lot of noise information, such as repeated comments, irrelevant content, and non-text elements. These noise information will interfere with the accuracy of the model and cause inaccurate recognition results. Review data is often heterogeneous, and the review data formats and languages ​​from different platforms are different. Existing technologies generally rely on simple word frequency statistics methods when extracting key demand features from reviews, ignoring the potential semantic information and contextual relevance in the reviews, and are unable to fully reflect the real needs of customers. These shortcomings lead to limitations in the accuracy and reliability of current technologies in identifying customer needs. Summary of the invention

[0004] In view of the problems existing in the above-mentioned existing customer demand identification method and system based on online review data, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that comment data usually contains a large amount of noise information, such as repeated comments, irrelevant content and non-text elements. These noise information will interfere with the accuracy of the model and cause inaccurate recognition results. Comment data are often heterogeneous, and comment data from different platforms have different formats and languages. When extracting key demand features in comments, the existing technology generally relies on simple word frequency statistics methods, ignoring the potential semantic information and contextual relevance in the comments, and cannot fully reflect the real needs of customers. These shortcomings lead to limitations in the accuracy and reliability of current technology in customer demand identification.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a customer demand identification method based on online review data, which includes obtaining online review data and performing standardization processing, and denoising the standardized online review data set; extracting features of the denoised online review data, and constructing a support vector machine (SVM) model to identify customer needs; constructing a visual interface to display the results of customer demand identification in real time, and securely storing the online review data and implementing access control.

[0007] As a preferred solution of the method for identifying customer needs based on online review data of the present invention, wherein: the obtaining of online review data and performing standardized processing refers to using a crawler tool to obtain customer online review data from an online platform to form an original data set, and the online platform includes an e-commerce and social media platform;

[0008] Use the online translation API to unify the language of the original data set and perform standardization processing, including using the Jieba word segmentation tool to segment the customer online review data set to remove stop words, using the text deduplication algorithm to detect and delete duplicate comments in the customer online review data set, and using regular expression matching to remove HTML tags, emoticons and non-text content from the customer online review data set to form a unified online review data set.

[0009] As a preferred solution of the customer demand identification method based on online review data of the present invention, wherein: the standardized online review data set is segmented, and each review A is segmented using the pre-trained word embedding model Word2Vec. i The jth word in is converted into the corresponding word vector A ij , and calculate for each comment A i The mean value B i , standard deviation C i and variance F i ;

[0010] Calculate the number of words in comment A i TF-IDF values ​​in the comment A and sort them from large to small, and select the first o as comment A. i The keywords form the keyword set K i ={t1,t2,…,t o}, where t o For the oth TF-IDF value, calculate the keyword set K i The mean μ k and standard deviation σ k And the deviation P relative to the overall keywords of the online review dataset i ;

[0011] Use VADER sentiment analyzer to analyze each comment Ai Perform sentiment analysis to get sentiment score S i ;

[0012] Construct a noise recognition function, the formula is:

[0013]

[0014] Where E i is the noise identification value of the i-th comment;

[0015] Based on the noise identification value E i and the review variance F i Calculate the noise reduction weight G for each comment i , the formula is:

[0016]

[0017] Use noise to weaken the weight G i Comment A i Adjusted to get the weakened comment A i ';

[0018] Calculate the global mean B and standard deviation σ of all the weakened comment data, and use the standard normal distribution integral to calculate the weakened comment A i 'Calculate the standard normal distribution integral value, the formula is:

[0019]

[0020] Among them, H i Get the standard normal distribution integral value for the i-th comment in the interval [c, d], where c and d are the lower and upper limits of the integral respectively;

[0021] Set the judgment threshold T, and set the standard normal distribution integral value H greater than the judgment threshold T i , as the normally distributed data A i ", delete the standard normal distribution integral value H that is less than and equal to the judgment threshold T i ;

[0022] Calculate the normal distribution data A i The mean μ k and variance σ k , calculate the time correlation adjustment coefficient N(k) and the time correlation noise characteristic h of each time period k k ;

[0023] Calculate the time-correlated noise characteristics h for all time periods k The mean μ h , process the noise in each time period and obtain the processed noise impact result f(h k), the formula is:

[0024]

[0025] Where β is the adjustment parameter;

[0026] Combined with the time correlation adjustment coefficient N(k) and the processed noise impact result f(h k ), for the normally distributed data A in the overall time period i " is processed to obtain the processed data set I k , the formula is:

[0027]

[0028] Where M is the total number of time periods, k is the kth time period, and T is the total duration of the time period;

[0029] Construct the denoising formula for online review data:

[0030]

[0031] Where J is the denoised online review dataset, and a and b are the upper and lower limits of the normal distribution integral, respectively.

[0032] As a preferred solution of the customer demand identification method based on online review data of the present invention, wherein: the extracting of denoised online review data features refers to extracting denoised online review data from a denoised online review data set J;

[0033] Use Hugging Face's transformers library to load the pre-trained BERT model and input the denoised online review data into the pre-trained BERT model to obtain a high-dimensional feature vector.

[0034] The high-dimensional vector output by the pre-trained BERT is standardized, the covariance matrix of the standardized high-dimensional feature vector is calculated, the covariance matrix is ​​eigen-decomposed, the eigenvalues ​​and corresponding eigenvectors are solved, the corresponding eigenvectors are sorted from large to small, and a selection threshold e is set based on the historical corresponding eigenvectors. The eigenvectors greater than the selection threshold e are selected to generate the online comment data feature vector, and the eigenvectors less than and equal to the threshold e are deleted.

[0035] As a preferred solution of the method for identifying customer needs based on online review data described in the present invention, wherein: the construction of a support vector machine (SVM) model for identifying customer needs refers to collecting annotated historical customer online review data for denoising and feature extraction to generate a training set;

[0036] Construct a support vector machine (SVM) model, including input layer, hidden layer, decision layer and output layer;

[0037] Set the input layer to the feature vector of online review data;

[0038] Use the training set to train the support vector machine (SVM) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters;

[0039] The feature vector of online review data is input into the trained support vector machine (SVM) model to obtain the classification results of customer needs.

[0040] As a preferred solution of the customer demand identification method based on online review data described in the present invention, wherein: the construction of a visualization interface to display the results of customer demand identification in real time refers to using the Dash framework of the visualization tool Python to build a visualization interface, and using a data visualization tool to display the classification results of customer needs in real time, allowing users who have passed real-name verification to view it.

[0041] As a preferred solution of the method for identifying customer needs based on online review data described in the present invention, wherein: the secure storage of online review data and implementation of access control refers to storing the classification results and timestamps of customer needs of the online review data in the database, and setting security access measures. The database backs up the stored data in the cloud, and regularly performs integrity checks on the stored data and backup data. After the test is completed, an integrity check record is generated and synchronously stored in the database.

[0042] Another object of the present invention is to provide a customer demand identification system based on online review data, which comprises:

[0043] An acquisition processing module is used to acquire online comment data and perform standardization processing, and perform denoising processing on the standardized online comment data set;

[0044] Extraction building module, used to extract the features of online review data after denoising and build a support vector machine (SVM) model to identify customer needs;

[0045] Build a storage module to construct a visual interface to display the results of customer demand identification in real time, securely store online review data and implement access control.

[0046] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned customer demand identification method based on online review data when executing the computer program.

[0047] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for identifying customer needs based on online review data.

[0048] The beneficial effects of the present invention are as follows: the present invention obtains online review data and performs standardization processing, denoises the standardized online review data set, extracts features of the denoised online review data, and constructs a support vector machine (SVM) model to identify customer needs, thereby improving the accuracy and efficiency of data processing, enhancing the ability to understand user needs, and improving the accuracy and robustness of customer demand identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0050] Figure 1 Flowchart of the customer needs identification method based on online review data.

[0051] Figure 2 Schematic diagram of the structure of the customer demand identification system based on online review data. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0053] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0054] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0055] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and provides a method for identifying customer needs based on online review data. The method for identifying customer needs based on online review data includes:

[0056] S1, obtaining online review data and performing standardization processing, and performing denoising processing on the standardized online review data set;

[0057] Specifically, obtaining online review data and performing standardized processing refers to using a crawler tool to obtain customer online review data from online platforms to form an original data set, wherein the online platforms include e-commerce and social media platforms;

[0058] Use the online translation API to unify the language of the original data set and perform standardization processing, including using the Jieba word segmentation tool to segment the customer online review data set to remove stop words, using the text deduplication algorithm to detect and delete duplicate comments in the customer online review data set, and using regular expression matching to remove HTML tags, emoticons and non-text content from the customer online review data set to form a unified online review data set.

[0059] By unifying all comment data into the same language, data inconsistency problems caused by language differences are eliminated, ensuring that subsequent analysis can be performed in the same context, making feature extraction, sentiment analysis and other links more accurate and effective. By removing these stop words, the words retained in the data set are more informative, thereby improving the effectiveness of subsequent model training. Text deduplication not only improves data quality, but also greatly reduces the waste of computing resources. Regular expression matching is used to clean the online comment data set, and irrelevant information in the data set is removed, ensuring the validity and relevance of the remaining data.

[0060] Furthermore, the standardized online review dataset is subjected to denoising, which means that the standardized online review dataset is subjected to word segmentation, and each comment A in the customer online review text dataset after word segmentation is embedded using the pre-trained word embedding model Word2Vec. i The jth word in is converted into the corresponding word vector A ij , and calculate for each comment A i The mean value B i , standard deviation C i and variance F i , the formula is:

[0061]

[0062] Where n is the comment A i The number of words in

[0063] Count each comment A i The number of times each word t appears in A is obtained by the word frequency TF(t,A i ), count the number of documents DF(t) in which word t appears in the online review dataset, and calculate the inverse document frequency IDF(t), the formula is:

[0064]

[0065] Where N is the total number of comments in the online review dataset;

[0066] Calculate the number of words in comment A i The TF-IDF value in is:

[0067] TF-IDF(t,A i )=TF(t,A i )×IDF(t);

[0068] Sort the TF-IDF values ​​from large to small, based on review A i The length m sets the number of keywords o, and the formula is:

[0069]

[0070] Select the first o as comment A i Keywords form the keyword set K i ={t1,t2,…,t o}, where t o For the oth TF-IDF value, calculate the keyword set K i The mean μ k and standard deviation σ k ;

[0071] Calculate the deviation P of each comment keyword set relative to the overall keywords of the online comment dataset i , the formula is:

[0072]

[0073] Use VADER sentiment analyzer to analyze each comment A i Perform sentiment analysis and obtain sentiment score S i ;

[0074] Construct a noise recognition function, the formula is:

[0075]

[0076] Where E i is the noise identification value of the i-th comment;

[0077] Emotional extremity and content deviation are two key factors in identifying noise. This method can dynamically identify high-noise data. Keywords or sentiment scores alone cannot provide the same accuracy. By building a complex noise identification function and combining it with variance-adjusted exponential decay, noise reduction is more accurate and adaptive, which is particularly suitable for social media comments with large emotional fluctuations.

[0078] Use the calculated noise identification value E of the comments i and the review variance F i Calculate the noise reduction weight G for each comment i , the formula is:

[0079]

[0080] The noise is adaptively attenuated through the exponential decay function of variance adjustment. It is designed based on the volatility of the review data. Compared with fixed weight attenuation, variance adjustment can more accurately deal with the diversity of data and avoid excessive or insufficient attenuation. Exponential decay is an effective way to deal with uncertainty and volatility in data. Variance adjustment further enhances adaptability. Replacement methods may lack such flexibility and accuracy.

[0081] Use noise to weaken the weight G i Comment A i Adjusted to get the weakened comment A i ', the formula is:

[0082] A i '=G i ·A i ;

[0083] Calculate the global mean B and standard deviation σ of all the comment data after weakening, and use the 68-95-99.7 rule to calculate the interval [c, d];

[0084] Using the standard normal distribution integral to weaken the comment A i 'Calculate the standard normal distribution integral value, the formula is:

[0085]

[0086] Among them, H i Get the standard normal distribution integral value for the i-th comment in the interval [c, d], where c and d are the lower and upper limits of the integral respectively;

[0087] Online review data often has a long-tail distribution characteristic. Simple statistical methods are difficult to identify normal and abnormal data points. However, the normal distribution integral can effectively distinguish the main data distribution from outliers, ensuring the quality and representativeness of the data. Compared with simple mean-variance screening, the normal distribution integral takes into account the overall distribution characteristics of the data, removes abnormal data more accurately, and retains high-quality data.

[0088] A for all weakened comments i The standard normal distribution integral value H i Perform statistical analysis and plot H i The distribution histogram of , select the lower quartile of the distribution histogram as the judgment threshold T, and set the standard normal distribution integral value H greater than the judgment threshold T i , as the normally distributed data A i ", the standard normal distribution integral value H that is less than and equal to the judgment threshold T i Delete it;

[0089] Calculate the normal distribution data A i The mean μ k and variance σ k , based on the mean μ k and variance σ k The time correlation adjustment coefficient N(k) is obtained as follows:

[0090]

[0091] Where k is the kth time period;

[0092] Calculate the time-correlated noise feature h for each time period k k , the formula is:

[0093]

[0094] where n k is the number of normally distributed data in the kth time period, A i ” ,k is the i-th normally distributed data in the k-th time period;

[0095] Calculate the time-correlated noise characteristics h for all time periods k The mean μ h , process the noise in each time period and obtain the processed noise impact result f(h k ), the formula is:

[0096]

[0097] Where β is the adjustment parameter, which is set based on historical noise processing data;

[0098] Combined with the time correlation adjustment coefficient N(k) and the processed noise impact result f(h k ), for the normally distributed data A in the overall time period i " is processed to obtain the processed data set I k , the formula is:

[0099]

[0100] Where M is the total number of time periods, and T is the total duration of the time periods;

[0101] Time-correlated noise processing takes into account the changing characteristics of data over time and ensures the consistency of data in time. The replacement method often ignores the time correlation, resulting in a decrease in data quality. The method combining normalization and time-correlated noise processing ensures the consistency of data in the time dimension and is more suitable for processing time-sensitive data sets.

[0102] Construct the denoising formula for online review data:

[0103]

[0104] Where J is the denoised online review dataset, a and b are the upper and lower limits of the normal distribution integral respectively;

[0105] The denoising formula integrates data feature extraction, noise processing, normal distribution analysis and time correlation processing to ensure that the output cleaned data set has the highest quality. The denoising formula systematically processes the data through the organic combination of multiple steps, making the cleaned data more accurate and reliable.

[0106] The pre-trained word embedding model Word2Vec is used to convert each word j in each comment in the customer online comment text dataset into a corresponding vector, which can capture the semantic relationship between words, rather than just simple statistics based on word frequency. This deep semantic capture enables the model to better understand the potential needs in the comments, especially when faced with comments containing complex expressions or metaphors. The word embedding model can significantly improve the accuracy of demand identification. Compared with the traditional bag-of-words model, Word2Vec can retain the order and context information between words, thereby providing a richer semantic representation. The introduction of TF-IDF effectively balances the importance of words in a single comment and the entire comment dataset, and can highlight those keywords that appear frequently in specific comments but are relatively rare in the overall dataset. The sorting based on TF-IDF values ​​and the selection of keyword sets can ensure that only words that are highly relevant to user needs are retained, reducing unnecessary noise interference and providing a basis for subsequent sentiment analysis and demand identification. The noise identification function can effectively identify abnormal comments according to the characteristics of the comments, and the comments are adjusted by the noise attenuation weights, which further reduces the impact of noise on the data set, avoids the misleading of noise data on demand identification, and significantly improves the accuracy of the model. Compared with the traditional simple noise filtering method, the denoising process of the present invention takes into account the multidimensional characteristics and time correlation of the comments, ensures the accuracy and effectiveness of the denoising process, and uses the standard normal distribution integral value to screen the comment data and perform statistical analysis on it, which not only effectively eliminates the noise data that may affect the analysis results, but also ensures that the retained data set has good statistical characteristics. Using the standard normal distribution integral and the 68-95-99.7 rule, the retention range of the comment data can be scientifically set to maximize the retention of data that is beneficial to demand identification. This innovative processing method not only improves the data quality, but also provides more reliable data support for the subsequent support vector machine model training and demand identification.

[0107] S2, extract the features of the denoised online review data and build a support vector machine (SVM) model to identify customer needs;

[0108] Specifically, extracting the denoised online comment data features refers to extracting the denoised online comment data from the denoised online comment data set J;

[0109] Use Hugging Face's transformers library to load the pre-trained BERT model and input the denoised online review data into the pre-trained BERT model to obtain a high-dimensional feature vector.

[0110] The high-dimensional vector output by the pre-trained BERT is standardized, the covariance matrix of the standardized high-dimensional feature vector is calculated, the covariance matrix is ​​eigen-decomposed, the eigenvalues ​​and corresponding eigenvectors are solved, the corresponding eigenvectors are sorted from large to small, and a selection threshold e is set based on the historical corresponding eigenvectors. The eigenvectors greater than the selection threshold e are selected to generate the online comment data feature vector, and the eigenvectors less than and equal to the threshold e are deleted.

[0111] The BERT model can consider the context information of the text at the same time through the bidirectional encoder structure. The generated feature vector not only contains the surface meaning of the comment, but also captures the deep semantic information implicit in the text. Compared with the traditional bag-of-words model or TF-IDF model, the application of the BERT model significantly improves the ability to understand complex comment texts, greatly improving the accuracy of demand identification. Standardization processing can eliminate the scale differences in the feature vector, so that features of different dimensions can be compared under the same standard. Calculating the covariance matrix and performing feature decomposition not only helps to reduce the dimension of the data and reduce the computational complexity, but also can retain the most informative feature vectors, providing high-quality input data for subsequent demand identification models. The selection threshold e is set based on historical data, and feature vectors greater than the threshold are selected as the final online comment data feature vectors. This process ensures that only those features that have made a significant contribution to demand identification are retained, further improving the accuracy of the model. The method of setting the selection threshold based on the historical corresponding feature vector has higher flexibility and adaptability, and can automatically adjust the selection criteria according to different data sets, thereby ensuring the efficiency and reliability of demand identification in various scenarios.

[0112] Furthermore, constructing a support vector machine (SVM) model for customer demand identification means collecting annotated customer historical online review data for denoising and feature extraction to generate a training set;

[0113] Construct a support vector machine (SVM) model, including input layer, hidden layer, decision layer and output layer;

[0114] Set the input layer to the feature vector of online review data;

[0115] Use the training set to train the support vector machine (SVM) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters;

[0116] The feature vector of online review data is input into the trained support vector machine (SVM) model to obtain the classification results of customer needs.

[0117] The feature vectors of online review data are often high in dimension, and the SVM model can effectively distinguish different categories of needs and reduce the risk of overfitting by constructing the optimal hyperplane. Compared with other classification algorithms, SVM can still maintain a high classification accuracy when the sample data distribution is unbalanced, which is especially important for complex and diverse online review data. When using the training set to train the SVM model, the introduction of the Adam optimizer and loss function further improves the optimization efficiency of the model. The adaptive learning rate adjustment mechanism of the Adam optimizer enables the model to converge quickly while avoiding falling into the local optimal solution, thereby ensuring the efficiency and stability of the final model. By inputting the feature vector of online review data into the trained SVM model, the classification results of customer needs can be obtained. This result has important practical value for enterprises and can help enterprises quickly identify the main needs and pain points of customers, so as to carry out targeted product improvements and marketing strategies.

[0118] S3. Build a visual interface to display the results of customer demand identification in real time, store online review data securely and implement access control;

[0119] Specifically, building a visualization interface to display the results of customer demand identification in real time refers to using the Dash framework of the visualization tool Python to build a visualization interface, using a data visualization tool to display the classification results of customer needs in real time, and allowing users who have passed the real-name verification to view it.

[0120] The ease of use and flexibility of the Dash framework enable developers to quickly build highly customized interfaces to meet specific needs. The built-in interactivity of the Dash framework allows users to dynamically adjust the content displayed on the interface to reflect data changes in real time. This is particularly important for the display of customer demand identification results, because users can quickly obtain demand classification results through an intuitive interface, thereby improving the efficiency and accuracy of data analysis. The real-time display of customer demand classification results through data visualization tools greatly improves the efficiency and effectiveness of information transmission. Real-time display means that the system can immediately reflect the latest analysis results, allowing users to obtain and process key information in a timely manner. This has important practical significance in a rapidly changing market environment, especially when immediate decision-making is required. Real-time data display can help users respond quickly to market changes. The use of data visualization tools enables complex customer demand analysis results to be presented in an intuitive graphical form, reducing the difficulty of understanding and expanding the applicability and ease of use of the results. When it comes to sensitive customer demand data, ensuring that only authenticated users can access this information is a necessary measure to protect data security. Through real-name verification, the system can track user operations and prevent data leakage or abuse, thereby enhancing the reliability of the entire system.

[0121] Furthermore, the online review data will be securely stored and access controlled, which means that the classification results and timestamps of customer needs of the online review data will be stored in the database, and security access measures will be set up. The database will back up the stored data in the cloud, and regularly perform integrity checks on the stored data and backup data. After the test is completed, the integrity test record will be generated and stored synchronously in the database.

[0122] In the era of digitalization and networking, data security has become an important guarantee for enterprise operations. Through strict security access measures, unauthorized access is prevented, and user privacy and corporate trade secrets are effectively protected. Cloud backup not only provides an additional data protection layer, but also greatly improves the speed and efficiency of data recovery. Whether in the case of hardware failure, natural disasters or human error, cloud backup can ensure the integrity and recoverability of data. This multi-level data protection measure enables enterprises to restore key data in the shortest time, thereby minimizing business interruption and economic losses caused by data loss. Integrity detection can timely detect any abnormal situation of data, including data corruption or unauthorized changes, and then take necessary repair measures. Through this preventive protection mechanism, the present invention not only ensures the continuous availability of data, but also provides transparency and traceability for data management. The generated integrity detection record is not only a proof of the health status of the data, but also can be used as a legal basis or part of the audit report to strengthen the compliance and responsibility management of the enterprise. The classification results and timestamp data of customer requirements are securely stored and backed up in the cloud, which not only improves the security of data management, but also has important commercial application value.

[0123] Example 2, reference Figure 2 , which is the second embodiment of the present invention. This embodiment is different from the previous embodiment and provides a customer demand identification system based on online review data, including:

[0124] An acquisition processing module is used to acquire online comment data and perform standardization processing, and perform denoising processing on the standardized online comment data set;

[0125] Extraction building module, used to extract the features of online review data after denoising and build a support vector machine (SVM) model to identify customer needs;

[0126] Build a storage module to construct a visual interface to display the results of customer demand identification in real time. Online review data will be securely stored and access control will be implemented.

[0127] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0128] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0129] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0130] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A customer demand identification method based on online review data, characterized in that: include, Obtain online review data and perform standardization, and perform denoising on the standardized online review data set; Extract the features of the denoised online review data and build a support vector machine (SVM) model to identify customer needs; Build a visual interface to display the results of customer demand identification in real time, securely store online review data and implement access control.

2. The method for identifying customer needs based on online review data according to claim 1, characterized in that: The obtaining of online review data and performing standardized processing refers to using a crawler tool to obtain customer online review data from online platforms to form an original data set, wherein the online platforms include e-commerce and social media platforms; Use the online translation API to unify the language of the original data set and perform standardization processing, including using the Jieba word segmentation tool to segment the customer online review data set to remove stop words, using the text deduplication algorithm to detect and delete duplicate comments in the customer online review data set, and using regular expression matching to remove HTML tags, emoticons and non-text content from the customer online review data set to form a unified online review data set.

3. The method for identifying customer needs based on online review data as claimed in claim 2, characterized in that: The denoising of the standardized online comment dataset refers to performing word segmentation on the standardized online comment dataset, using the pre-trained word embedding model Word2Vec to segment each comment A. i The jth word in is converted into the corresponding word vector A ij , and calculate for each comment A i The mean value B i , standard deviation C i and variance F i ; Calculate the number of words in comment A i TF-IDF values ​​in the comment A and sort them from large to small, and select the first o as comment A. i The keywords form the keyword set K i ={t1,t2,…,t o }, where t o For the oth TF-IDF value, calculate the keyword set K i The mean μ k and standard deviation σ k And the deviation P relative to the overall keywords of the online review dataset i ; Use VADER sentiment analyzer to analyze each comment A i Perform sentiment analysis to obtain sentiment score S i ; Construct a noise recognition function, the formula is: Where E i is the noise identification value of the i-th comment; Based on the noise identification value E i and the review variance F i Calculate the noise reduction weight G for each comment i , the formula is: Use noise to weaken the weight G i Comment A i Adjusted to get the weakened comment A i '; Calculate the global mean B and standard deviation σ of all the weakened comment data, and use the standard normal distribution integral to calculate the weakened comment A i 'Calculate the standard normal distribution integral value, the formula is: Among them, H i Get the standard normal distribution integral value for the i-th comment in the interval [c, d], where c and d are the lower and upper limits of the integral respectively; Set the judgment threshold T, and set the standard normal distribution integral value H greater than the judgment threshold T i , as the normally distributed data A i ", delete the standard normal distribution integral value H that is less than and equal to the judgment threshold T i ; Calculate the normal distribution data A i The mean μ k and variance σ k , calculate the time correlation adjustment coefficient N(k) and the time correlation noise characteristic h of each time period k k ; Calculate the time-correlated noise characteristics h for all time periods k The mean μ h , process the noise in each time period and obtain the processed noise impact result f(h k ), the formula is: Where β is the adjustment parameter; Combined with the time correlation adjustment coefficient N(k) and the processed noise impact result f(h k ), for the normally distributed data A in the overall time period i " is processed to obtain the processed data set I k , the formula is: Where M is the total number of time periods, k is the kth time period, and T is the total duration of the time period; Construct the denoising formula for online review data: Where J is the denoised online review dataset, and a and b are the upper and lower limits of the normal distribution integral, respectively.

4. The method for identifying customer needs based on online review data as claimed in claim 3, characterized in that: The extracting of denoised online comment data features refers to extracting denoised online comment data from the denoised online comment data set J; Use Hugging Face's transformers library to load the pre-trained BERT model and input the denoised online review data into the pre-trained BERT model to obtain a high-dimensional feature vector. The high-dimensional vector output by the pre-trained BERT is standardized, the covariance matrix of the standardized high-dimensional feature vector is calculated, the covariance matrix is ​​eigen-decomposed, the eigenvalues ​​and corresponding eigenvectors are solved, the corresponding eigenvectors are sorted from large to small, and a selection threshold e is set based on the historical corresponding eigenvectors. The eigenvectors greater than the selection threshold e are selected to generate the online comment data feature vector, and the eigenvectors less than and equal to the threshold e are deleted.

5. The method for identifying customer needs based on online review data as claimed in claim 4, characterized in that: The construction of the support vector machine (SVM) model for customer demand identification refers to collecting annotated historical customer online review data for denoising and feature extraction to generate a training set; Construct a support vector machine (SVM) model, including input layer, hidden layer, decision layer and output layer; Set the input layer to the feature vector of online review data; Use the training set to train the support vector machine (SVM) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters; The feature vector of online review data is input into the trained support vector machine (SVM) model to obtain the classification results of customer needs.

6. The method for identifying customer needs based on online review data according to claim 5, characterized in that: The said constructing a visualization interface to display the results of customer demand identification in real time refers to using the Dash framework of the visualization tool Python to construct a visualization interface, using a data visualization tool to display the classification results of customer needs in real time, and allowing users who have passed the real-name verification to view them.

7. The method for identifying customer needs based on online review data according to claim 6, characterized in that: The secure storage and access control of online review data refers to storing the classification results and timestamps of customer requirements of the online review data in the database, and setting security access measures. The database will back up the stored data in the cloud, and regularly perform integrity checks on the stored data and backup data. After the test is completed, an integrity test record will be generated and stored synchronously in the database.

8. A customer demand identification system based on online review data based on the customer demand identification method based on online review data according to any one of claims 1 to 7, characterized in that: include, An acquisition processing module is used to acquire online comment data and perform standardization processing, and perform denoising processing on the standardized online comment data set; Extraction building module, used to extract the features of online review data after denoising and build a support vector machine (SVM) model to identify customer needs; Build a storage module to construct a visual interface to display the results of customer demand identification in real time, securely store online review data and implement access control.

9. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the customer demand identification method based on online review data described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying customer needs based on online review data described in any one of claims 1 to 7 are implemented.

Images