Customer demand identification method and system based on online review data

Abstract

The present invention discloses a method and system for identifying customer needs based on online review data, which relates to the technical fields of data mining and natural language processing. The method comprises obtaining and standardizing online review data, denoising the standardized online review data set, extracting features from the denoised online review data, and constructing a support vector machine (SVM) model for identifying customer needs. The method also comprises constructing a visual interface to display the results of customer need identification in real time, securely storing the online review data, and implementing access control. By obtaining and standardizing online review data, denoising the standardized online review data set, extracting features from the denoised online review data, and constructing a support vector machine (SVM) model for identifying customer needs, the method improves the accuracy and efficiency of data processing, enhances the ability to understand user needs, and improves the accuracy and robustness of customer need identification.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining and natural language processing, and in particular to a method and system for identifying customer needs based on online review data. Background Art

[0002] In today's digital age, online review data has gradually become an important resource for understanding customer needs and behaviors. With the rapid development of Internet technology, the popularity of e-commerce platforms and social media has enabled users to express their satisfaction, suggestions and dissatisfaction with products and services through online reviews. These review data are directly derived from users' real experience and are of high value. Faced with massive and unstructured review data, traditional customer demand identification methods seem to be powerless. In order to effectively mine valuable customer needs from these reviews, researchers have proposed a variety of analysis methods based on natural language processing (NLP), machine learning (ML) and big data technologies. These methods can automatically process and analyze review data, identify hidden patterns and trends, and thus help companies better understand market demand and optimize products and services.

[0003] Existing technologies still have many shortcomings in identifying customer needs based on online review data. Review data usually contains a large amount of noise information, such as repeated comments, irrelevant content, and non-text elements. This noise information will interfere with the accuracy of the model and lead to inaccurate recognition results. Review data is often heterogeneous, and review data from different platforms have different formats and languages. When extracting key demand features from reviews, existing technologies generally rely on simple word frequency statistics methods, ignoring the potential semantic information and contextual relevance in the reviews, and cannot fully reflect the real needs of customers. These shortcomings lead to limitations in the accuracy and reliability of current technologies in customer demand identification. Summary of the Invention

[0004] In view of the problems existing in the above-mentioned existing customer demand identification methods and systems based on online review data, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that review data usually contains a large amount of noise information, such as repeated comments, irrelevant content and non-text elements. These noise information will interfere with the accuracy of the model and lead to inaccurate recognition results. Review data is often heterogeneous, and the review data formats and languages ​​from different platforms are different. When extracting key demand features in reviews, the existing technology generally relies on simple word frequency statistics methods, ignoring the potential semantic information and contextual relevance in the reviews, and cannot fully reflect the real needs of customers. These shortcomings lead to limitations in the accuracy and reliability of current technology in customer demand identification.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a customer demand identification method based on online review data, which includes obtaining online review data and standardizing it, and denoising the standardized online review data set; extracting features of the denoised online review data, and constructing a support vector machine (SVM) model to identify customer needs; constructing a visual interface to display the results of customer demand identification in real time, and securely storing the online review data and implementing access control.

[0007] As a preferred embodiment of the method for identifying customer needs based on online review data of the present invention, the method of obtaining and standardizing online review data refers to using a crawler tool to obtain customer online review data from online platforms to form an original data set, wherein the online platforms include e-commerce and social media platforms;

[0008] Use the online translation API to unify the language of the original dataset and perform standardization processing, including using the Jieba word segmentation tool to segment the customer online review dataset to remove stop words, using the text deduplication algorithm to detect and delete duplicate comments in the customer online review dataset, and using regular expression matching to remove HTML tags, emoticons and non-text content from the customer online review dataset to form a unified online review dataset.

[0009] As a preferred solution of the customer demand identification method based on online review data of the present invention, wherein: the standardized online review data set is segmented, and each review A is segmented using the pre-trained word embedding model Word2Vec. i The jth word in is converted into the corresponding word vector A ij , and calculate each comment A i The mean value B i , standard deviation C i and variance F i ;

[0010] Calculate the number of words in comment A i The TF-IDF values ​​in the TF-IDF table are sorted from large to small, and the first o words are selected as comment A. i Keywords and form a keyword set, calculate the comment A i The deviation P of the keyword set relative to the overall keywords of the online review dataset i , the formula is:

[0011]

[0012] Use VADER sentiment analyzer to analyze each comment A i Perform sentiment analysis to obtain sentiment score S i ;

[0013] Construct the noise recognition function, the formula is:

[0014]

[0015] Among them E i is the noise identification value of the i-th comment;

[0016] Based on the noise identification value E i and review variance F i Calculate the noise reduction weight G for each comment i , the formula is:

[0017]

[0018] Use noise to weaken the weight G i Comment A i Adjust and get the weakened comment A′ i ;

[0019] Calculate the global mean B and standard deviation σ of all the comment data after weakening, and use the standard normal distribution integral to the weakened comment A i 'Calculate the standard normal distribution integral value, the formula is:

[0020]

[0021] Among them, H i Get the standard normal distribution integral value for the i-th comment in the interval [c, d], where c and d are the lower and upper limits of the integral respectively;

[0022] Set the judgment threshold T, and set the standard normal distribution integral value H greater than the judgment threshold T i Corresponding weakened comment A i ', as the normally distributed data A i ", delete the standard normal distribution integral value H that is less than and equal to the judgment threshold T i The corresponding weakened comment A′ i ;

[0023] Calculate the normal distribution data A in the kth time period i The mean μ k and variance σ k , calculate the time correlation adjustment coefficient N(k) and the time correlation noise characteristic h of each time period k k , the formula is:

[0024]

[0025] where n k is the number of normally distributed data in the kth time period, A″ i,kis the i-th normally distributed data in the k-th time period;

[0026] Calculate the time-correlated noise characteristics h for all time periods k The mean μ h , process the noise in each time period and obtain the noise impact result f(h k ), the formula is:

[0027]

[0028] Where β is the adjustment parameter;

[0029] Combined with the time correlation adjustment coefficient N(k) and the processed noise impact result f(h k ), for the normally distributed data A within the entire time period i " is processed to obtain the processed data I k , the formula is:

[0030]

[0031] Where M is the total number of time periods, and T is the total duration of the time period;

[0032] Construct a denoising formula for online review data:

[0033]

[0034] Where J is the denoised online review dataset, N is the number of reviews in the online review dataset, and a and b are the upper and lower limits of the normal distribution integral, respectively.

[0035] As a preferred solution of the customer demand identification method based on online review data of the present invention, wherein: said extracting the denoised online review data features refers to extracting the denoised online review data from the denoised online review dataset J;

[0036] Use Hugging Face's transformers library to load the pre-trained BERT model and input the denoised online review data into the pre-trained BERT model to obtain a high-dimensional feature vector.

[0037] The high-dimensional vectors output by the pre-trained BERT are normalized, the covariance matrix of the normalized high-dimensional feature vector is calculated, the covariance matrix is ​​eigendecomposed, the eigenvalues ​​and corresponding eigenvectors are solved, the corresponding eigenvectors are sorted from large to small, a selection threshold e is set based on the historical corresponding eigenvectors, eigenvectors greater than the selection threshold e are selected, and feature vectors of online comment data are generated. The eigenvectors less than or equal to the threshold e are deleted.

[0038] As a preferred solution of the method for identifying customer needs based on online review data of the present invention, wherein: the construction of a support vector machine (SVM) model for identifying customer needs refers to collecting annotated historical customer online review data, performing denoising and feature extraction, and generating a training set;

[0039] Construct a support vector machine (SVM) model, including input layer, hidden layer, decision layer, and output layer;

[0040] Set the input layer to the feature vector of online review data;

[0041] Use the training set to train the support vector machine (SVM) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters.

[0042] The feature vector of online review data is input into the trained support vector machine (SVM) model to obtain the classification results of customer needs.

[0043] As a preferred solution of the customer demand identification method based on online review data described in the present invention, wherein: the construction of a visual interface to display the results of customer demand identification in real time refers to using the Dash framework of the visualization tool Python to build a visual interface, and using a data visualization tool to display the classification results of customer needs in real time, allowing users who have passed real-name verification to view them.

[0044] As a preferred solution of the customer demand identification method based on online review data described in the present invention, the secure storage of online review data and the implementation of access control refer to storing the classification results and timestamps of customer needs of the online review data in the database, and setting security access measures. The database will back up the stored data in the cloud, and regularly perform integrity checks on the stored data and backup data. After the test is completed, an integrity test record will be generated and stored synchronously in the database.

[0045] Another object of the present invention is to provide a customer demand identification system based on online review data, which comprises:

[0046] An acquisition and processing module is used to acquire and standardize online review data, and then perform denoising on the standardized online review data set;

[0047] Extraction and construction module, used to extract the features of denoised online review data and build a support vector machine (SVM) model to identify customer needs;

[0048] Build a storage module to create a visual interface to display the results of customer demand identification in real time, securely store online review data and implement access control.

[0049] A computer device includes: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned customer demand identification method based on online review data when executing the computer program.

[0050] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for identifying customer needs based on online review data.

[0051] The beneficial effects of the present invention are as follows: the present invention obtains online review data and performs standardization processing, denoises the standardized online review data set, extracts the features of the denoised online review data, and constructs a support vector machine (SVM) model to identify customer needs, thereby improving the accuracy and efficiency of data processing, enhancing the ability to understand user needs, and improving the accuracy and robustness of customer demand identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 Flowchart of the customer needs identification method based on online review data.

[0054] Figure 2 Schematic diagram of the structure of the customer demand identification system based on online review data. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0058] Example 1, with reference to Figure 1 , which is the first embodiment of the present invention, provides a method for identifying customer needs based on online review data. The method for identifying customer needs based on online review data includes:

[0059] S1. Obtain online review data and perform normalization processing, and then perform denoising processing on the normalized online review dataset;

[0060] Specifically, obtaining online review data and performing normalized processing refers to using crawler tools to obtain customer online review data from online platforms, including e-commerce and social media platforms, to form an original data set;

[0061] Use the online translation API to unify the language of the original dataset and perform standardization processing, including using the Jieba word segmentation tool to segment the customer online review dataset to remove stop words, using the text deduplication algorithm to detect and delete duplicate comments in the customer online review dataset, and using regular expression matching to remove HTML tags, emoticons and non-text content from the customer online review dataset to form a unified online review dataset.

[0062] By unifying all comment data into the same language, data inconsistencies caused by language differences are eliminated, ensuring that subsequent analysis can be performed in the same context, making feature extraction, sentiment analysis and other processes more accurate and effective. By removing these stop words, the words retained in the dataset are more informative, thereby improving the effectiveness of subsequent model training. Text deduplication not only improves data quality but also greatly reduces the waste of computing resources. Regular expression matching is used to clean the online comment dataset, removing irrelevant information in the dataset and ensuring the validity and relevance of the remaining data.

[0063] Furthermore, the standardized online review dataset is subjected to denoising, which means segmenting the standardized online review dataset and using the pre-trained word embedding model Word2Vec to embed each comment A in the segmented customer online review text dataset. i The jth word in is converted into the corresponding word vector A ij , and calculate each comment A i The mean value B i , standard deviation C i and variance F i , the formula is:

[0064]

[0065] Where n is the number of comments A i The number of words in

[0066] Count each comment A i The number of times each word t appears in the word is obtained, and the word frequency TF(t,A i ), count the number of documents DF(t) in which word t appears in the online review dataset, and calculate the inverse document frequency IDF(t). The formula is:

[0067]

[0068] Where N is the total number of comments in the online review dataset;

[0069] Calculate the number of words in comment A i The TF-IDF value in is:

[0070] TF-IDF(t,A i )=TF(t,A i )×IDF(t);

[0071] Sort the TF-IDF values ​​from large to small, based on comment A i The length m sets the number of keywords o, and the formula is:

[0072]

[0073] Select the first o as comment A i Keywords, forming the keyword set K i ={t1,t2,…,t o}, where t o For the oth TF-IDF value, calculate the keyword set K i The mean μ' and standard deviation σ';

[0074] Calculate the deviation P of each comment keyword set relative to the overall keywords of the online comment dataset i , the formula is:

[0075]

[0076] Use VADER sentiment analyzer to analyze each comment A i Perform sentiment analysis and obtain sentiment score S i ;

[0077] Construct the noise recognition function, the formula is:

[0078]

[0079] Among them E i is the noise identification value of the i-th comment;

[0080] Emotional extremity and content deviation are two key factors in identifying noise. This method can dynamically identify high-noise data. Relying solely on keywords or sentiment scores cannot provide the same accuracy. By constructing a complex noise identification function and combining it with variance-adjusted exponential decay, noise reduction is more precise and adaptive, making it particularly suitable for social media comments with large emotional fluctuations.

[0081] Use the calculated noise identification value E of the comments i and review variance F i Calculate the noise reduction weight G for each comment i , the formula is:

[0082]

[0083] Adaptively attenuating noise through a variance-adjusted exponential decay function is designed based on the volatility of review data. Compared to fixed-weight attenuation, variance adjustment can more accurately address data diversity and avoid over- or under-attenuation. Exponential decay is an effective method for handling uncertainty and volatility in data, and variance adjustment further enhances adaptability. Alternative methods may lack this flexibility and accuracy.

[0084] Use noise to weaken the weight G i Comment A i Adjust and get the weakened comment A′ i , the formula is:

[0085] A i '=G i ·A i ;

[0086] Calculate the global mean B and standard deviation σ of all the comment data after weakening, and use the 68-95-99.7 rule to calculate the interval [c, d];

[0087] Using the standard normal distribution integral to weaken the comment A i 'Calculate the standard normal distribution integral value, the formula is:

[0088]

[0089] Among them, H i Get the standard normal distribution integral value for the i-th comment in the interval [c, d], where c and d are the lower and upper limits of the integral respectively;

[0090] Online review data often has a long-tail distribution characteristic. Simple statistical methods have difficulty identifying normal and abnormal data points. However, the normal distribution integral can effectively distinguish the main data distribution from outliers, ensuring the quality and representativeness of the data. Compared with simple mean-variance screening, the normal distribution integral considers the overall distribution characteristics of the data, more accurately removes abnormal data, and retains high-quality data.

[0091] For all weakened comments A′ i The standard normal distribution integral value H i Perform statistical analysis and draw H i The distribution histogram of the distribution histogram is selected, and the lower quartile of the distribution histogram is selected as the judgment threshold T. The standard normal distribution integral value H greater than the judgment threshold T is i The corresponding weakened comment A′ i , as the normally distributed data A″ i , the standard normal distribution integral value H that is less than and equal to the judgment threshold T i The corresponding weakened comment A′ i Delete;

[0092] Calculate the normal distribution data A in the kth time period i The mean μ k and variance σ k , based on the mean μ k and variance σ k The time correlation adjustment coefficient N(k) is obtained as follows:

[0093]

[0094] Where k is the kth time period;

[0095] Calculate the time-correlated noise feature h for each time period k k , the formula is:

[0096]

[0097] where n k is the number of normally distributed data in the kth time period, A″ i,k is the i-th normally distributed data in the k-th time period;

[0098] Calculate the time-correlated noise characteristics h for all time periods k The mean μ h , process the noise in each time period and obtain the noise impact result f(h k ), the formula is:

[0099]

[0100] Where β is the adjustment parameter, which is set based on historical noise processing data;

[0101] Combined with the time correlation adjustment coefficient N(k) and the processed noise impact result f(h k ), for the normally distributed data A within the entire time period i " is processed to obtain the processed data I k , the formula is:

[0102]

[0103] Where M is the total number of time periods, and T is the total duration of the time period;

[0104] Time-correlated noise processing takes into account the temporal variation of data and ensures data consistency over time. Replacement methods often ignore temporal correlation, resulting in degraded data quality. The method combining normalization and time-correlated noise processing ensures data consistency over time and is more suitable for processing time-sensitive datasets.

[0105] Construct a denoising formula for online review data:

[0106]

[0107] Where J is the denoised online review dataset, N is the number of reviews in the online review dataset, a and b are the upper and lower limits of the normal distribution integral respectively;

[0108] The denoising formula integrates data feature extraction, noise processing, normal distribution analysis, and time correlation processing to ensure that the output cleaned data set has the highest quality. The denoising formula systematically processes data through the organic combination of multiple steps, making the cleaned data more accurate and reliable.

[0109] The pre-trained word embedding model Word2Vec is used to convert each word j in each comment in the customer online comment text dataset into a corresponding vector, which can capture the semantic relationship between words, rather than just simple statistics based on word frequency. This deep semantic capture enables the model to better understand the potential needs in the comments, especially when facing comments containing complex expressions or metaphors. The word embedding model can significantly improve the accuracy of demand identification. Compared with the traditional bag-of-words model, Word2Vec can retain the order and context information between words, thereby providing a richer semantic representation. The introduction of TF-IDF effectively balances the importance of words in a single comment and the entire comment dataset, and can highlight those keywords that appear frequently in specific comments but are relatively rare in the overall dataset. The sorting based on TF-IDF values ​​and the selection of keyword sets can ensure that only words that are highly relevant to user needs are retained, reducing unnecessary noise interference and facilitating subsequent sentiment analysis and demand identification. It lays a solid foundation for the recognition of abnormal comments. The noise recognition function can effectively identify abnormal comments according to the characteristics of the comments, and adjusts the comments through the noise attenuation weight, which further reduces the impact of noise on the data set, avoids the misleading of noise data on demand identification, and significantly improves the accuracy of the model. Compared with the traditional simple noise filtering method, the denoising processing of the present invention takes into account the multidimensional characteristics and time correlation of the comments, ensuring the accuracy and effectiveness of the denoising process. The standard normal distribution integral value is used to screen the comment data and perform statistical analysis on it. It not only effectively eliminates the noise data that may affect the analysis results, but also ensures that the retained data set has good statistical properties. Using the standard normal distribution integral and the 68-95-99.7 rule, the retention range of the comment data can be scientifically set to maximize the retention of data that is beneficial to demand identification. This innovative processing method not only improves the quality of the data, but also provides more reliable data support for the subsequent support vector machine model training and demand identification.

[0110] S2, extract the features of the denoised online review data and build a support vector machine (SVM) model to identify customer needs;

[0111] Specifically, extracting denoised online review data features refers to extracting denoised online review data from a denoised online review dataset J;

[0112] Use Hugging Face's transformers library to load the pre-trained BERT model and input the denoised online review data into the pre-trained BERT model to obtain a high-dimensional feature vector.

[0113] The high-dimensional vectors output by the pre-trained BERT are normalized, the covariance matrix of the normalized high-dimensional feature vector is calculated, the covariance matrix is ​​eigendecomposed, the eigenvalues ​​and corresponding eigenvectors are solved, the corresponding eigenvectors are sorted from large to small, a selection threshold e is set based on the historical corresponding eigenvectors, eigenvectors greater than the selection threshold e are selected, and feature vectors of online comment data are generated. The eigenvectors less than or equal to the threshold e are deleted.

[0114] The BERT model, through its bidirectional encoder structure, is able to simultaneously consider contextual information within a text. The generated feature vector not only captures the surface meaning of the review but also captures the deeper semantic information implicit in the text. Compared with traditional bag-of-words or TF-IDF models, the BERT model significantly improves its ability to understand complex review text, significantly improving the accuracy of demand identification. Normalization eliminates scale differences in feature vectors, allowing features of different dimensions to be compared under the same standard. Calculating the covariance matrix and performing feature decomposition not only helps reduce data dimensionality and computational complexity but also retains the most informative feature vectors, providing high-quality input data for subsequent demand identification models. A selection threshold e is set based on historical data, and feature vectors greater than this threshold are selected as the final feature vectors for online review data. This process ensures that only features that significantly contribute to demand identification are retained, further improving model accuracy. Setting the selection threshold based on historical feature vectors offers greater flexibility and adaptability, automatically adjusting the selection criteria to suit different datasets, thereby ensuring efficient and reliable demand identification in various scenarios.

[0115] Furthermore, building a support vector machine (SVM) model for customer demand identification involves collecting annotated historical customer online review data, performing denoising and feature extraction, and generating a training set.

[0116] Construct a support vector machine (SVM) model, including input layer, hidden layer, decision layer, and output layer;

[0117] Set the input layer to the feature vector of online review data;

[0118] Use the training set to train the support vector machine (SVM) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters.

[0119] The feature vector of online review data is input into the trained support vector machine (SVM) model to obtain the classification results of customer needs.

[0120] The eigenvectors of online review data are often high in dimensionality, and the SVM model can effectively distinguish different categories of needs and reduce the risk of overfitting by constructing the optimal hyperplane. Compared with other classification algorithms, SVM can still maintain a high classification accuracy when the sample data distribution is unbalanced, which is especially important for complex and diverse online review data. When using the training set to train the SVM model, the introduction of the Adam optimizer and loss function further improves the optimization efficiency of the model. The adaptive learning rate adjustment mechanism of the Adam optimizer enables the model to converge quickly while avoiding falling into local optimal solutions, thereby ensuring the efficiency and stability of the final model. By inputting the eigenvectors of online review data into the trained SVM model, the classification results of customer needs can be obtained. This result has important practical value for enterprises and can help enterprises quickly identify the main needs and pain points of customers, thereby implementing targeted product improvements and marketing strategies.

[0121] S3. Build a visual interface to display the results of customer demand identification in real time, securely store online review data, and implement access control;

[0122] Specifically, building a visual interface to display the results of customer demand identification in real time refers to using the Dash framework of the visualization tool Python to build a visual interface, using a data visualization tool to display the classification results of customer needs in real time, and allowing users who have passed real-name verification to view it.

[0123] The Dash framework's ease of use and flexibility enable developers to quickly build highly customized interfaces to meet specific needs. The framework's built-in interactivity allows users to dynamically adjust interface content to reflect data changes in real time. This is particularly important for presenting customer demand identification results, as users can quickly access demand classification results through an intuitive interface, thereby improving the efficiency and accuracy of data analysis. Real-time display of customer demand classification results through data visualization tools greatly enhances the efficiency and effectiveness of information transmission. Real-time display means the system can immediately reflect the latest analysis results, allowing users to obtain and process key information in a timely manner. This is of great practical significance in a rapidly changing market environment, especially when immediate decision-making is required. Real-time data display can help users quickly respond to market changes. The use of data visualization tools allows complex customer demand analysis results to be presented in an intuitive graphical format, reducing the difficulty of understanding and expanding the applicability and usability of the results. When dealing with sensitive customer demand data, ensuring that only authenticated users can access this information is a necessary measure to protect data security. Real-name verification allows the system to track user operations, prevent data leakage or abuse, and enhance the reliability of the entire system.

[0124] Furthermore, online review data will be securely stored and access control will be implemented, which means that the classification results and timestamps of customer needs of online review data will be stored in the database, and security access measures will be set up. The database will back up the stored data in the cloud and regularly perform integrity checks on the stored data and backup data. After the test is completed, an integrity test record will be generated and stored synchronously in the database.

[0125] In the era of digitalization and networking, data security has become a critical guarantee for enterprise operations. Through strict security access measures, unauthorized access is prevented, effectively protecting user privacy and corporate trade secrets. Cloud backup not only provides an additional layer of data protection, but also significantly improves the speed and efficiency of data recovery. Whether in the event of hardware failure, natural disasters, or human error, cloud backup can ensure data integrity and recoverability. This multi-layered data protection measure enables enterprises to restore critical data in the shortest possible time, thereby minimizing business interruption and economic losses caused by data loss. Integrity detection can promptly detect any data anomalies, including data corruption or unauthorized changes, and then take necessary repair measures. Through this preventive protection mechanism, the present invention not only ensures the continuous availability of data, but also provides transparency and traceability for data management. The generated integrity detection records are not only proof of data health, but can also be used as legal basis or part of audit reports to strengthen corporate compliance and accountability management. The classification results and timestamp data of customer requirements are securely stored and backed up in the cloud, which not only improves the security of data management but also has important commercial application value.

[0126] Example 2, reference Figure 2 , which is the second embodiment of the present invention. This embodiment is different from the previous embodiment and provides a customer demand identification system based on online review data, including:

[0127] An acquisition and processing module is used to acquire and standardize online review data, and then perform denoising on the standardized online review data set;

[0128] Extraction and construction module, used to extract the features of denoised online review data and build a support vector machine (SVM) model to identify customer needs;

[0129] Build a storage module to create a visual interface to display the results of customer demand identification in real time. Online review data will be securely stored and access control will be implemented.

[0130] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0131] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0132] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0133] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A customer demand identification method based on online review data, characterized by: include, Obtain online review data and perform normalization processing, and then perform denoising processing on the normalized online review dataset; Extract the features of denoised online review data and build a support vector machine (SVM) model to identify customer needs; Build a visual interface to display the results of customer demand identification in real time, securely store online review data and implement access control; The denoising process of the standardized online review dataset includes: performing word segmentation on the standardized online review dataset, using the pre-trained word embedding model Word2Vec to segment each review A i The jth word in is converted into the corresponding word vector A ij , and calculate each comment A i The mean value B i , standard deviation C i and variance F i ; Calculate the number of words in comment A i The TF-IDF values ​​in the TF-IDF table are sorted from large to small, and the first o words are selected as comment A. i Keywords and form a keyword set, calculate the comment A i The deviation P of the keyword set relative to the overall keywords of the online review dataset i , the formula is: Use VADER sentiment analyzer to analyze each comment A i Perform sentiment analysis to obtain sentiment score S i ; Construct the noise recognition function, the formula is: Among them E i is the noise identification value of the i-th comment; Based on the noise identification value E i and review variance F i Calculate the noise reduction weight G for each comment i , the formula is: Use noise to weaken the weight G i Comment A i Adjust and get the weakened comment A′ i ; Calculate the global mean B and standard deviation σ of all the comment data after weakening, and use the standard normal distribution integral to the weakened comment A i 'Calculate the standard normal distribution integral value, the formula is: Among them H i Get the standard normal distribution integral value for the i-th comment in the interval [c, d], where c and d are the lower and upper limits of the integral respectively; Set the judgment threshold T, and set the standard normal distribution integral value H greater than the judgment threshold T i Corresponding weakened comment A i ', as the normally distributed data A i ", delete the standard normal distribution integral value H that is less than and equal to the judgment threshold T i The corresponding weakened comment A′ i ; Calculate the normal distribution data A in the kth time period i The mean μ k and variance σ k , calculate the time correlation adjustment coefficient N(k) and the time correlation noise characteristic h of each time period k k , the formula is: where n k is the number of normally distributed data in the kth time period, A″ i,k is the i-th normally distributed data in the k-th time period; Calculate the time-correlated noise characteristics h for all time periods k The mean μ h , process the noise in each time period and obtain the noise impact result f(h k ), the formula is: Where β is the adjustment parameter; Combined with the time correlation adjustment coefficient N(k) and the processed noise impact result f(h k ), get I k , the formula is: Where M is the total number of time periods, and T is the total duration of the time period; Construct a denoising formula for online review data: Where J is the denoised online review dataset, N is the number of reviews in the online review dataset, and a and b are the upper and lower limits of the normal distribution integral, respectively.

2. The method for identifying customer needs based on online review data according to claim 1, wherein: The acquisition and standardization of online review data refers to using crawler tools to acquire customer online review data from online platforms to form an original data set. The online platforms include e-commerce and social media platforms. Use the online translation API to unify the language of the original dataset and perform standardization processing, including using the Jieba word segmentation tool to segment the customer online review dataset to remove stop words, using the text deduplication algorithm to detect and delete duplicate comments in the customer online review dataset, and using regular expression matching to remove HTML tags, emoticons and non-text content from the customer online review dataset to form a unified online review dataset.

3. The method for identifying customer needs based on online review data according to claim 2, wherein: Extracting the denoised online review data features refers to extracting the denoised online review data from the denoised online review dataset J; Use Hugging Face's transformers library to load the pre-trained BERT model and input the denoised online review data into the pre-trained BERT model to obtain a high-dimensional feature vector. The high-dimensional vectors output by the pre-trained BERT are normalized, the covariance matrix of the normalized high-dimensional feature vector is calculated, the covariance matrix is ​​eigendecomposed, the eigenvalues ​​and corresponding eigenvectors are solved, the corresponding eigenvectors are sorted from large to small, a selection threshold e is set based on the historical corresponding eigenvectors, eigenvectors greater than the selection threshold e are selected, and feature vectors of online comment data are generated. The eigenvectors less than or equal to the threshold e are deleted.

4. The method for identifying customer needs based on online review data according to claim 3, wherein: Constructing a support vector machine (SVM) model to identify customer needs involves collecting annotated historical customer online review data, performing denoising and feature extraction, and generating a training set. Construct a support vector machine (SVM) model, including input layer, hidden layer, decision layer, and output layer; Set the input layer to the feature vector of online review data; Use the training set to train the support vector machine (SVM) model, and use the loss function and Adam optimizer to iteratively optimize the model parameters. The feature vector of online review data is input into the trained support vector machine (SVM) model to obtain the classification results of customer needs.

5. The method for identifying customer needs based on online review data according to claim 4, characterized in that: The said construction of a visual interface to display the results of customer demand identification in real time refers to using the Dash framework of the visualization tool Python to construct a visual interface, using a data visualization tool to display the classification results of customer needs in real time, and allowing users who have passed real-name verification to view them.

6. The method for identifying customer needs based on online review data according to claim 5, characterized in that: The secure storage and access control of online review data refers to storing the classification results and timestamps of customer requirements of the online review data in the database, and setting up security access measures. The database will back up the stored data in the cloud, and regularly perform integrity checks on the stored data and backup data. After the test is completed, an integrity test record will be generated and stored synchronously in the database.

7. A customer demand identification system based on online review data, implementing the customer demand identification method based on online review data according to any one of claims 1 to 6, characterized in that: include, An acquisition and processing module is used to acquire and standardize online review data, and then perform denoising on the standardized online review data set; Extraction and construction module, used to extract the features of denoised online review data and build a support vector machine (SVM) model to identify customer needs; Build a storage module to create a visual interface to display the results of customer demand identification in real time, securely store online review data and implement access control.

8. A computer device comprising: memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the customer demand identification method based on online review data according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying customer needs based on online review data according to any one of claims 1 to 6 are implemented.

Images