Comment theme recognition method, system and equipment based on unsupervised word reconstruction and storage medium
By adopting the unsupervised word reconstruction method and attention mechanism in the deep emotion classification algorithm, combined with the advanced word feature representation of the BERT model, the problems of lack of labeled data and the influence of noise data are solved, and the effect of efficient extraction of subject words in unmarked text is achieved.
Patent Information
- Application Number
- CN202411870698.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-18
- Publication Date
- 2025-03-18
AI Technical Summary
The prior art lacks high-quality labeled data in deep emotion classification algorithms, and there is noisy data and lack of information in weak labeled comment texts, which makes the model unable to effectively learn the fine-grained emotional characteristics of comment texts.
The comment topic recognition method based on unsupervised word reconstruction is adopted, and the text features are weighted through the attention mechanism, and the feature reconstruction idea is introduced to integrate the subject word feature information in an unsupervised way, and the BERT model is used to extract advanced word feature representations.
It realizes accurate extraction of topic words in unmarked input text, improves the accuracy and accuracy of topic extraction, and can effectively process unmarked text data.
Smart Images
Figure CN119647463A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer natural language processing, and relates to a review topic recognition method, system, device and storage medium based on unsupervised word reconstruction. Background Art
[0002] Aspect-level sentiment analysis technology is a challenging research topic in the field of natural language processing. The performance of most deep learning methods depends on large-scale manually annotated datasets, but annotating a large number of high-quality datasets is a time-consuming and labor-intensive task. Therefore, the lack of annotated datasets has become a bottleneck problem restricting deep sentiment classification algorithms. Fortunately, users of different online platforms have generated a large number of labeled public opinion texts (such as product reviews and movie reviews with ratings, Twitter comments with emoticons, etc.), which can be used as weakly annotated datasets for sentiment classification tasks. However, there are noisy data in the weakly annotated data where the labels are inconsistent with the actual sentiment semantics of the text, which will have a serious negative impact on the training process, so it cannot be directly used as strong annotated data for training deep models. In addition, most weakly annotated comment texts do not clearly indicate the aspect words described, and models that lack the guidance of aspect information cannot effectively learn the fine-grained sentiment features of the comment text. Summary of the invention
[0003] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a review topic identification method, system, device and storage medium based on unsupervised word reconstruction, which can accurately identify the topic words in the unlabeled input text.
[0004] In order to achieve the above object, the present invention adopts the following technical solutions: A review topic identification method based on unsupervised word reconstruction includes the following processes: S1, obtain training text data, and encode the training text data to obtain a vector matrix; S2, obtains the text features of the training text data by performing attention weighted operations on the vector matrix; S3, reconstruct the text features after dimensionality reduction calculation to obtain the linear combination of the topic word vectors; S4, classify the linear combination of topic word vectors into corresponding topics, and select the result with the highest probability as the final topic word.
[0005] Preferably, in S1, the training text data is encoded as a single text sequence.
[0006] Preferably, in S1, a BERT encoder is used for encoding.
[0007] Preferably, the vector matrix includes word feature representations of each word. In S2, all word feature representations are summed and then averaged to obtain a global context vector, the attention weight assigned to each word is obtained, and weighted operations are performed on each word feature representation to obtain text features.
[0008] Preferably, in S3, the reconstruction process is:
[0009]
[0010] in, is the parameter matrix, is the bias vector; is a scalar, given by the unsupervised k -means clustering algorithm determines the number of topic words that appear in the representative data set; Indicates that the data set k The weight vector of candidate topic words, each dimension element in the vector represents the probability that the training text data belongs to the relevant topic word; is the topic word vector matrix, using k -means clustering algorithm k The characteristic representation of the cluster center is used to represent the matrix Initialize; The reconstructed feature representation of the training text data is a linear combination of the topic word vectors.
[0011] Preferably, in S3, the reconstruction process is trained by optimizing the objective function, with the goal of reconstructing the text features obtained after the attention weighted operation into a feature representation that incorporates the keyword information.
[0012] Preferably, the objective function is optimized for:
[0013]
[0014]
[0015] Among them, the optimization objective function consists of two parts: is the reconstruction loss term, As a regular term, it is used to ensure the diversity of topic word vectors. is a hyperparameter used to control the regularization term In overall loss The weight of is the reconstruction loss term, where represents the entire training data set, It is a dataset The Input text; represents the number of negative samples randomly sampled from the training set, where j Features of negative samples The detailed calculation process is: , that is, for j Word features corresponding to all words in the negative sample Sum and then average; and Both represent inner product operations; is the regularization term, where is through the topic word vector matrix After row standardization, is an identity matrix; Represents column and norm operations on matrices.
[0016] A review topic identification system based on unsupervised word reconstruction, comprising: An encoding module is used to obtain training text data and encode the training text data to obtain a vector matrix; A weighting module is used to obtain text features of training text data by performing attention weighting operations on the vector matrix; The reconstruction module is used to reconstruct the text features after dimensionality reduction calculation to obtain the linear combination of the topic word vectors; The classification module is used to classify the linear combination of topic word vectors into corresponding topics and select the result with the highest probability as the final topic word.
[0017] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the review topic identification method based on unsupervised word reconstruction are implemented.
[0018] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the review topic identification method based on unsupervised word reconstruction.
[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses the attention mechanism to weight the words in the text, emphasizes the most relevant words in the text, and ignores other irrelevant words, thereby improving the extraction accuracy of the subject words. The feature reconstruction idea is introduced, and the feature information of the subject words in the data set is integrated in an unsupervised manner, which prompts the model to focus on mining the subject words with the highest semantic consistency with the input text subject during the training process, thereby further improving the accuracy of subject extraction and realizing the accurate extraction of subject words from unlabeled input text using unsupervised learning.
[0020] Furthermore, a representation learning method based on the BERT model is adopted, which can fully extract the high-level word feature representation of each word in the input text. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 Schematic diagram of the UAM-BERT network architecture of the present invention. DETAILED DESCRIPTION
[0022] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0023] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs. The terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" used in this article includes any and all combinations of one or more related listed items. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0025] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.
[0026] like Figure 1 As shown, the review topic identification method based on unsupervised word reconstruction described in the present invention is based on the UAM-BERT network architecture, and its purpose is to accurately identify the subject words in the unlabeled input text. Thanks to the powerful representation learning ability of the BERT model, the UAM-BERT framework can fully extract the high-level word feature representation of each word in the input text. The UAM-BERT framework models word-level features, uses an attention mechanism to emphasize the most relevant words in the text, and ignores other irrelevant words. The feature reconstruction idea adopted by this framework integrates the feature information of the subject words in the data set in an unsupervised manner, which can prompt the model to focus on mining the subject words with the highest semantic consistency with the input text theme during the training process.
[0027] Example 1 Encoder BERT BASE The calculation structure uses The UAM-BERT model takes a single text sequence as input and obtains training text data. The training text data is in the form of a sequence The computational structure of the model is described in detail as an example. The BERT encoder processes the input text sequence After encoding, we get the vector matrix , .in, ,and The words The word feature representation.
[0028] In the attention layer, the attention weight is first calculated through the attention mechanism , the calculation process is as follows: (1) (2) (3) in, It is called the global context vector, which is obtained by summing and averaging the feature representations of all words; It is a parameter matrix that can filter out words with lower importance in the text sequence; Represented as an input text sequence Corresponding words in The assigned attention weight represents the current word Becomes input text The probability of the subject word in Figure 1 All shades of medium gray The value can be constructed as an attention weight vector Attention weights The larger the value, the more The greater the probability of becoming a keyword.
[0029] After that, the input sequence is obtained by performing weighted operations on the feature representations of each word Text features , the calculation process is as follows: (4) The feature reconstruction layer uses text features As input, there are two calculation steps. First, the text features Perform dimensionality reduction calculation to obtain vector ; Then for the vector Reconstruct. The detailed calculation structure is as follows: (5) (6) in, is the parameter matrix, is the bias vector; is a scalar, given by the unsupervised k-means clustering algorithm determines the number of topic words that appear in the representative data set. Indicates that the data set k The weight vector of candidate keywords, each dimension element in the vector represents the input text The probability of belonging to the relevant subject term. is the topic word vector matrix, using k -means clustering algorithm k The characteristic representation of the cluster center is used to represent the matrix Initialize. Represents input text The reconstructed feature representation is equivalent to the linear combination of the topic word vectors.
[0030] The UAM-BERT model is based on the idea of feature reconstruction, which aims to convert the text features obtained after attention weighted operation into Reconstructed into features that incorporate keyword information , and minimize the error in the reconstruction process. Based on the above considerations, the following optimization objective function is proposed: (7) (8) (9) Among them, the formula It is the optimization objective function of the UAM-BERT model, which consists of two parts: is the reconstruction loss term, As a regular term, it is used to ensure the diversity of topic word vectors. is a hyperparameter used to control the regularization term In overall loss The weight of . is the reconstruction loss term, where represents the entire training data set, It is a dataset The Input text; represents the number of negative samples randomly sampled from the training set, where j Features of negative samples The detailed calculation process is: , that is, for j Word features corresponding to all words in the negative sample Sum and then average; and Both represent inner product operations. is the regularization term, where is through the topic word vector matrix After row standardization, is an identity matrix; Represents column and norm operations on matrices.
[0031] From the above losses, we can see that the reconstruction loss term can be maximized and Minimize the inner product between With multiple negative samples It can be seen that the reconstruction loss term aims to ensure the reconstruction feature With the current input sequence Text features Similar to, but different from, the features of multiple negative samples Under the regularization constraint, the model forces the reconstruction features in the embedding space Closer , and N The comparison between negative samples prompts the model to learn to reconstruct features The difference between the negative sample and the topic word vector matrix In general, we expect the model to learn the most representative features in the data set during training. k However, the overall optimization objective function is not When adding regularization terms, the topic word vector matrix Each topic word vector in will eventually converge to the same during the training process, resulting in the model being unable to efficiently capture diverse topic word information. The regularization term introduced in This can effectively avoid this situation. The regularization term firstly applies the topic word vector matrix Each row of ,and The operation makes the diagonal elements of the newly obtained matrix equal to 0, and the off-diagonal elements correspond to the inner product between any two different topic word vectors. The value of will continue to decrease, which will cause the inner product between different topic word vectors to continue to decrease. When it decreases to 0 The value of reaches the minimum. Therefore, the regularization term Encourage word vector matrix The orthogonality between each row vector in is ensured, thus ensuring the uniqueness and diversity of different topic word vectors.
[0032] Through the above-mentioned unsupervised word reconstruction optimization method, the model is guided to reconstruct words that conform to the semantic pattern of the subject words, capture the subject words in the text through the attention weight distribution, and learn the subject semantics in the comments without manual labeling.
[0033] Finally, the classification layer of BERT is used to transform Classify into corresponding topics and select the result with the highest probability as the final keyword.
[0034] The present invention combines the BERT model, attention mechanism and feature reconstruction idea, can realize efficient unsupervised topic extraction, is suitable for processing various unlabeled text data, has high practical value and broad application prospects.
[0035] The UAM-BERT network architecture models the input text at the word level, uses the attention mechanism to enable the model to effectively capture the most relevant aspect word features in the input text, and introduces the idea of feature reconstruction to guide the model to learn the attention weight vector that can truly reflect the distribution of aspect words in the data. The experimental results on three real data sets prove the superiority of the UAM-BERT method, and it also has good generalization capabilities even on data sets in other fields.
[0036] Example 2 The present invention evaluates the effect of the UAM-BERT method on two publicly available restaurant review datasets and beer review datasets. The restaurant review dataset consists of 52,574 unlabeled review sentences and 3,400 labeled data with aspect word labels, where the aspect word labels in the labeled data include food, staff, ambience, price, anecdotes and miscellaneous. The beer review dataset consists of 1,586,259 unlabeled review sentences and 9,245 labeled data with aspect word labels, where the aspect word labels in the labeled data include feel, look, smell, taste and overall. In addition, in order to verify the generalization ability of the UAM-BERT method on datasets in other fields, this paper also conducted an evaluation experiment on the Amazon product review dataset. The Amazon product review dataset contains review corpora of three products: digital cameras, mobile phones and laptops, consisting of 1,136,271 unlabeled sentences and 3,520 sentences manually labeled with aspect word labels. It should be noted that, since the present invention focuses more on the extraction of a single aspect word, in this work, the present invention removes the review texts with multiple aspect words and only retains the review texts with a single aspect word.
[0037] Experimental details The present invention counted the sentence lengths in three data sets, and the longest sentence lengths were 85, 67 and 121 respectively. In order to fully obtain all the information of the sentence, the present invention sets the input lengths of the text sequences in these three data sets to 85, 67 and 121 respectively. In the experiment, the batch size value is set to 32. The number of negative samples N obtained by random sampling in Formula 8 is 20, and the hyperparameter η in Formula 7 is 1. Regarding the value of the number of aspect words k in each data set, the present invention first uses the unsupervised k-means algorithm to cluster the training set data to obtain the optimal value of k. After experiments, the present invention sets the value of the number of aspect words k in the restaurant review data set and the beer review data set to 6 and 5 respectively; and sets the value of k in the Amazon product review data set to 20. When using the test set data to evaluate the model, it is taken into account that there are no explicit aspect words in the input text 1s. The model uses the Adam optimizer during training, and the learning rate is set to .
[0038] Results Analysis This paper conducts comparative experiments with multiple methods on restaurant review and beer review datasets. The evaluation indicators are precision, recall and F1 value. The other baseline methods are briefly introduced as follows: LocLDA. The standard LDA (Latent Dirichlet Allocation) model.
[0039] 2) SAS. A hybrid topic model that can mine aspect words and opinion words at the same time. This model is more advantageous than the topic model in identifying aspect words.
[0040] 3) BTM. Bidirectional topic model for short texts. This model performs better than the traditional LDA model in identifying more consistent aspect words.
[0041] 4) ABAE: An aspect extraction model based on the attention mechanism, which uses neural word embedding technology to improve the consistency of aspect words by utilizing the distribution of word co-occurrence.
[0042] Table 1 Aspect word recognition results of different methods on restaurant review dataset
[0043] The experimental results on the restaurant review dataset are shown in Table 1. As can be seen from Table 1, in the experiment of identifying three aspect words, namely food, staff and ambience, the UAM-BERT method achieved the best results in the three evaluation indicators of precision, recall and F1. Compared with the ABAE method, UAM-BERT has more advantages in extracting explicit aspect words.
[0044] Table 2 Aspect word recognition results of different methods on the beer review dataset
[0045] The experimental results on the beer review dataset are shown in Table 2. As can be seen from Table 2, the UAM-BERT method achieved the best results in most experiments. In all aspect word recognition experiments, the recognition effects of "taste" and "smell" were average. This is because these two types of data in the test set usually have implicit aspect words, while there are fewer data with explicit aspect words, which cannot be well divided into the "taste" or "smell" categories, which ultimately resulted in all experimental methods only achieving relatively general experimental results.
[0046] Table 3 Aspect word recognition results of different methods on the Amazon review dataset
[0047] The present invention also conducted an evaluation experiment on the Amazon dataset to verify the effectiveness of the UAM-BERT method on datasets in other fields. Amazon data is collected on the e-commerce platform, and user reviews involve many aspects of the products, so it is impossible to use the evaluation indicators in the above experiment for evaluation. Taking the above factors into consideration, the present invention uses accuracy as the evaluation indicator in this evaluation experiment. The experimental results are shown in Table 3. As can be seen from the table, the UAM-BERT method achieved the highest accuracy, verifying the method's ability to be generalized on datasets in other fields.
[0048] The following are device embodiments of the present invention, which can be used to perform method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.
[0049] In another embodiment of the present invention, a review topic identification system based on unsupervised word reconstruction is provided. The review topic identification system based on unsupervised word reconstruction can be used to implement the above-mentioned review topic identification method based on unsupervised word reconstruction. Specifically, the review topic identification system based on unsupervised word reconstruction includes an encoding module, a weighting module, a reconstruction module and a classification module.
[0050] The encoding module is used to obtain training text data and encode the training text data to obtain a vector matrix.
[0051] The weighting module is used to obtain the text features of the training text data by performing attention weighted operations on the vector matrix.
[0052] The reconstruction module is used to reconstruct text features after dimensionality reduction calculation to obtain a linear combination of topic word vectors.
[0053] The classification module is used to classify the linear combination of topic word vectors into corresponding topics, and select the result with the highest probability as the final topic word.
[0054] In yet another embodiment of the present invention, a terminal device is provided. The terminal device includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement corresponding method processes or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of the comment topic recognition method based on unsupervised word reconstruction, including: S1, obtaining training text data, encoding the training text data to obtain a vector matrix; S2, obtaining text features of the training text data through attention weighted operations on the vector matrix; S3, reconstructing the text features after dimensionality reduction calculations to obtain a linear combination of topic word vectors; S4, classifying the linear combination of topic word vectors into corresponding topics, and selecting the result with the highest probability as the final topic word.
[0055] In another embodiment, the present invention further provides a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory.
[0056] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the comment topic identification method based on unsupervised word reconstruction in the above-mentioned embodiment; one or more instructions in the computer-readable storage medium are loaded by the processor and the following steps are executed: S1, obtaining training text data, encoding the training text data to obtain a vector matrix; S2, obtaining text features of the training text data through attention weighted operations on the vector matrix; S3, reconstructing the text features after dimensionality reduction calculations to obtain a linear combination of topic word vectors; S4, classifying the linear combination of topic word vectors into corresponding topics, and selecting the result with the highest probability as the final topic word.
[0057] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0058] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0059] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0061] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0062] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0063] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0064] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0065] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
[0066] It should be understood that the above description is for illustration and not for limitation. Many embodiments and many applications beyond the examples provided will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this patent should not be determined with reference to the above description, but rather with reference to the preceding claims and the full scope of equivalents to which such claims are entitled. For the purpose of comprehensiveness, all articles and references, including disclosures of patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the preceding claims is not intended to be a waiver of such subject matter, nor should it be considered that the applicant has not considered such subject matter to be part of the disclosed inventive subject matter.
Claims
1. A review topic identification method based on unsupervised word reconstruction, characterized in that: The process includes: S1, obtain training text data, and encode the training text data to obtain a vector matrix; S2, obtains the text features of the training text data by performing attention weighted operations on the vector matrix; S3, reconstruct the text features after dimensionality reduction calculation to obtain the linear combination of the topic word vectors; S4, classify the linear combination of topic word vectors into corresponding topics, and select the result with the highest probability as the final topic word.
2. The review topic identification method based on unsupervised word reconstruction according to claim 1 is characterized in that: In S1, the training text data is encoded as a single text sequence.
3. The review topic identification method based on unsupervised word reconstruction according to claim 1 is characterized in that: In S1, BERT encoder is used for encoding.
4. The review topic identification method based on unsupervised word reconstruction according to claim 1 is characterized in that: The vector matrix includes the word feature representation of each word. In S2, all word feature representations are summed and then averaged to obtain the global context vector, the attention weight assigned to each word is obtained, and the weighted operation is performed on each word feature representation to obtain the text feature.
5. The review topic identification method based on unsupervised word reconstruction according to claim 1 is characterized in that: In S3, the reconstruction process is: in, is the parameter matrix, is the bias vector; is a scalar, given by the unsupervised k -means clustering algorithm determines the number of topic words that appear in the representative data set; Indicates that the data set k The weight vector of candidate topic words, each dimension element in the vector represents the probability that the training text data belongs to the relevant topic word; is the topic word vector matrix, using k -means clustering algorithm k The characteristic representation of the cluster center is used to represent the matrix Initialize; The reconstructed feature representation of the training text data is a linear combination of the topic word vectors.
6. The review topic identification method based on unsupervised word reconstruction according to claim 1 is characterized in that: In S3, the reconstruction process is trained by optimizing the objective function, with the goal of reconstructing the text features obtained after the attention weighted operation into a feature representation that incorporates the keyword information.
7. The review topic identification method based on unsupervised word reconstruction according to claim 1 is characterized in that: Optimizing the objective function for: Among them, the optimization objective function consists of two parts: is the reconstruction loss term, As a regular term, it is used to ensure the diversity of topic word vectors. is a hyperparameter used to control the regularization term In overall loss The weight of is the reconstruction loss term, where represents the entire training data set, It is a dataset The Input text; represents the number of negative samples randomly sampled from the training set, where j Features of negative samples The detailed calculation process is: , that is, for j Word features corresponding to all words in the negative sample Sum and then average; and Both represent inner product operations; is the regularization term, where is through the theme word vector matrix After row standardization, is an identity matrix; Represents column and norm operations on matrices.
8. A review topic identification system based on unsupervised word reconstruction, characterized in that: include: An encoding module is used to obtain training text data and encode the training text data to obtain a vector matrix; A weighting module is used to obtain text features of training text data by performing attention weighting operations on the vector matrix; The reconstruction module is used to reconstruct the text features after dimensionality reduction calculation to obtain the linear combination of the topic word vectors; The classification module is used to classify the linear combination of topic word vectors into corresponding topics and select the result with the highest probability as the final topic word.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the review topic identification method based on unsupervised word reconstruction as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the review topic identification method based on unsupervised word reconstruction as described in any one of claims 1 to 7 are implemented.