Public opinion tendency identification method, system and device based on weak supervision comparative learning and storage medium

Abstract

The invention discloses a public opinion tendency recognition method, system and device based on weak supervision comparative learning and a storage medium, and the method comprises the following steps: S1, building a classification model, and dividing a weak annotation training text into a positive set with positive emotion polarity and a negative set with negative emotion polarity; s2, using a BERT encoder to process the weak labeling training text into a feature vector; s3, performing nonlinear transformation and aspect extraction on the feature vectors, and performing feature fusion; s4, using supervised comparative learning to train an aspect extraction process; s5, using a labeled training text to supervise and finely adjust the trained classification model; and S6, inputting the text data into the fine-tuned classification model, and identifying whether the sentiment polarity of each text is positive or negative. The emotion polarity of the weakly labeled data can be accurately identified, and more valuable information can be obtained from public opinion information.

Description

Technical Field

[0001] The present invention belongs to the field of public opinion recognition, and relates to a method, system, device and storage medium for identifying public opinion tendency based on weakly supervised contrastive learning. Background Art

[0002] The performance of most existing deep learning-based sentiment classification methods relies on large-scale manually annotated data, but manually annotating data requires a huge amount of manpower and material resources. In data-scarce scenarios, limited annotated data makes it difficult to fully train large-scale language models, which restricts the performance of deep learning methods. At present, the lack of annotated data has become a bottleneck problem restricting aspect-level sentiment classification algorithms. The interactive function of social platforms provides a kind of weakly annotated information for social comments, but this annotated information is inconsistent with the actual content of the comments, and the noise problem caused by mislabeled data will affect the training process of the model. In addition, most research works usually regard aspect-level sentiment classification and aspect extraction as two independent single tasks, without considering the correlation between the two tasks. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method, system, device and storage medium for identifying public opinion tendencies based on weakly supervised contrastive learning, which can accurately identify the sentiment polarity of weakly labeled data and obtain more valuable information in public opinion information.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: A method for identifying public opinion tendency based on weakly supervised contrastive learning includes the following processes: S1, obtaining training text data, dividing the training text data into weakly annotated training text and annotated training text, establishing a classification model, using the weakly annotated training text as training data for the classification model, and dividing the weakly annotated training text into a positive set with positive sentiment polarity and a negative set with negative sentiment polarity; S2, uses BERT encoder to process weakly annotated training text into feature vectors; S3, after the feature vectors are subjected to nonlinear transformation and aspect extraction, feature fusion is performed; S4, training the aspect extraction process using supervised contrastive learning; S5, supervised fine-tuning of the trained classification model using annotated training text; S6, input the text data into the fine-tuned classification model to identify the sentiment polarity of each text as positive or negative.

[0005] Preferably, in S1, the training data is discretized into a positive set and a negative set through a threshold.

[0006] Preferably, in S2, the training data is processed into a text sequence, and each input text sequence is encoded using a BERT encoder to generate a feature vector.

[0007] Preferably, in S3, feature fusion is performed in a nonlinear manner.

[0008] Preferably, in S4, a training objective function based on contrast loss is used to train aspect extraction, and the training goal is to make samples from the same training subset close to each other, and samples from different training subsets move away from each other, while constraining noise examples not to move to the wrong category.

[0009] Preferably, in S5, the annotated training text is input into the classification model, and the predicted value of the annotated training text is output.

[0010] Furthermore, in the supervised fine-tuning process, the supervised contrastive learning term is used as the regularization term. According to the cross entropy loss of the predicted value and the true value of the labeled training text, the goal is to fully learn the contrast pattern between similar and non-similar samples in the labeled training text, and the loss function is set to perform supervised fine-tuning on the classification model.

[0011] A public opinion tendency identification system based on weakly supervised contrastive learning, comprising: A training data processing module is used to obtain training text data, divide the training text data into weakly annotated training text and annotated training text, establish a classification model, use the weakly annotated training text as training data for the classification model, and divide the weakly annotated training text into a positive set with positive sentiment polarity and a negative set with negative sentiment polarity; The encoding module is used to process the weakly labeled training text into feature vectors using the BERT encoder; The feature fusion module is used to perform feature fusion after the feature vectors are subjected to nonlinear transformation and aspect extraction respectively; A training module for training the aspect extraction process using supervised contrastive learning; The fine-tuning module is used to supervise the training classification model using labeled training text; The recognition module is used to input text data into the fine-tuned classification model to identify the sentiment polarity of each text as positive or negative.

[0012] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for identifying public opinion tendency based on weakly supervised contrastive learning are implemented.

[0013] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for identifying public opinion tendency based on weakly supervised contrastive learning.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention adopts the transfer learning paradigm of "pre-training and then fine-tuning". In the pre-training stage, taking massive weakly labeled data as the research object, on the basis of the unsupervised topic recognition method, by introducing the idea of ​​pseudo-label generation, the form of the contrast loss function is improved, so that it can guide the deep model to better learn the rich inter-class and intra-class contrast patterns in massive weakly labeled data. At the same time, multiple sampling of non-similar samples makes the noise data naturally diluted, thereby minimizing the negative impact of noise data in weakly labeled data and realizing the noise-resistant learning of the model. In the fine-tuning stage, a multi-task objective function for small sample scenarios is designed. The knowledge of the pre-trained model is inherited through supervised contrast learning items, which can further improve the learning effect of the model on downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the topic-level sentiment classification model TL-BERT based on transfer learning of the present invention; Figure 2 It is a schematic diagram of the aspect extraction module PL-BERT of the present invention; Figure 3 This is a graph showing the results of a small sample comparison experiment in Example 2 of the present invention. DETAILED DESCRIPTION

[0016] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0017] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs. The terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" used in this article includes any and all combinations of one or more related listed items. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0019] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0020] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0021] like Figure 1 As shown in the figure, the public opinion tendency identification method based on weakly supervised contrastive learning of the present invention adopts the topic-level sentiment classification model TL-BERT based on transfer learning (the overall structure diagram is shown in Figure 1 As shown in Figure 2, this classification model adopts the transfer learning paradigm of “pre-training and then fine-tuning”, which includes the following training steps: First, a weakly supervised contrastive learning strategy is designed to pre-train massive weakly labeled data, aiming to learn an excellent embedding space and thus obtain a good feature representation of the input text; then, a multi-task objective function is designed for the fine-tuning stage to better inherit the knowledge of the pre-trained model through supervised contrastive learning items.

[0022] Example 1 Step 1: Establish a topic-level sentiment classification model TL-BERT based on transfer learning, and use a weakly supervised contrastive learning strategy to pre-train massive weakly labeled data.

[0023] Step 1-1: Obtain training text data, divide the training text data into weakly annotated training text and annotated training text, and set new data weak label assignment rules for specific weak supervision scenarios. Weakly annotated training text can set the threshold discretization into two categories, and emoticons can set the correspondence between label symbols and emotional polarity labels according to the intuitive tendency conveyed by the symbols. Other weak supervision scenarios can set rules to obtain weak labels according to specific circumstances, such as likes and dislikes in community comments. Taking the two-category scenario as an example, after the label assignment is completed, the weakly annotated training text is divided into 2 subsets, namely the positive set representing positive emotional polarity and the negative set representing positive emotional polarity. and the negative set representing negative sentiment polarity .

[0024] Step 1-2: Use the data preprocessing module to obtain the input sequence of text for the weakly labeled dataset after assigning labels, and then use the BERT encoder to encode each input text sequence to generate a high-level feature representation. For example, the text sequence Input into the BERT encoder to obtain its semantic features , here It is the feature vector corresponding to the symbol [CLS] after BERT encoding.

[0025] Step 1-3: Get Input to the nonlinear transformation layer, after nonlinear transformation, As input sequence The high-level feature representation of is as follows: , in and is the weight matrix.

[0026] Step 1-4: Feature fusion layer and As input, It is the aspect extraction module PL-BERT in the WML framework (its structure is as follows Figure 2 shown). Focus on learning input text The overall semantic information of More attention The keyword information in the fusion vector It can obtain more fine-grained emotional information. The feature fusion is performed in a nonlinear way, and the calculation process is as follows: , in, and All are parameter matrices; and The dimensions of are all d-dimensional, The dimension of is u-dimensional, and , dimensionality reduction operation makes it easier for the neural network to integrate fine-grained emotional information with category distinction. The embedding layer does not involve the specific calculation process and is only used to store the input sequence obtained in the previous layer. The final eigenvector of , thus establishing a weakly supervised sentiment classification module.

[0027] Step 1-5: There is a certain proportion of noise samples in the weakly labeled data. It is necessary to design a noise-resistant training strategy to weaken the negative impact of the noise data. The training goal needs to ensure that samples from the same training subset are close to each other, and samples from different training subsets are far away from each other. In this changing process, the training goal also needs to constrain the noise examples from moving to the wrong category. To this end, a novel training objective function based on contrast loss is proposed , this loss function can make full use of the rich contrast patterns between weakly labeled data, making the contrast loss more suitable for weakly supervised scenarios.

[0028] The weakly-supervised multi-task learning framework (WML) uses topic-level sentiment classification as the main training task. During the pre-training process of the weakly supervised sentiment classification module, the aspect extraction process is also trained using supervised contrastive learning. Therefore, the multi-task loss of the WML framework in the pre-training stage is as follows:

[0029] in, It is a hyperparameter used to adjust the WML framework (PL-BERT loss in aspect extraction module) and (Supervised Contrastive Learning Loss) The weight between the two losses. The overall loss of the WML framework during the pre-training phase yes and is a linear weighted combination of .

[0030] In the second step, supervised fine-tuning, a classification layer is added on top of the embedding layer to form the final classification model.

[0031] Step 2-1: Migrate the parameters obtained in the pre-training phase to the corresponding layers in the classification model, and randomly initialize the parameters in the classification layer.

[0032] Step 2-2: Labeled training text Input into the classification model and obtain through the classification layer The predicted value of .

[0033] The calculation formula is: , in is the input sequence The characteristic vector of is the parameter matrix, b is the bias term, Indicates the model's response to the current input text The predicted value of .

[0034] Step 2-3: In order to better solve the problem of insufficient model training caused by lack of data, supervised contrastive learning is used as a regularization term, the purpose is to fully learn the contrast pattern between similar and non-similar samples in the labeled training text data. The loss function is as follows: , The above formula is the optimization objective function of the weakly supervised sentiment classification module in the multi-task fine-tuning stage. It consists of two parts: represents the supervised contrastive learning loss, Represents the cross entropy loss between the predicted value and the true value of the labeled training text, is a hyperparameter used to adjust the weights of the two.

[0035] Step three: Use the trained classification model to input the text data into the classification model to identify whether the sentiment polarity of each text is positive or negative.

[0036] Experiments have shown that, thanks to the powerful representation learning ability of contrast pre-training and the good migration ability of pre-trained models, fine-tuning the model using only a very small amount of labeled data in the fine-tuning stage can significantly improve the classification performance of the deep model, which helps the model greatly reduce the labeled samples and training costs. The supervised contrast regularization introduced in this method better inherits the knowledge of the pre-training model, and has better generalization ability and more stable performance improvement in small sample sentiment classification tasks.

[0037] Economic benefits: Companies around the world are using social media analysis technology to gain insights into consumer perceptions and improve their services and products. However, there is not much data on the Internet that can clearly express consumer insights, and there is more weakly labeled information such as ratings. The algorithm proposed in this project solves this problem, allowing companies and institutions to obtain more valuable information from public opinion information, formulate ever-changing marketing strategies, attract, convert, upsell, cross-sell products, and target potential target customers, ultimately increasing receivables.

[0038] Example 2 The following is an explanation of the public opinion tendency identification method based on weakly supervised contrastive learning described in the present invention using actual data: The experimental data uses two datasets: Amazon product review dataset and Twitter review dataset. The Amazon dataset contains the review corpus of products in digital cameras, cell phones and laptops3, consisting of 1,136,271 weakly annotated sentences with user ratings and 11,754 manually annotated sentences. The Twitter dataset consists of 1,100,000 weakly annotated tweets with emoticons and 8,047 annotated data. In the experiment, the weakly annotated data is regarded as a pre-training set, and the annotated dataset is randomly divided into a training set (50%), a validation set (20%) and a test set (30%). Tables 1 and 2 show the statistics of the Amazon product review dataset and the Twitter review dataset, respectively. In addition, in order to verify the generalization ability of the WML method on Chinese datasets and multi-classification datasets, the same verification experiments were then conducted on the Douban movie review dataset, the Sina Weibo comment dataset and the SST-5 dataset. The Douban movie review dataset contains 600,000 weakly annotated sentences with ratings and 10,000 strongly annotated sentences. The Sina Weibo comment dataset contains 360,000 weakly annotated sentences with emoticons and 10,000 strongly annotated sentences. The SST-5 dataset contains 11,855 annotated sentences, and its sentiment polarity is divided into 5 categories: very positive (label 4), positive (label 3), neutral (label 2), negative (label 1), and very negative (label 0). The SST-5 dataset is randomly divided into a pre-training set (50%), a fine-tuning set (20%), and a test set (30%). 20% of the data is randomly extracted from the pre-training set and its true label is changed to other labels. For all the above pre-training sets and training sets, the UAM-BERT method is first used to train them to obtain the aspect recognition model of the corresponding dataset; then the WML framework is used for subsequent tasks.

[0039] Table 1 Statistics of Amazon product review dataset

[0040] Table 2 Statistics of Twitter comment dataset

[0041] 1. Experimental Details As shown in Table 1 and Table 2, after counting the length of sentences in the labeled datasets, the longest lengths of sentences in the two datasets are 121 and 47 respectively. In the experiment, the batch_size value is set to 32. Table 3 shows the temperature coefficient , , loss weighted hyperparameters α, β, , , The value of . We conducted multiple experiments on the values ​​of , α, and β and found that the performance of the model was more stable when τ was 0.5 and the loss weighted hyperparameters α and β were both 0.2. The model uses the Adam optimizer and the learning rate is set to .

[0042] Table 3 Hyperparameter values

[0043] 2. Comparative Experiments on Similar Algorithms Table 4 Accuracy of different methods

[0044] The experimental results of the three groups of baseline methods in the dashed box prove that the TL strategy used in the weakly supervised sentiment classification module can effectively improve the classification performance of deep models with different encoding structures (i.e., CNN, BiLSTM, BERT), among which the classification accuracy is improved by up to 3.4%. When training deep models directly on weakly labeled data, the noise data will cause serious interference to the training process that cannot be ignored, thus significantly reducing the performance of each deep model. Therefore, all weak methods have poor results, which indirectly proves the necessity of noise-resistant pre-training methods. The method TL-BERT proposed in this project fully absorbs the sentiment semantic information in weakly labeled data while effectively resisting noise, further learns the contrast pattern between similar and non-similar samples in the fine-tuning stage, and also benefits from the additional contextual information provided by the aspect extraction module. Therefore, it achieves the best classification performance on 5 different datasets, which fully proves the superiority and good generalization ability of the proposed method.

[0045] 3. Small sample comparison experiment In order to evaluate the effectiveness of the proposed method in the extremely small sample scenario, the algorithm was trained on different scales of Amazon labeled data. n% (n=0.5, 1, 1.5, 2, …, 5) of the data was randomly selected from the labeled training set as the training set to train the model. Figure 3As shown in the figure. In all small sample experiments, the performance of the TL-BERT method is better than other methods. Even if only 0.5% (i.e. 32 samples) of data is used as the training set to fine-tune the model, the TL strategy can still significantly improve the performance of deep models using different encoding structures, and the performance is improved by up to 14.5%. When less than 2.5% of the data is used as the training set to fine-tune the model, the performance of the SCFT method drops sharply. The experiment shows that this fine-tuning method is less effective in the case of very small samples. The performance of the BERT method is similar to that of the SCFT method. This is because large-scale model parameters are very easy to overfit on very small amounts of data, resulting in the model being unable to capture the overall distribution of the data in the training set. The performance lines of the other six groups of baseline methods are relatively similar: when more than 3% of the data is used as the training set to fine-tune the model, the performance curve is relatively stable; when less than 1% of the data is used as the training set to fine-tune the model, the performance curve drops sharply. Among the six groups of baseline methods, the BiLSTM and CNN methods perform significantly better, mainly due to the powerful representation learning capabilities of these two deep models. SSWE and SentiWV perform better than SVM and NBSVM, but these two methods are limited by the linear model structure, so their performance is poor.

[0046] 4. Study on anti-noise performance The WCL method used in the TL-BERT framework has a natural noise resistance capability. Anti-noise experiments were conducted using symmetric and asymmetric noise labels of different scales. The Amazon labeled dataset was used for verification. First, the Amazon labeled dataset was divided into a pre-training set (50%), a fine-tuning set (20%), and a test set (30%). For the pre-training set, the correct labels were randomly replaced with opposite labels to generate symmetric and asymmetric noise, respectively. From the experimental results of the symmetric noise labels in Table 5, it can be observed that the accuracy of the model is symmetrically distributed. For the experimental results of the asymmetric noise, it can be observed that the overall accuracy of the model is relatively stable and the fluctuation range is small. From the observation results, it can be seen that in the weak supervision scenario with a fixed noise ratio, the noise labels with unbalanced ratios cannot destroy the robustness of the WCL method under the noise labels.

[0047] Table 5. Experimental results of noise resistance of symmetric and asymmetric noise labels

[0048] The following are device embodiments of the present invention, which can be used to perform method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0049] In another embodiment of the present invention, a public opinion tendency identification system based on weakly supervised contrastive learning is provided. The public opinion tendency identification system based on weakly supervised contrastive learning can be used to implement the above-mentioned public opinion tendency identification method based on weakly supervised contrastive learning. Specifically, the public opinion tendency identification system based on weakly supervised contrastive learning includes a training data processing module, an encoding module, a feature fusion module, a training module, a fine-tuning module and a recognition module.

[0050] Among them, the training data processing module is used to obtain training text data, divide the training text data into weakly annotated training text and annotated training text, establish a classification model, use the weakly annotated training text as training data for the classification model, and divide the weakly annotated training text into a positive set with positive sentiment polarity and a negative set with negative sentiment polarity.

[0051] The encoding module is used to process the weakly labeled training text into a feature vector using the BERT encoder.

[0052] The feature fusion module is used to perform feature fusion on feature vectors after they have undergone nonlinear transformation and aspect extraction.

[0053] The training module is used to train the aspect extraction process using supervised contrastive learning.

[0054] The fine-tuning module is used to perform supervised fine-tuning on the trained classification model using labeled training text.

[0055] The recognition module is used to input text data into the fine-tuned classification model to identify the sentiment polarity of each text as positive or negative.

[0056] In another embodiment of the present invention, a terminal device is provided, the terminal device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGAs), or other processors. GateArray, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of the public opinion tendency identification method based on weakly supervised contrastive learning, including: S1, obtaining training text data, dividing the training text data into weakly labeled training text and labeled training text, establishing a classification model, using the weakly labeled training text as training data for the classification model, and dividing the weakly labeled training text into a positive set with positive sentiment polarity and a negative set with negative sentiment polarity; S2, using a BERT encoder to process the weakly labeled training text into a feature vector; S3, after the feature vector is subjected to nonlinear transformation and aspect extraction respectively, feature fusion is performed; S4, using supervised contrastive learning to train the aspect extraction process; S5, using the labeled training text to supervise fine-tune the trained classification model; S6, inputting the text data into the fine-tuned classification model to identify whether the sentiment polarity of each text is positive or negative.

[0057] In another embodiment, the present invention further provides a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory.

[0058] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the method for identifying public opinion tendency based on weakly supervised contrastive learning in the above-mentioned embodiment; one or more instructions in the computer-readable storage medium are loaded by the processor and the following steps are executed: S1, obtaining training text data, dividing the training text data into weakly annotated training text and annotated training text, establishing a classification model, using the weakly annotated training text as training data for the classification model, and dividing the weakly annotated training text into a positive set with positive sentiment polarity and a negative set with negative sentiment polarity; S2, using a BERT encoder to process the weakly annotated training text into feature vectors; S3, performing feature fusion after the feature vectors are subjected to nonlinear transformation and aspect extraction respectively; S4, using supervised contrastive learning to train the aspect extraction process; S5, using the annotated training text to supervise fine-tune the trained classification model; S6, inputting the text data into the fine-tuned classification model to identify whether the sentiment polarity of each text is positive or negative.

[0059] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0060] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0061] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0062] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0063] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0064] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0065] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0066] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0067] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0068] It should be understood that the above description is for illustration and not for limitation. Many embodiments and many applications beyond the examples provided will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this patent should not be determined with reference to the above description, but rather with reference to the preceding claims and the full scope of equivalents to which such claims are entitled. For the purpose of comprehensiveness, all articles and references, including disclosures of patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the preceding claims is not intended to be a waiver of such subject matter, nor should it be considered that the applicant has not considered such subject matter to be part of the disclosed inventive subject matter.

Claims

1. A method for identifying public opinion tendency based on weakly supervised contrastive learning, characterized in that: The process includes: S1, obtaining training text data, dividing the training text data into weakly annotated training text and annotated training text, establishing a classification model, using the weakly annotated training text as training data for the classification model, and dividing the weakly annotated training text into a positive set with positive sentiment polarity and a negative set with negative sentiment polarity; S2, uses BERT encoder to process weakly annotated training text into feature vectors; S3, after the feature vectors are subjected to nonlinear transformation and aspect extraction, feature fusion is performed; S4, training the aspect extraction process using supervised contrastive learning; S5, supervised fine-tuning of the trained classification model using annotated training text; S6, input the text data into the fine-tuned classification model to identify the sentiment polarity of each text as positive or negative.

2. The method for identifying public opinion tendency based on weakly supervised contrastive learning according to claim 1 is characterized in that: In S1, the training data is discretized into positive and negative sets through thresholds.

3. The method for identifying public opinion tendency based on weakly supervised contrastive learning according to claim 1 is characterized in that: In S2, the training data is processed into text sequences, and the BERT encoder is used to encode each input text sequence to generate a feature vector.

4. The method for identifying public opinion tendency based on weakly supervised contrastive learning according to claim 1 is characterized in that: In S3, a nonlinear approach is used for feature fusion.

5. The method for identifying public opinion tendency based on weakly supervised contrastive learning according to claim 1 is characterized in that: In S4, a training objective function based on contrastive loss is used to train aspect extraction. The training goal is to make samples from the same training subset close to each other, and samples from different training subsets move away from each other, while constraining noise examples not to move to the wrong category.

6. The method for identifying public opinion tendency based on weakly supervised contrastive learning according to claim 1 is characterized in that: In S5, the labeled training text is input into the classification model, and the predicted value of the labeled training text is output.

7. The method for identifying public opinion tendency based on weakly supervised contrastive learning according to claim 6 is characterized in that: In S5, during the supervised fine-tuning process, the supervised contrastive learning term is used as the regularization term. According to the cross entropy loss of the predicted value and the true value of the labeled training text, the goal is to fully learn the contrast pattern between similar and non-similar samples in the labeled training text, and the loss function is set to supervise the fine-tuning of the classification model.

8. A public opinion tendency identification system based on weakly supervised contrastive learning, characterized in that: include: A training data processing module is used to obtain training text data, divide the training text data into weakly annotated training text and annotated training text, establish a classification model, use the weakly annotated training text as training data for the classification model, and divide the weakly annotated training text into a positive set with positive sentiment polarity and a negative set with negative sentiment polarity; The encoding module is used to process the weakly labeled training text into feature vectors using the BERT encoder; The feature fusion module is used to perform feature fusion after the feature vectors are subjected to nonlinear transformation and aspect extraction respectively; A training module for training the aspect extraction process using supervised contrastive learning; The fine-tuning module is used to supervise the training classification model using labeled training text; The recognition module is used to input text data into the fine-tuned classification model to identify the sentiment polarity of each text as positive or negative.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for identifying public opinion tendency based on weakly supervised contrastive learning as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying public opinion tendency based on weakly supervised contrastive learning as described in any one of claims 1 to 7 are implemented.

Images