Scientific data set quotation sentiment classification method
By constructing a deep learning model that integrates keyword weights and a sentiment dictionary, the problem of misclassification of sentiment in cited texts by deep learning methods is solved, achieving higher classification accuracy and efficiency, and is suitable for sentiment analysis of scientific datasets.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2026-03-10
AI Technical Summary
Existing deep learning methods struggle to accurately locate key information in complex cited texts, leading to misjudgments in sentiment classification. Furthermore, machine learning methods require a large amount of training data and expert feature engineering, consuming significant human and material resources.
A deep learning model integrating keyword weights and a sentiment dictionary is constructed, including a text embedding layer, a sentiment word embedding layer, a keyword weighting layer, and a sentiment classification layer. Embedding vectors are obtained through a pre-trained language model BERT, and sentiment classification is performed by combining a domain sentiment dictionary and a keyword dictionary.
It improves the accuracy and efficiency of sentiment classification, reduces the dependence on training data, and enhances the model's precision, recall, and F1 score, especially showing significant advantages in citation sentiment analysis of scientific datasets.
Smart Images
Figure CN121636696A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a method for classifying the sentiment of scientific dataset citations by integrating keyword weights and a sentiment dictionary. Background Technology
[0002] Sentiment classification primarily employs three methods: sentiment lexicon, machine learning, and deep learning. Citation sentiment classification also utilizes these three approaches. Text sentiment classification based on sentiment lexicons extracts sentiment words from the text and calculates corresponding sentiment scores to determine the sentiment of a sentence. While sentiment lexicon methods do not require complex data labeling, they are highly dependent on the sentiment lexicon, and the classification results are significantly influenced by it. With the development of machine learning, it has been applied to sentiment classification, using methods such as Bayesian methods, support vector machines, and conditional random fields to build sentiment classification models. Compared to sentiment lexicon methods, machine learning methods offer significant improvements in performance. However, machine learning methods require large amounts of training data and expert feature engineering, consuming substantial human and material resources. In recent years, deep learning has developed rapidly, demonstrating powerful performance in many fields. Deep learning networks such as RNNs, LSTMs, and BERT can automatically extract text features to fit complex data, thereby greatly improving the accuracy of sentiment classification.
[0003] The sentiment of cited texts is characterized by its subtle expression and difficulty in analysis. Existing deep learning methods cannot accurately locate key information in complex sentence environments. If there is too much interference in the cited text, the model will have difficulty judging the sentiment of the text and will make misjudgments. Summary of the Invention
[0004] To address the problems existing in the prior art, the purpose of this invention is to provide a method for classifying the sentiment of scientific dataset citations by integrating keyword weights and a sentiment dictionary.
[0005] The technical solution of this invention is as follows:
[0006] A method for citation sentiment classification of scientific datasets, comprising the following steps:
[0007] 1) Label each cited text in the scientific dataset to obtain a training sample; construct a keyword dictionary corresponding to the scientific dataset, the keyword dictionary including the name of the scientific dataset and the names of its sub-datasets; construct a deep learning model, including a text embedding layer, a sentiment word embedding layer, a keyword weighting layer and a sentiment classification layer;
[0008] 2) Train the deep learning model using the training samples, wherein,
[0009] 21) The text embedding layer will use the pre-trained language model BERT to obtain the embedding vector H of the training samples;
[0010] 22) The sentiment word embedding layer matches the words in each training sample with the words in the domain sentiment dictionary. It concatenates the matched positive sentiment words to obtain the corresponding positive sentiment word sentence P, and concatenates the matched negative sentiment words to obtain the corresponding negative sentiment word sentence N. The positive sentiment word sentence P and the negative sentiment word sentence N are then input into the pre-trained language model BERT to obtain the corresponding positive sentiment word embedding vector H. P and negative sentiment word embedding vector H N ;
[0011] 23) The keyword weight layer determines the position of the keyword appearing in each training sample according to the keyword dictionary, and then assigns weights to each word in the corresponding training sample according to the position of each word segment in the training sample from the keyword, generating the keyword weight vector Q of the corresponding training sample; then calculates the keyword weight fusion vector V = Q*H + H corresponding to the training sample;
[0012] 24) The embedding vector H of the training samples and the embedding vector H of the positive sentiment words in the sentiment classification layer. P , negative sentiment word embedding vector H N The fusion vector E corresponding to the training sample is obtained by integrating the keyword weight fusion vector V;
[0013] Then the fusion vector E is input into the fully connected layer for sentiment classification prediction;
[0014] 25) Based on the set loss function The deep learning model is optimized.
[0015] Where N is the number of training samples, K represents the total number of categories, and y ik This represents the label value of the i-th training sample belonging to the k-th category. This represents the probability that the i-th training sample belongs to the k-th class.
[0016] 3) Input a text to be predicted from the scientific dataset into the optimized deep learning model to predict the sentiment category of the text to be predicted.
[0017] Furthermore, each evaluation of the scientific dataset and its context are treated as a citation text.
[0018] Furthermore, the citation text is labeled with: negative sentiment, neutral, and positive sentiment.
[0019] Furthermore, a domain sentiment dictionary for sentiment classification of scientific dataset citations is constructed based on a sentiment lexicon ontology. The method is as follows: the citation text of the scientific dataset is segmented into words, stop words are removed, and a data dictionary is constructed; the similarity between words in the data dictionary and words in the sentiment lexicon ontology is calculated, and when the similarity score exceeds a set threshold, the corresponding words are added to the domain sentiment dictionary.
[0020] Furthermore, the domain sentiment dictionary includes a domain positive sentiment dictionary and a domain negative sentiment dictionary.
[0021] Furthermore, the keyword weight layer assigns a larger weight to each keyword and its M adjacent word segments in the training sample, and a smaller weight to other word segments in the training sample; then, it generates the keyword weight vector Q of the training sample based on the weight of each word segment in the training sample.
[0022] Furthermore, if a word belongs to the adjacent word segments of two or more keywords M, the weights assigned to that word are summed up to obtain the final weight of that word.
[0023] Furthermore, the fusion vector
[0024] A server is characterized by comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods described above.
[0025] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.
[0026] This technical solution first constructs a deep learning model, which is mainly divided into four parts: text embedding layer, sentiment word embedding layer, keyword weighting layer, and sentiment classification layer.
[0027] This document describes the construction of a self-built scientific dataset citation dataset, using the "Geospatial Data Cloud" dataset as an example. The dataset was constructed as follows: A full-text precise search was performed on CNKI (China National Knowledge Infrastructure) using the keywords "geospatial data cloud" and "gscloud.cn". 300 articles were randomly selected, and full-text PDF files were obtained. These PDF files were then converted to Word documents, and the text and tables within the Word documents were extracted. Citations were extracted from the obtained text using regular expression matching based on relevant keywords. Considering that evaluations of the dataset may not only exist within the citations but also in the immediate context, three sentences—the sentence containing the keyword and one sentence before and after it—were extracted as the citation context. In other words, each citation text includes multiple sentences. After extracting the citation context, the data was manually annotated by professionals. A total of 1355 cited texts were obtained, filtered, and labeled using this method. Of these, 63 were negative, 1086 were neutral, and 206 were positive. We divided the data into training, validation, and test sets in a 6:2:2 ratio, with 811, 271, and 273 texts respectively, ensuring that the ratio of each sentiment polarity was also 6:2:2 in the training, validation, and test sets. The proportion of positive and negative sentiment texts in the training data was too small. To make the training dataset more balanced, we performed data augmentation. Specifically, we used synonym replacement and random deletion to double the number of positive and negative sentiment texts in the training dataset, expanding them to 619 and 160 texts respectively.
[0028] 1. Text Embedding Layer
[0029] The text embedding layer will use the pre-trained language model BERT to obtain the vector representation of the sentence. For a cited text {x1,..,x...} n BERT adds special markers [CLS] and [SEP] to the text. [CLS] represents the sentence classification marker, primarily guiding the model to perform text classification tasks. When classifying input text, the hidden representation of the [CLS] marker is used to generate predictions for classification labels. The hidden representation of the [CLS] marker contains the semantic information of the entire input text, making it suitable for classification tasks. [SEP] represents the sentence separation marker, used to separate different text segments, especially when the input text consists of multiple sentences or text segments. The presence of the [SEP] marker helps the model understand the structure of the text and distinguish different text segments. This invention uses BERT's [CLS] vectors for text classification. For a cited text {x1,...,x...} n The complete input sequence to BERT is X = {x} cls ,x1,..,x n,x seq The text embedding vector is obtained as shown in Formula 1:
[0030] h cls ,h1,...,h n ,h seq =BERT(x cls ,x1,..,x n ,x seq (1)
[0031] Where H = h cls ,h1,...,h n ,h seq It means X = {x} cls ,x1,...,x n ,x seq The vector output after BERT, H, is the embedding vector of the scientific dataset citations.
[0032] 2. Emotional word embedding layer
[0033] Before embedding sentiment terms, a domain sentiment dictionary needs to be pre-constructed. This invention, based on the Dalian University of Technology sentiment lexicon ontology, extends the domain sentiment dictionary required for citation sentiment classification in scientific datasets to construct the necessary domain sentiment dictionary. The construction process is as follows: all data in the scientific dataset citation dataset is segmented, stop words are removed to form a data dictionary. Using the open-source synonym toolkit Synonyms and the pre-trained model BERT, the similarity between words in the data dictionary and words in the Dalian University of Technology sentiment lexicon ontology is calculated. When a word's similarity score in both Synonyms and BERT exceeds 0.8, it is considered a domain sentiment term required by this invention and added to the domain sentiment dictionary. For ease of subsequent processing, this invention divides the domain sentiment dictionary into a domain positive sentiment dictionary and a domain negative sentiment dictionary.
[0034] During training, the words in the training samples of the training set are matched with those in the domain sentiment lexicon. The matched positive sentiment words are then concatenated to form new positive sentiment word sentences P = {p1, p2, ..., p...} lp The matched negative sentiment words are concatenated into a new negative sentiment word sentence N = {n1, n2, ..., nn}. ln}; where l p This indicates the number of times a word in a sentence matches a positive sentiment word in the domain's positive sentiment dictionary. nThis indicates the number of matches between words in a sentence and negative sentiment words in the domain's negative sentiment dictionary. Positive sentiment words are presented in the form of "development, suitability, improvement, contribution," and negative sentiment words are concatenated in the same format to form negative sentiment word sentences. Input P and N into BERT to obtain positive and negative sentiment word embeddings:
[0035]
[0036] Where H p =h p cls ,h p 1,...,h p lp ,h p seq Statements expressing positive emotions P = {p1, p2, ..., p lp The output after BERT, H n =h n cls ,h n 1,...,h n ln ,h n seq Statements expressing negative emotions N = {n1, n2, ..., n} ln The output after BERT processing.
[0037] 3 Keyword Weighting Layer
[0038] The keyword weighting layer is used to assign greater weight to keywords. Constructing the keyword attention layer first requires a keyword dictionary. The keyword dictionary in this invention consists of words related to scientific datasets. The dataset name is a predefined combination of the names of the scientific datasets to be processed and related words to their sub-datasets. For example, the "Geospatial Data Cloud" scientific dataset includes sub-datasets such as "LANDSAT series data," "MODIS land standard products," and "DEM digital elevation data." "LANDSAT," "MODIS," and "DEM" are defined as keywords. Based on the keyword dictionary, the location of keywords is identified, and words near the keywords are assigned more weight.
[0039] This invention adjusts the weight of words near a keyword by using a keyword weight box. The size of the keyword weight box is adjustable; in this invention, the size is set to 10 characters vertically, i.e., 20 words adjacent to the keyword on the left and right. A keyword weight vector Q is obtained through the keyword weight box. The specific calculation method for the keyword weight vector Q is as follows: words within the keyword weight box are assigned a weight of 1, and this weight can be accumulated; that is, if the same word appears in two keyword weight boxes simultaneously, its weight is 2. Then, the keyword weight fusion vector V is calculated from the weight vector Q using the following formula:
[0040] V=Q*H+H (4)
[0041] 4. Sentiment Classification Layer
[0042] For a given cited text, the model obtains H from the text embedding layer and H from the sentiment word embedding layer. P H N The V obtained from the keyword weight layer is combined by concatenation to obtain the fusion vector E:
[0043]
[0044] Then, the fused vector E is input into a fully connected layer, which performs sentiment classification on the cited text.
[0045] p(y|x)=softmax(WE) (6)
[0046] Where p(y|x) is the probability that the cited text x belongs to the category, W is the weight matrix of the fully connected layer, and softmax() is the activation function. The loss function is calculated as follows:
[0047]
[0048] Where N represents the number of training samples, K represents the total number of categories, and y ik This represents the label value of the i-th training sample belonging to the k-th category. This represents the probability that the i-th training sample belongs to the k-th class.
[0049] The optimized deep learning model inputs a text to be predicted from a scientific dataset to predict its sentiment category. Table 1 shows examples of the model's classification results, and Ours describes the method of this invention. BERT classified sentence 1 as negative because it only saw "uneven distribution and varying sizes," but failed to recognize that this did not describe the scientific dataset "modis," thus misclassifying it. Our model, however, focused on the fact that "this paper uses modis data for crop types," correctly classified the sentence. In sentence 2, BERT classified it as neutral, while our model, by leveraging the sentiment lexicon to capture words like "abundant" and "free," classified it as positive, consistent with the correct result. These two examples demonstrate that our two modules are not only theoretically feasible but also effective in practical applications.
[0050] Table 1 Sample Display
[0051]
[0052]
[0053] Compared with the prior art, the positive effects of the present invention are as follows:
[0054] This invention combines a key dictionary to construct a keyword weight layer and a domain sentiment dictionary. It then uses keywords and domain sentiment words to assist in sentiment analysis. Experimental results on the constructed scientific citation dataset are shown in Table 2. The results indicate that compared to the baseline model, this model improves precision, recall, and F1 score by 14.6%, 12.4%, and 13.7%, respectively, providing a reliable basis for citation sentiment analysis of scientific datasets and the evaluation of the academic influence of scientific datasets.
[0055] Table 2 Overall Experimental Results
[0056] Model Macro_P Macro_R Macro_F1 Accuracy BiLSTM 56.3 48.4 51.1 79.0 BiGRU 60.8 50.8 54.1 80.1 Xlnet 60.0 63.2 58.7 81.6 TextCNN 67.1 60.6 63.1 85.2 BERT 71.8 75.6 73.0 89.3 (SC_KP) 82.2 85.0 83.0 92.3 w / o Emotional Dictionary 74.3 78.7 75.4 90.8 w / o keywords 72.2 81.4 75.1 90.4 Attached Figure Description
[0057] Figure 1 This is a flowchart of the method of the present invention.
[0058] Figure 2 This is a model diagram of the present invention. Detailed Implementation
[0059] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.
[0060] Example 1:
[0061] This document describes the construction of a self-built scientific dataset citation dataset, using the "Geospatial Data Cloud" dataset as an example. The dataset was constructed as follows: A full-text precise search was performed on CNKI (China National Knowledge Infrastructure) using the keywords "geospatial data cloud" and "gscloud.cn". 300 articles were randomly selected, and full-text PDF files were obtained. These PDF files were then converted to Word documents, and the text and tables within the Word documents were extracted. Citations were extracted from the obtained text content using regular expression matching based on relevant keywords. Considering that evaluations of the dataset may not only exist in the citations but also in the immediate context, it was necessary to extract the sentences containing the keywords, along with the three sentences preceding and following them, as the citation context. After extracting the citation context, the data was manually annotated by professionals. A total of 1355 cited texts were obtained, filtered, and labeled using this method. Among them, there were 63 negative sentiment texts, 1086 neutral sentiment texts, and 206 positive sentiment texts. We divided the data into training, validation, and test sets in a 6:2:2 ratio, with 811, 271, and 273 texts respectively, ensuring that the ratio of each sentiment polarity in the training, validation, and test sets was also 6:2:2. The proportion of positive and negative sentiment texts in the training data was too small. To make the training dataset more balanced, we performed data augmentation. Specifically, we used synonym replacement and random deletion to double the number of positive and negative sentiment texts in the training dataset, expanding them to 619 and 160 texts respectively. Using a sentiment word embedding layer, we obtained a domain sentiment dictionary, acquiring 81 domain negative sentiment words and 73 domain positive sentiment words. The obtained sentiment dictionary is shown in Table 3.
[0062] Table 3 Domain-Specific Emotional Keywords
[0063]
[0064] BERT is used to embed scientific datasets, obtaining vector forms of the text. A keyword weighting layer is then applied to assign more weight to keywords. In this example, the keywords are "Landsat," "LANDSAT," and "MYD." For instance, in the sentence "Because this paper uses modis data for crop type identification, with a spatial resolution of 250m and a temporal resolution of 8 days, the distribution of plots in Shaanxi Province is uneven and varies in size...", "modis" is a keyword and will be given greater weight when passing through the keyword weighting layer.
[0065] The four vectors—text embedding, text embedding after keyword weighting, positive sentiment word embedding, and negative sentiment word embedding—are concatenated and input into a linear classification network for sentiment classification.
[0066] The results of this embodiment are shown in Table 1. The model of this invention exhibits significant advantages over the baseline model BERT. In terms of macro-mean precision, the model of this invention achieves a 14.6% improvement; in terms of macro-mean recall, it improves by 12.4%; and in terms of macro-mean F1 score, it improves by 13.7%, while also achieving a 3% improvement in classification accuracy. Therefore, it can be concluded that the model proposed in this invention achieves significant performance advantages on this dataset, particularly in macro-mean precision, macro-mean recall, and macro-mean F1 score. These results demonstrate that the model of this invention has better performance on classification tasks, providing strong support for research and applications in related fields.
[0067] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.
Claims
1. A scientific dataset citation sentiment classification method, comprising the steps of: 1) labeling each citation text of a scientific dataset to obtain a training sample; constructing a keyword dictionary corresponding to the scientific dataset, the keyword dictionary including the scientific dataset name and its sub-dataset name in the scientific dataset; and constructing a deep learning model including a text embedding layer, a sentiment word embedding layer, a keyword weight layer, and a sentiment classification layer; 2) training the deep learning model using the training sample, wherein, 21) the text embedding layer obtains an embedding vector H of the training sample using a pre-trained language model BERT; 22) The sentiment word embedding layer matches the words of each training sample with the words in the domain sentiment dictionary, splices the matched positive sentiment words to obtain the positive sentiment word sentence P corresponding to the training sample, and splices the matched negative sentiment words to obtain the negative sentiment word sentence N corresponding to the training sample; the positive sentiment word sentence P and the negative sentiment word sentence N are respectively input into the pre-trained language model BERT to obtain the corresponding positive sentiment word embedding vector H P and the negative sentiment word embedding vector H N ; 23) the keyword weight layer determines the position of the keyword appearing in each training sample according to the keyword dictionary, and then assigns weights to each word in the training sample according to the position of the keyword, to generate a keyword weight vector Q of the corresponding training sample; and then calculates the keyword weight fusion vector V = Q*H + H of the training sample; 24) The emotion classification layer integrates the embedding vector H of the training sample, the positive sentiment word embedding vector H P , the negative sentiment word embedding vector H N and the keyword weight fusion vector V to obtain the fusion vector E corresponding to the training sample; then input the fusion vector E into a fully connected layer for sentiment classification prediction; 25) according to a set loss function optimizing the deep learning model; wherein N is the number of training samples, K represents the total number of categories of classification, y ik represents the label value of the i-th training sample belonging to the k-th category, represents the probability of predicting that the i-th training sample belongs to the k-th category; 3) inputting a to-be-predicted citation text of a scientific dataset into the optimized deep learning model to predict the sentiment category of the to-be-predicted citation text.
2. The method of claim 1, wherein, Each review of a scientific dataset and its context is taken as a citation text.
3. The method of claim 2, wherein, The labeling of the citation text includes negative sentiment, neutral, and positive sentiment.
4. The method according to claim 1 or 2 or 3, characterized in that, A domain sentiment dictionary for scientific dataset citation sentiment classification is constructed based on a sentiment lexicon ontology, the method comprising: segmenting the citation text of a scientific dataset, removing stop words, and constructing a data dictionary; calculating the similarity between the words in the data dictionary and the words in the sentiment lexicon ontology, and when the similarity score exceeds a set threshold, adding the corresponding words to the domain sentiment dictionary.
5. The method of claim 4, wherein, The domain sentiment dictionary includes a domain positive sentiment dictionary and a domain negative sentiment dictionary.
6. The method according to claim 1 or 2 or 3, characterized in that, The keyword weight layer assigns a larger weight to each keyword and its adjacent M words in the training sample, and a smaller weight to other words in the training sample; and then generates a keyword weight vector Q of the training sample according to the weight of each word in the training sample.
7. The method of claim 6, wherein, If a word belongs to two or more keywords within M adjacent words, the weight assigned to the word is accumulated as the final weight of the word.
8. The method of claim 1, wherein, The fusion vector 9. A server, characterized by A computer program is stored in a memory and executed by a processor, the computer program comprising instructions for executing the method of any one of claims 1 to 8.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 8.