Social media and artificial intelligence based disease related public opinion analysis system

Abstract

The application discloses a disease-related public opinion analysis system based on social media and artificial intelligence, comprising a data acquisition module, a data preprocessing module and a data analysis module; the data acquisition module is used for collecting social media texts and a keyword list related to diseases, collecting dynamics and messages within a set time; the data preprocessing module screens social media texts related to diseases and removes characters irrelevant to public opinion analysis; the data analysis module constructs and trains a social media text pre-training model, a named entity recognition model and a target entity sentiment recognition model, and predicts the sentiment polarity of the target entity. The application constructs multiple disease-related models, can enhance the adaptability of the public opinion analysis system in the medical field, automatically extracts entity information in social texts, automatically judges the sentiment polarity of disease-related entities, can obtain more fine-grained real-time public opinion analysis results, and provides more accurate public opinion monitoring services.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of natural language processing or public opinion analysis technology, and particularly relates to a disease-related public opinion analysis system based on social media and artificial intelligence. BACKGROUND

[0002] With the rapid progress of Internet technology, more and more people tend to share life dynamics or express views and attitudes on hot topics on social media such as Sina Weibo, Facebook and Twitter. In recent years, in particular, the influence of popular and infectious diseases on people's lives has been enormous, triggering widespread attention and discussion on disease-related topics on social media.

[0003] From the perspective of public health, social media data contains a large amount of disease-related public opinion information with practicality and communicability, so through in-depth analysis of this information, the user's emotional tendency and implied attitude can be mined, which not only helps the health department to carry out real-time monitoring of related diseases, timely track the latest developments of disease development, and master the transmission rules of diseases, but also provides effective reference for the prevention and control measures and policy making of the epidemic.

[0004] Generally, public opinion analysis based on social media and artificial intelligence mainly targets text data, involving text mining and sentiment analysis in natural language processing. The existing disease-related public opinion analysis system mainly has the following problems[1-2]: First, it relies too much on sentiment rules or sentiment dictionaries, which are difficult to update and maintain and have poor generalization; second, the field adaptability is poor, and the natural language processing tools used by existing systems are mostly designed for general formal text, and their performance often decreases significantly on disease-related social media texts with more diverse expressions and imprecise grammar rules, which cannot meet the needs of disease-related public opinion analysis; third, the analysis granularity is rough, and most public opinion analysis systems can only perform sentiment recognition at the sentence level, and cannot analyze the sentiment polarity at the entity level, which greatly limits the precision and practicality of the disease-related public opinion analysis system.

[0005] [1] Yang Kang. Microblog public opinion analysis system design and implementation based on text sentiment[D]. Northwest University, 2018

[0006] [2] Duarte, Natasha, Emma Llanso, and Anna C. Loup. "Mixed Messages? The Limits of Automated Social Media Content Analysis." FAT 106 (2018).

[0007] [3] PEI Qingqi, WANG Yuyan. Network public opinion text information sentiment polarity classification processing system and method [P]. Shaanxi:

[0008] CN111209401A, 2020-05-29.

[0009] [4] WANG Zili, ZANG Dongsong, TANG Wenjie, KANG Yuyu, NIE Li-xiang. A method and device for sentiment polarity analysis of network public opinion [P].

[0010] Beijing: CN109446404B, 2022-04-08. SUMMARY

[0011] In view of the above, the purpose of the present application is to solve the problems of difficult updating and maintenance, poor field adaptability and rough analysis granularity of the current disease-related public opinion analysis system. By combining social media big data and natural language processing technology, a disease-related public opinion analysis system based on social media and artificial intelligence is provided, which can analyze the fine-grained public opinion related to diseases on social media without manually defining complex sentiment rules, thereby obtaining people's views or attitudes towards a certain disease and its changing trend in real time, comprehensively and accurately, and providing effective support for subsequent public opinion early warning, management and decision-making.

[0012] The purpose of the present application is achieved by the following technical solutions: a disease-related public opinion analysis system based on social media and artificial intelligence, which comprises a data acquisition module, a data preprocessing module and a data analysis module.

[0013] Data acquisition module: used for collecting social media text and disease-related keyword table, collecting the dynamic and message published by people within a set time period;

[0014] Data preprocessing module: using the disease-related keyword table to filter and disease-related social media text, removing the characters irrelevant to public opinion analysis from the filtered social media text, and outputting the preprocessed social media text;

[0015] Data analysis module: constructing a social media text pre-training model, including using the preprocessed social media text, incrementally pre-training the pre-training language model in the general field to obtain the social media text pre-training model;

[0016] The pre-training model of the social media text is used to construct a named entity recognition model and a target entity sentiment recognition model; the named entity recognition model and the target entity sentiment recognition model are trained, including randomly sampling the pre-processed social media text, distributing it to the labeling personnel for entity and sentiment labeling, then calculating the consistency of the labeling and feeding back to the labeling personnel for evaluation, after several iterations, the labeled samples meeting the requirements are used as the dataset of the named entity recognition model and the target entity sentiment recognition model, and the dataset is used for model training;

[0017] The trained named entity recognition model is used to extract entities from the pre-processed social media text, and then the extracted results are used as the input of the target entity sentiment recognition model to predict the sentiment polarity of all target entities.

[0018] Further, the data acquisition module acquires social media data and a disease-related keyword table by connecting a data open platform of social media or using a data crawler technology; when acquiring text data using the data open platform of social media, an application needs to be provided to the open platform first, and after authentication, structured data is obtained.

[0019] Further, in the data acquisition module, the disease-related keywords include the name of the disease, the corresponding symptom name, and the name of the treatment drug.

[0020] Further, in the data preprocessing module, the text exclusion criteria of the social media text are one of the following conditions: user forwarded text, text containing URL, and text with a length less than 5; and the inclusion criteria of the sample are at least matching one disease-related keyword.

[0021] Further, in the data preprocessing module, the characters irrelevant to the public opinion analysis are removed from the screened social media text, including removing HTML characters in the text and converting emoticons into strings.

[0022] Further, in the data analysis module, the construction of the social media text pre-training model specifically includes: using two pre-training tasks of a masked language model and next sentence prediction, processing all unlabeled social media texts into the format of the two tasks as pre-training corpus, incrementally pre-training the pre-training language model in the general field to obtain the social media text pre-training model.

[0023] Further, in the data analysis module, the defined entity types include the following seven types: person name, place name, organization name, disease name, symptom name, drug name, and vaccine; among them, the person name, organization name, drug name, and vaccine are selected as target entity types, and each target entity type contains three sentiment polarities: positive, neutral, and negative.

[0024] Further, the model training process using the data set is as follows:

[0025] The data set of the named entity recognition model and the target entity sentiment recognition model is further divided into training set, validation set and test set respectively; the two models are trained using the training set, the model parameters are optimized on the validation set through grid search under the optimization index of annotation consistency, and finally the performance is evaluated on the test set.

[0026] Further, in the data analysis module, the constructed named entity recognition model uses a social media text pre-training model to encode the input text, and a linear layer maps the text representation to entity labels; the loss function L used for training the named entity recognition model is ner The cross-entropy is:

[0027]

[0028] Wherein, is the true label of the token, represents the probability that the model predicts the label of the token as the true label, N is the number of training samples in the data set, and M is the number of tokens in each sample;

[0029] During model training, all parameters of the social media text pre-training model are fine-tuned using the AdamW learner according to the training data, and the hyperparameters of the model are adjusted using grid search on the validation set, and finally the optimal model is selected for performance evaluation on the test set; the evaluation indicators used are accuracy and F1 value; the entity information of all unlabeled social media texts in the data preprocessing module is extracted using the evaluated model, as the input of the target entity sentiment recognition model.

[0030] Further, in the data analysis module, the input of the target entity sentiment recognition model is spliced from the social media text and the target entity text, in the format of "[CLS] social media text [SEP] target entity text [SEP]", wherein [CLS] and [SEP] are two special marks used by the pre-training language model; the social media text pre-training model first extracts features from the input text sequence, and then the softmax classifier of the target entity sentiment recognition model predicts the sentiment polarity of the target entity in the social media text based on the features of the [CLS] special mark; the loss function L used for training the target entity sentiment recognition model is tsa The cross-entropy is:

[0031]

[0032] Wherein, is the true sentiment polarity of the target entity, The representative model predicts the probability of the target entity being a real sentiment polarity, N is the number of training samples in the data set, and C is the number of target entities of each sample;

[0033] During model training, the parameters of the model are fine-tuned using the AdamW learner, the model hyperparameters are adjusted using grid search on the validation set, and the optimal model is selected for performance evaluation on the test set; the evaluation indicators used include: accuracy and F1 value; the target entity sentiment recognition model after evaluation is used to predict the sentiment polarity of all target entities extracted by the named entity recognition model.

[0034] The beneficial effects of the present application are:

[0035] Current disease-related public opinion analysis systems are generally based on artificial construction of sentiment dictionaries or sentiment rules, and rely on general natural language processing tools, which have problems such as coarse analysis granularity, poor medical field adaptability, difficult updating and maintenance, and insufficient generalization. Based on a large amount of disease-related social media text data, combined with natural language processing and deep learning technology, an automatic disease-related public opinion analysis system with fine granularity is constructed. A large-scale disease-related social media text data is used to incrementally pre-train a general pre-training language model, and a disease-related social media pre-training language model is constructed to enhance the adaptability of the public opinion analysis system in the medical field; on this basis, a named entity recognition model based on the social media pre-training language model is constructed to automatically extract entity information in social text; a target entity sentiment recognition model based on the social media pre-training language model is constructed to automatically determine the sentiment polarity of disease-related entities. Through the foregoing entity extraction and sentiment analysis, more fine-grained real-time public opinion analysis results can be obtained, providing more accurate public opinion monitoring services, which not only helps government departments to track the latest developments of diseases and master the transmission rules of diseases, but also assists in disease prevention and control and public opinion management, and provides a reasonable basis for policy making. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a system structure diagram of the present application;

[0037] Figure 2 is a named entity recognition model structure diagram constructed by the present application;

[0038] Figure 3 is a target entity sentiment recognition model structure diagram constructed by the present application. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0040] In the following description, a lot of specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the present application, therefore, the present application is not limited to the specific embodiments disclosed below.

[0041] As shown in Figure 1 The present application provides a disease-related public opinion analysis system based on social media and artificial intelligence, comprising a data acquisition module, a data preprocessing module and a data analysis module.

[0042] The data acquisition module is used to collect social media text data and disease-related keyword list as the basis for system development, collects the dynamics and messages published by people within a certain period of time through connecting the data open platform of social media or using data crawler technology, and collects disease-related keywords by searching open medical database or literature resources.

[0043] The data preprocessing module uses the disease-related keyword list to filter and disease-related social media text, removes characters irrelevant to public opinion analysis from the filtered social media text, and outputs the preprocessed social media text.

[0044] The data analysis module constructs a social media text pre-training model, including using the preprocessed social media text to incrementally pre-train a general domain pre-training language model to obtain a social media text pre-training model.

[0045] Based on the social media text pre-training model, a named entity recognition model and a target entity sentiment recognition model are constructed, and the named entity recognition model and the target entity sentiment recognition model are trained, including randomly sampling the preprocessed social media text, distributing it to annotators for entity and sentiment annotation, then calculating the consistency of the annotation and feeding back to the annotators for evaluation, and after multiple iterations, the annotated samples meeting the requirements are used as the dataset of the named entity recognition model and the target entity sentiment recognition model; the dataset is used for model training.

[0046] The trained named entity recognition model is used to extract entities from the preprocessed social media text, and the extracted results are used as the input of the target entity sentiment recognition model to predict the sentiment polarity of all target entities.

[0047] Further, the data acquisition module collects social media data and disease-related keyword list by connecting the data open platform of social media or using data crawler technology.

[0048] Further, in the data collection module, the disease-related keywords include the name of the disease, the corresponding symptom name and the name of the treatment drug; when collecting text data using the social media data open platform, first, an application is provided to the open platform, and after authentication, structured data is obtained.

[0049] Further, in the data preprocessing module, the text exclusion criteria of the social media text are one of the following conditions: user forwarded text, text containing URL and text with length less than 5; the inclusion criteria of the sample are at least matching to one disease-related keyword.

[0050] Further, in the data preprocessing module, the characters irrelevant to the screened social media text and public opinion analysis include: removing HTML characters in the text and converting emoticons into strings.

[0051] Further, in the data analysis module, the construction of the social media text pre-training model specifically includes: in order to improve the domain adaptability of the named entity recognition and target entity sentiment recognition model and enhance its performance on disease-related social media text, two pre-training tasks of mask language model and next sentence prediction are adopted, all unlabeled social media texts are processed into the format of the two tasks as pre-training corpus, and the general domain pre-training language model (such as BERT) is incrementally pre-trained to obtain a social media text pre-training language model.

[0052] In the named entity recognition model annotation process in the embodiment of the application, the following steps are included:

[0053] In the pre-annotation stage, all annotators are required to perform 3 rounds of annotation. The F1 value is the harmonic mean of the precision and recall. All annotators are assigned the same data and need to annotate according to the annotation guide. After annotation, the project leader compares all labels and determines the final true label, which is used to calculate the consistency between annotators. Select the annotators with F1 greater than 80% to enter the formal annotation process. The annotation guide is also iteratively updated throughout the process.

[0054] In the formal annotation stage, the annotators are divided into three groups, two people in each group to annotate the pre-data to ensure that each data is annotated twice. When there is inconsistency, another annotator intervenes to determine the final annotation of the tweet.

[0055] After the formal annotation is completed, the project team performs quality control inspection on the marked results to ensure that the annotation results meet the requirements of the annotation guide. The final annotation consistency: the F1 value is not less than 85.0%.

[0056] In the target entity sentiment recognition data annotation process in the embodiment of the application, the following steps are included:

[0057] First, 6 rounds of emotion recognition pre-annotation were conducted, and the annotation guidelines were updated in each iteration. Accuracy was used as a measure of inter-annotator agreement. The way to determine the true label is the same as the above-mentioned named entity recognition. The annotators who meet the consistency standard are selected to participate in the subsequent annotation process.

[0058] The annotators were randomly paired (4 pairs in total), and the same data to be annotated was assigned to each group. When there is inconsistency, a third-party annotator determines the final annotation.

[0059] The project team conducted a second review to ensure that the annotation met the guidelines. The final inter-annotator agreement accuracy was 78.4%.

[0060] Further, in the data analysis module, the defined entity types include the following seven types: Person, Location, Organization, Disease, Symptom, Drug, and Vaccine. Among them, Person, Organization, Drug, and Vaccine are selected as target entity types, and each target entity type contains three sentiment polarities: Positive, Neutral, and Negative.

[0061] Further, the specific process of using the dataset to train the model is as follows:

[0062] The dataset of the named entity recognition model and the target entity sentiment recognition model is further divided into training set, validation set and test set according to the ratio of 70:15:15; the training set is used to train the two models, and the model parameters are optimized on the validation set through grid search under the optimization index of F1 value, and finally the performance is evaluated on the test set.

[0063] Further, in the data analysis module, the constructed named entity recognition model uses a social media text pre-trained language model to encode the input text, and a linear layer maps the text representation to entity labels; the loss function L ner is the cross-entropy:

[0064]

[0065] where, is the true label of the token, represents the probability that the model predicts the label of the token as the true label, N is the number of training samples in the dataset, and M is the number of tokens in each sample;

[0066] During the model training, all parameters of the social media text pre-training language model are fine-tuned using the AdamW learner according to the training data, and the hyperparameters of the model are adjusted using grid search on the validation set, and finally the optimal model is selected for performance evaluation on the test set; the evaluation indicators used are accuracy and F1 value; after the evaluation is completed, the entity information of all unlabeled social media texts in the data processing module is extracted using the model as the input of the target entity sentiment recognition model.

[0067] Further, in the data analysis module, the input of the target entity sentiment recognition model is spliced from the social media text and the target entity text, and the format is "[CLS] social media text [SEP] target entity text [SEP]", wherein [CLS] and [SEP] are two special marks used by the pre-training language model; the social media text pre-training language model first extracts features from the input text sequence, and then the softmax classifier of the target entity sentiment model predicts the sentiment polarity of the target entity in the social media text based on the features of the [CLS] special mark; the loss function L of the target entity sentiment model used in training is tsa is the cross-entropy:

[0068]

[0069] wherein, is the true sentiment polarity of the target entity, represents the probability that the model predicts the target entity as the true sentiment polarity, N is the number of training samples in the data set, and C is the number of target entities for each sample;

[0070] During the model training, the parameters of the model are fine-tuned using the AdamW learner, the hyperparameters of the model are adjusted using grid search on the validation set, and the optimal model is selected for performance evaluation on the test set; the evaluation indicators used include: accuracy and F1 value; after the evaluation is completed, the sentiment polarity of all target entities extracted by the named entity recognition model is predicted using the model.

[0071] The disease-related public opinion analysis system constructed by the application is tested and evaluated on the disclosed popular disease-related tweets, the F1 value of the named entity recognition model thereof is 83.88%, and the accuracy and F1 value of the target entity sentiment analysis model are 76.29% and 70.03%, respectively.

[0072] Figure 1 is a system structure diagram of the application.

[0073] Figure 2is the structure diagram of the named entity recognition model constructed by the present application. Taking the tweet text "They call it covid rash" as an example, first, special marks [CLS] and [SEP] are added before and after the text sequence to obtain a new sequence "[CLS]They call it covid rash[SEP]", then the text is mapped to the semantic space by the word embedding layer of the social media text pre-training language model, and then the text semantics is encoded by 12 encoding layers in turn, and finally the semantic features extracted by the last encoding layer are input to the linear layer and the entity label sequence "O O O B-Symptom E-Symptom" is output. Wherein "O" indicates that the corresponding word does not belong to the entity, "B-Symptom" and "E-Symptom" respectively represent the start and end positions of the symptom entity. By comparing the input sequence and the output sequence, the symptom entity "covid rash" can be identified from "They call it covid rash".

[0074] Figure 3 is the structure diagram of the target entity sentiment recognition model constructed by the present application. Taking the tweet text "Vitamins are good for you" and the target entity "Vitamins" as an example, first, the text and the target entity are connected using the special mark [SEP], and the special marks [CLS] and [SEP] are added at the beginning and end of the text sequence respectively, to obtain a new sequence "[CLS]Vitamins are good for you[SEP]Vitamins[SEP]", then each word is mapped to the semantic space by the word embedding layer of the social media text pre-training language model, and the semantic features containing context information are extracted by 12 encoding layers in turn, finally the feature vector corresponding to the "[CLS]" mark (corresponding to the output of the 12th encoding layer) is taken as the input of the softmax classifier, and the sentiment prediction of the target entity "Vitamins" is completed by the classifier.

[0075] In this example, the target entity sentiment recognition model captures the word "good" representing positive evaluation in the tweet text, so it can accurately predict the sentiment polarity of the target entity "Vitamins" as "Positive".

[0076] Through the diagram, it is proved that the correlation between the sentiment score and the key events and major policies can also reflect the evaluation tendency of users on related institutions and figures, so as to more accurately and efficiently capture the changes of public opinion and provide basis for the government's decision-making.

[0077] The above description is only the preferred embodiment of one or more embodiments of the specification, and is not used to limit one or more embodiments of the specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the specification should be included in the protection range of one or more embodiments of the specification.

Claims

1.A disease-related public opinion analysis system based on social media and artificial intelligence, characterized in that, The system comprises a data collection module, a data preprocessing module and a data analysis module. The data collection module is used for collecting social media texts and a disease-related keyword list, and collecting posts and messages published by people within a set time period. The data preprocessing module is used for screening the social media texts related to the disease by using the disease-related keyword list, removing characters irrelevant to public opinion analysis from the screened social media texts, and outputting the preprocessed social media texts. The data analysis module is used for constructing a social media text pretraining model, including incrementally pretraining a pretraining language model in a general field by using the preprocessed social media texts, to obtain the social media text pretraining model. Based on the social media text pretraining model, a named entity recognition model and a target entity sentiment recognition model are constructed; the constructed named entity recognition model encodes the input text by using the social media text pretraining model, and maps the text representation to an entity label by a linear layer; the named entity recognition model and the target entity sentiment recognition model are trained, including randomly sampling the preprocessed social media texts, distributing them to labelers for entity and sentiment labeling, then calculating the consistency of the labeling and feeding back to the labelers for evaluation, and after multiple iterations, the labeled samples meeting the requirements are used as the dataset of the named entity recognition model and the target entity sentiment recognition model, and the dataset is used for model training. The trained named entity recognition model is used to extract entities from the preprocessed social media texts, and the extracted results are used as the input of the target entity sentiment recognition model to predict the sentiment polarity of all target entities. 2.The social media and artificial intelligence based disease related public sentiment analysis system according to claim 1, wherein, The data collection module collects social media data and a disease-related keyword list by connecting a data opening platform of social media or using a data crawler technology; when collecting text data by using the data opening platform of social media, an application is provided to the opening platform first, and after authentication, structured data is obtained. 3.The social media and artificial intelligence based disease related public sentiment analysis system according to claim 1, wherein, In the data collection module, the disease-related keywords include the name of the disease, the corresponding symptom name and the name of the treatment drug. 4.The social media and artificial intelligence based disease related public sentiment analysis system according to claim 1, wherein, In the data preprocessing module, the text exclusion criteria of the social media texts are that the texts meet one of the following conditions: the texts are forwarded by a user, the texts contain a URL, and the length of the texts is less than 5; and the inclusion criteria of the samples are that at least one disease-related keyword is matched. 5.The social media and artificial intelligence based disease related public sentiment analysis system as claimed in claim 1 wherein, In the data preprocessing module, removing characters irrelevant to public opinion analysis from the screened social media texts includes removing HTML characters in the texts and converting emoticons into strings. 6.The social media and artificial intelligence based disease related public sentiment analysis system as claimed in claim 1 wherein, In the data analysis module, constructing the social media text pretraining model specifically includes: using two pretraining tasks of a mask language model and next sentence prediction, processing all unlabeled social media texts into the formats of the two pretraining tasks as pretraining corpus, and incrementally pretraining a pretraining language model in a general field to obtain the social media text pretraining model. 7.The social media and artificial intelligence based disease related public sentiment analysis system as claimed in claim 1 wherein, The defined entity types in the data analysis module include the following seven types: person name, place name, organization name, disease name, symptom name, drug name and vaccine; wherein, the four entity types of person name, organization name, drug name and vaccine are selected as target entity types, and each target entity type contains three sentiment polarities: positive, neutral and negative. 8.The social media and artificial intelligence based disease related public sentiment analysis system as claimed in claim 1 wherein, The specific process of training the model using the data set is as follows: The data set of the named entity recognition model and the target entity sentiment recognition model is further divided into training set, validation set and test set; the training set is used to train the two models, the model parameters are optimized on the validation set through grid search under the optimization index of annotation consistency, and finally the performance is evaluated on the test set. 9.The social media and artificial intelligence based disease related public sentiment analysis system according to claim 8, wherein, In the data analysis module, the loss function L used in the named entity recognition model training is ner is cross-entropy: wherein, the true label of the token, the probability that the token's label is the true label predicted by the model, N is the number of training samples in the dataset, and M is the number of tokens per sample. During the model training, the AdamW learner is used to fine-tune all parameters of the social media text pre-training model according to the training data, the hyperparameters of the model are adjusted on the validation set using grid search, and finally the optimal model is selected for performance evaluation on the test set; the evaluation indexes used are accuracy and F1 value annotation consistency; the evaluated model is used to extract entity information of all unlabeled social media texts in the data preprocessing module as the input of the target entity sentiment recognition model. 10.The social media and artificial intelligence based disease related public sentiment analysis system as claimed in claim 1 wherein, In the data analysis module, the input of the target entity sentiment recognition model is spliced from the social media text and the target entity text, and the format is "[CLS] social media text [SEP] target entity text [SEP]", wherein [CLS] and [SEP] are two special marks used by the pre-training language model; the social media text pre-training model first extracts features from the input text sequence, and then the softmax classifier of the target entity sentiment recognition model predicts the sentiment polarity of the target entity in the social media text based on the features of the [CLS] special mark; the loss function L used in the training of the target entity sentiment recognition model is cross-entropy: tsa cross-entropy wherein, a true sentiment polarity of a target entity, representing a probability that the target entity is of the true sentiment polarity, N is a number of training samples in the dataset, and C is a number of target entities per sample. During the model training, the AdamW learner is used to fine-tune the parameters of the model, the hyperparameters of the model are adjusted on the validation set using grid search, and finally the optimal model is selected for performance evaluation on the test set; the evaluation indexes used include: accuracy and F1 value; the evaluated target entity sentiment recognition model is used to predict the sentiment polarity of all target entities extracted by the named entity recognition model.

Images