Multi-modal sentiment analysis method for short text with emoticons

Abstract

The invention discloses a multi-modal sentiment analysis method for short texts with emoticons, which comprises the following steps of: acquiring a short text data set, and preprocessing data; constructing an emoticon sample set; performing feature vectorization; constructing an ET-BiLSTM-CNN hybrid analysis model, performing collaborative modeling of global and local semantics, and weighting the obtained emoticon vectors in combination with an attention mechanism; training the hybrid analysis model by using the training set; and performing short text sentiment classification by utilizing the trained hybrid analysis model. The invention provides an emoticon vectorization method, which makes up for the deficiency of emoticon vectorization in the existing research, provides richer semantic information for short text sentiment analysis, combines context to optimize semantic representation, introduces an attention mechanism to strengthen emoticon features, and fuses with text word vectors to improve the text sentiment analysis accuracy. And finally, sentiment analysis and classification are carried out, so that the analysis accuracy of the short text with the emotion symbols is greatly improved. The text content and the emoticon information can be synchronously obtained and fused at the same time.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of natural language processing, and particularly relates to a multi-modal sentiment analysis method for short text with emoticons. BACKGROUND

[0002] In the process of digitization, graphical expression is being integrated into daily communication at an unprecedented speed, and emoji has gradually become an indispensable emotional carrier in online communication. The widespread application of mobile internet technology and the vigorous development of social media have provided a fertile ground for such visual expression. This way of conveying emotions through non-verbal symbols is reshaping people's online interaction patterns. According to the data released by Unicode Consortium, an organization responsible for the digitization of world languages, in 2021, 92% of global network users use emoji in their daily communication. In the communication environment of social network platforms, text information often leads to semantic bias due to the lack of body language and voice features. As a new type of emotional medium, visual emoji effectively breaks through the limitations of digital communication through concrete expression. These graphical symbols not only accurately restore the emotional hints in the conversation context, but also trigger the receiver's imagination of the speaker's gestures and expressions. These symbols have a dual regulatory effect, which not only strengthens the emotional expression of text, but also directly subverts the original meaning of the text. Figure 11 It can be seen that emoji not only can strengthen the emotional expression of text, but also can directly subvert the original meaning of the text.

[0003] Sentiment analysis can be used as an important means to assist enterprises, governments and other institutions in public opinion analysis or business guidance by preprocessing and analyzing text to mine the sentiment tendency of the text. However, the applicant found in the research that the existing research on the sentiment of text with emoji mainly has the following two limitations: first, emoji is treated as a binary indicator feature, and its sentiment tendency is measured only by manual annotation or simple polarity classification, ignoring the polysemy of emoji in different contexts and the synergistic effect with text, which makes it difficult for the model to accurately capture complex emotional expression; second, emoji and text are considered separately, and most existing researches use rule-based or static word vectors. These methods are difficult to fully establish the interaction between emoji and text, resulting in information loss or limited feature expression ability when the model integrates emoji information, which affects the accuracy and robustness of sentiment analysis.

[0004] Some natural language processing technologies using artificial intelligence have also appeared in the prior art, such as patent No. "2022103546549", named "a text sentiment analysis method based on BERT improved model", the applicant found in the research that the method mainly uses BERT improved model to extract semantic features, uses BILSTM network to extract semantic features again, and introduces attention mechanism, which can improve the accuracy of sentiment analysis results to a certain extent. The applicant found in the research that the model mainly focuses on mining deep semantic information of context, but relatively ignores the weighting of emoticons, so it will directly affect the accuracy of the final analysis result. SUMMARY

[0005] In view of the above problems of the prior art, the technical problem to be solved by the present application is to provide a multi-modal sentiment analysis method for short text with emoticons to solve the problem of poor analysis accuracy of short text with emoticons in the prior art.

[0006] In order to solve the above technical problems, the present application adopts the following technical scheme:

[0007] A multi-modal sentiment analysis method for short text with emoticons, which comprises the following steps:

[0008] S1, collecting short text data set and performing data preprocessing;

[0009] S2, constructing emoticon sample set;

[0010] S3, performing feature vectorization;

[0011] S4, constructing ET-BiLSTM-CNN hybrid analysis model, and performing collaborative modeling of global and local semantics, wherein the CNN module is used to extract local semantic features, the ET-BiLSTM module encodes the global context dependence of the text through bidirectional LSTM, dynamically fuses the forward and reverse hidden states, and combines the attention mechanism to weight the obtained emoticon vector;

[0012] S5: training the hybrid analysis model using the training set;

[0013] S6: using the trained hybrid analysis model to perform short text sentiment classification.

[0014] By using the above scheme, the semantic representation is optimized in combination with the context, the attention mechanism is introduced to strengthen the emoticon features, the text word vector is fused, and finally the sentiment analysis classification is performed, so as to greatly improve the analysis accuracy of short text with emoticons.

[0015] As preferred: the data preprocessing in step S1 at least includes removing punctuation, converting case, removing stop words, word segmentation and special character cleaning. With the above scheme, reasonable preprocessing can convert the original short text data into a standardized form suitable for feature extraction and model training, which can improve the accuracy of later model training, reduce model complexity, etc.

[0016] As preferred: the feature vectorization in step S3 includes text vectorization and emoji vectorization, wherein the emoji vectorization includes the following steps:

[0017] S3.1, according to the name and its interpretation content in the constructed emoji sample set, mathematical modeling is carried out to extract relevant features;

[0018] S3.2, mapping each emoji name to a dense vector x i , feature extraction is carried out by Word2Vec word vector tool;

[0019] S3.3, by vectorizing the emoji interpretation content after word segmentation, a word vector sequence is obtained, and the sum of the word vector sequence is obtained to obtain an interpretation vector v j ; then the Sigmoid function is used to convert the vector inner product into a probability output, and the dot product result is mapped to a probability value of 0 to 1, representing the possibility of the emoji belonging to a specific emotion category;

[0020] S3.4, the vector representation is optimized by maximizing the likelihood function, so that the model can accurately distinguish the sentiment polarity of the emoji;

[0021] S3.5, define a loss function to quantify the difference between the probability and the true label;

[0022] S3.6, on the basis of considering the number of occurrences of the emoji, a fixed dimension vector is generated in one-hot encoding form, and the emoji vector representation is X i vector and quantity characteristics integration.

[0023] As preferred: the ET-BiLSTM-CNN hybrid analysis model includes a vector representation layer, a feature extraction layer, a semantic feature fusion layer, and a classification layer, wherein the feature extraction layer includes an ET-BiLSTM feature extraction layer for context feature extraction, and a CNN feature extraction layer for local feature extraction;

[0024] The ET-BiLSTM feature extraction step includes bidirectional context encoding, emoji attention mechanism and multi-modal feature fusion.

[0025] As preferred: the emoji attention mechanism is based on a traditional attention mechanism, assigns weights to each word in the text through emoji features, quantifies the influence of emojis on the sentiment polarity of the text, and splices the text representation h, the emoji vector e and the emoji weighted text representation s into a joint feature vector; comprising the following steps:

[0026] Firstly, define the score function to measure the correlation between the word h i and the emoji e

[0027] score(h i ,e)=v T tanh(W h h i +W E e+b)

[0028] Wherein, W h ∈R a×d , W E ∈R a×d is a projection matrix, and v∈R d is a weight matrix.

[0029] Secondly, normalize the scores of all words by Softmax, and the attention weight a i of each hidden state is defined as Wherein a i ∈[0,1];

[0030] Finally, the sentence representation s is defined as the weighted sum of the hidden states h i of each word by weighted aggregation of context information, and the calculation formula is By adopting the above scheme, the influence of emojis on the text sentiment can be quickly quantified, and the model accuracy is improved.

[0031] As preferred: the semantic feature fusion layer combines the local features h cnn extracted by CNN and the context features h ET-BiLSTM generated by ET-BiLSTM into a joint representation by vector splicing operation

[0032]

[0033] Compared with the prior art, the beneficial effects of the present application are:

[0034] The multi-modal sentiment analysis method of the short text with emoticons provided by the application constructs a new multi-modal sentiment analysis model, proposes an emoticon vectorization method, makes up for the deficiency of emoticon vectorization in the existing research, provides more rich semantic information for short text sentiment analysis, optimizes semantic representation combined with context, introduces attention mechanism to strengthen emoticon features, fuses with text word vectors, and finally performs sentiment analysis and classification, thereby greatly improving the analysis accuracy of short text with emoticons. The synchronous acquisition and fusion processing of text content and emoticon information can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The flowchart of the application is shown in the figure;

[0036] Figure 2 The ET-BiLSTM-CNN hybrid analysis model is shown in the figure;

[0037] Figure 3 The CNN local feature extraction model is shown in the figure;

[0038] Figure 4 The ET-BiLSTM global feature extraction model is shown in the figure;

[0039] Figure 5 The data acquisition example is shown in the figure;

[0040] Figure 6 The average loss change under different batch processing is shown in the figure;

[0041] Figure 7 The average loss change under different learning rates is shown in the figure;

[0042] Figure 8 The average loss change under different unit numbers is shown in the figure;

[0043] Figure 9 The ablation experiment result is shown in the figure;

[0044] Figure 10 The attention mechanism model is shown in the figure.

[0045] Figure 11 The influence of emoticons on text sentiment is shown in the figure. DETAILED DESCRIPTION

[0046] The application will be further described in detail below with reference to the accompanying drawings.

[0047] Reference Figures 1 to 11 The multi-modal sentiment analysis method of the short text with emoticons shown mainly includes the following steps:

[0048] S1, collect short text data sets and preprocess data: the data sets in this application are derived from public data, such as comments on some portals in the target field. It should be noted that relevant data with emoticons should be collected to ensure that effective samples are obtained.

[0049] Then the data is preprocessed, mainly to convert the original text data into a standardized form suitable for feature extraction and model training. This process includes key steps such as removing punctuation, converting case, removing stop words, tokenizing, and cleaning special characters.

[0050] S2, construct an emoticon sample set, as there are few vectorization of emoticons at present, how to convert emoticons to word vectors is a more difficult problem. Emoticon sentiment dictionary is an effective tool that can establish a corresponding relationship between emoticons and emotions. Due to the lack of sufficient existing samples, a sample set containing various emoticons must be constructed to meet the requirements of model training.

[0051] Common emojis and their related information can be obtained through Unicode Emoji standards or from emoji libraries such as Emojipedia. The pictures, names and interpretations of emojis can be collected.

[0052] The emojiall website contains all common emoji pictures, names and interpretations of the symbols. Its interpretation on the website is "expressing happiness, warmth, happiness or gratitude", so the sample of this emoji can be written as Smile, expressing happiness, warmth, happiness or gratitude). After processing the selected common emojis as described above, a complete sample set can be constructed, and all collected information can be arranged in table form for subsequent model training and analysis.

[0053] It should be noted that there are differences between the actual use semantics of some network emoticons and the official interpretation marked by the platform, and the related interpretation content needs to be corrected.

[0054] S3, feature vectorization;

[0055] S4, construct an ET-BiLSTM-CNN hybrid analysis model for global and local semantic collaborative modeling, where the CNN module is used to extract local semantic features, and the ET-BiLSTM module encodes the global context dependency of the text through bidirectional LSTM, dynamically fuses the forward and reverse hidden states, and combines the attention mechanism to weight the obtained emoticon vectors;

[0056] S5: train the hybrid analysis model using the training set;

[0057] S6: performing short text sentiment classification by using the trained mixed analysis model.

[0058] In specific implementation, the data preprocessing of the present application at least includes removing punctuation, converting case, removing stop words, word segmentation, and special character cleaning. The purpose of word segmentation is to cut continuous text into the smallest unit with semantic independence, i.e., a word. The purpose of stop word processing is to filter common words with little semantic value in the text, reducing noise such as high-frequency virtual words like "of", "was", "is", etc. Part-of-speech tagging aims to classify each word in the text and label its corresponding part of speech, such as noun, verb, adjective, etc., which is the premise and foundation for advanced tasks such as syntactic analysis, information extraction, sentiment analysis, etc. Through accurate part-of-speech tagging, the machine can understand the structure and grammatical relationship of language, improving the effect of subsequent processing.

[0059] Among them, word segmentation mainly adopts the Jieba word segmentation method based on rules and statistics, which is a technology that performs word segmentation by analyzing the co-occurrence information of words in a large amount of corpus. The core idea of this method is to use statistical models to calculate the relationship between words and then determine the segmentation points in the text. The statistical model learns a large amount of labeled corpus to obtain the boundary information of words and the probability distribution between words, so that it can perform word segmentation without an explicit dictionary. Unlike dictionary-based word segmentation methods, this method can effectively handle out-of-vocabulary words and new words because it determines the boundaries of words through context information. This makes it advantageous in handling colloquial language, network language, and emerging words. In this embodiment, to improve computational efficiency, it is combined with rule-based word segmentation or hybrid word segmentation methods to further optimize the word segmentation effect.

[0060] In specific implementation, the feature vectorization in step S3 includes text vectorization and emoji vectorization, wherein the emoji vectorization includes the following steps:

[0061] S3.1, first, according to the name and its interpretation content in the constructed emoji sample set, mathematical modeling is performed to extract relevant features.

[0062] S3.2, secondly, map each emoji name to a dense vector x i , and perform feature extraction through the Word2Vec word vector tool.

[0063] S3.3, by vectorizing the segmented emoji interpretation content, a word vector sequence is obtained, and the sum of the word vector sequence is obtained as the interpretation vector v j ; then, the Sigmoid function is used to convert the vector inner product into a probability output, mapping the dot product result to a probability value between 0 and 1, representing the possibility of the emoji belonging to a specific sentiment category, which can be represented by formula (1).

[0064]

[0065] S3.4, Finally, the vector representation is optimized by maximizing the likelihood function, which enables the model to accurately distinguish the sentiment polarity of the emoji, and this process also integrates the name of the emoji x i disambiguation feature v j , to achieve end-to-end sentiment-aware vectorization.

[0066] S3.5, Next, the loss function is defined to quantify the difference between the probability and the true label, which can be represented by equation (2): First, define the data set D = {(v j , y v ) | v j ∈ R n , y v ∈ {0, 1}}, where y v is the label, which takes the value 1 or 0 according to whether the descriptive statement is a positive or negative sample. The final optimized 50-dimensional emoji vector X i is calculated using the gradient descent method.

[0067]

[0068] S3.6, Of course, the number of occurrences of the emoji also needs to be considered. In the review text, there are multiple uses of the same emoji, which can enhance the expression of emotion. The number of occurrences of the emoji in the review text is counted and encoded as a quantity feature vector C i , which is mapped to a discretized segmented vector to quantify the sentiment intensity. The specific rules are as follows: According to the actual data distribution, set the threshold interval, such as 1-2 times for low frequency, 3-5 times for medium frequency, and more than 5 times for high frequency, and use one-hot encoding to generate a fixed-dimensional vector. The emoji vector representation is X i . The integration of the vector and the quantity feature can be represented by equation (3):

[0069] E t = {X i , C i} (3)

[0070] As for text vectorization, it is mainly through the feature vector obtained by training the Word2Vec model, and the text is generated as a corresponding word vector sequence X = [X1, X2,..., X T ] according to the word order.

[0071] As Figure 2As shown, the ET-BiLSTM-CNN hybrid analysis model constructed in this application mainly includes a vector representation layer, a feature extraction layer, a semantic feature fusion layer, and a classification layer. The feature extraction layer includes an ET-BiLSTM feature extraction layer for contextual feature extraction. The ET-BiLSTM module encodes the global contextual dependency of the text through bidirectional LSTM, dynamically fuses the forward and backward hidden states, and combines an attention mechanism to weight the obtained emoji vectors. It also includes a CNN feature extraction layer for local feature extraction.

[0072] The vector representation layer generates the corresponding text representation using vectorization methods. In the diagram, X represents the text feature vector, and E represents the emoji vector. The semantic feature fusion layer uses vector concatenation to jointly represent the local features extracted by the CNN and the contextual features generated by the ET-BiLSTM. The Softmax classification layer combines attention-weighted text and emojis to output a sentiment polarity probability distribution, with the optimization objective being to minimize the cross-entropy loss.

[0073] refer to Figure 2 and Figure 3 In this embodiment, the core computational process of CNN extracting local semantic features of text is as follows: First, local word vector fragments X are extracted from the input word vector sequence X using a sliding window. (i,i+r-1) Secondly, a convolution operation is performed on the segment, and the combination pattern of local features is captured through a linear transformation of the convolution kernel weight matrix W and the bias b. Finally, a high-order local feature representation h is generated through a non-linear activation function f. cnn This process is computed in parallel using multi-channel convolution kernels, and is represented by formula (4):

[0074] h cnn =f(W*V (i,i+r-1) +b) (4)

[0075] refer to Figure 2 and Figure 4 This application uses an ET-BiLSTM network to jointly model text sentiment features through bidirectional context encoding and an emoji attention mechanism. The model diagram is shown below. Figure 4 As shown, its core process is as follows:

[0076] 1) Bidirectional context encoding. First, equations (5) and (6) represent the hidden state calculations of the forward and backward LSTMs, respectively, through recursive processing of word x. t and pass the hidden state h t-1 This captures contextual dependencies from left to right and from right to left, respectively. Then, using formula (7), the learnable weight parameters w are used... t and v tThe hidden states from the forward and backward views are linearly weighted and fused, and a bias term b is added. t This generates the final bidirectional context-aware text representation.

[0077]

[0078]

[0079]

[0080] 2) Emoji Attention Mechanism. Based on the attention mechanism, weights are assigned to each word in the text using emoji features, thereby quantifying the impact of emojis on the emotional polarity of the text. Finally, the text representation h, the emoji vector e, and the emoji-weighted text representation s are concatenated into a joint feature vector to model the bidirectional relationship between emojis and context.

[0081] First, define the score function and the term h. i The correlation with the emoji e is expressed by formula (8):

[0082] score(h i ,e)=v T tanh(W h h i +W E e+b) (8)

[0083] Among them, W h ∈R a×d W E ∈R a×d Let v ∈ R be the projection matrix. d Let a be the weighted moment. Then, normalize the scores of all words using Softmax, and set the attention weight 'a' for each hidden state. i Defined using formula (9), where a i ∈[0,1]:

[0084]

[0085] Finally, by weighted aggregation of contextual information, the sentence representation s is defined as the hidden state h of each word. i The weighted sum is calculated using formula (10).

[0086]

[0087] 3) Multimodal feature fusion. The basic text representation, emoji representation, and emoji-weighted text representation are concatenated to form a joint feature vector, expressed by formula (11):

[0088]

[0089] In this application's hybrid model, the semantic feature fusion layer uses vector concatenation operations to combine the local features h extracted by the CNN. cnn Contextual features h generated by ET-BiLSTM ET-BiLSTM Specifically, the n-gram level features captured by the CNN and the sequence dependency features modeled by the ET-BiLSTM are concatenated to form a complementary pair, as expressed by formula (12):

[0090]

[0091] refer to Figures 1 to 11 The multimodal sentiment analysis method for short texts with emojis, as shown, first involves preparing sufficient high-quality corpora as model input. Then, feature engineering methods are used to vectorize the original text and emojis separately. Finally, a classifier is selected for sentiment polarity classification, and pattern recognition and inductive reasoning are performed on the transformed features to generate the final predicted labels. This achieves the transformation from unstructured text to structured category calculation.

[0092] This embodiment focuses on short text analysis of the comment section of a scenic spot entrance. Over 10,000 valid samples were collected. After preprocessing, the collected data was manually labeled to indicate the sentiment polarity of the texts, categorizing them into positive and negative polarities. The distribution of sentiment polarity in the experimental data is shown in Table 1. The statistics show that 5150 samples were labeled with 0 (negative sentiment), while 5311 samples were labeled with 1 (positive sentiment), for a total of 10462 samples. The relatively balanced number of positive and negative sentiment samples helps avoid class imbalance, providing a reliable data foundation for model training and ensuring the performance stability of the sentiment classifier across different polarities.

[0093] Table 1: Statistics on Emotional Polarity

[0094] Sentiment polarity Number 0 5150 1 5311 Total number 10462

[0095] Table 2 presents the distribution characteristics of emojis in the text dataset. Experimental data shows that there are a total of 4,759 text samples containing emojis, while the number of samples without emojis is 5,703. This distribution indicates that emojis appear in 45% of the texts, meaning that nearly half of the texts use emojis as a means of expressing emotion. This provides sufficient data support for studying the impact of emojis on sentiment analysis.

[0096] Table 2: Statistics on whether emojis are present

[0097] Whether the text contains emojis Number Yes 4759 No 5703

[0098] Examples of constructing symbol sample sets are shown in Table 3:

[0099] Table 3. Examples of Emoji Sample Sets

[0100]

[0101] In this embodiment, a stratified sampling method was used to divide the data into a training set and a validation set, with a ratio of 8:2. To ensure the reliability of the experiment, the original characteristics of the data distribution, including the frequency ratio of sentiment tags and emojis, were preserved. A systematic comparative experiment was designed for the key hyperparameters of the model. Using the controlled variable method, the impact of different parameter combinations on model performance was examined sequentially, and the optimal parameter configuration was finally determined as the basic setting of the benchmark model. The parameter selection process strictly followed machine learning best practices, and the main considerations included validation set accuracy, F1 score, and training stability. Each parameter underwent multiple rounds of cross-validation to ensure the statistical significance of the results. The optimization process of several core parameters will be described in detail below, including but not limited to key hyperparameters such as learning rate, batch size, and network layer dimension. The determination of these parameters took into account both theoretical basis and empirical analysis of experimental data.

[0102] Emojis are represented using pre-trained word vectors with a dimension of 50, consistent with the dimension of text word vectors. This dimension choice effectively preserves the semantic information of emojis while avoiding overfitting caused by excessively high dimensionality. Experiments show that 50-dimensional representation achieves the optimal balance between computational efficiency and feature representation capability.

[0103] (1) Batch size: This represents the number of samples selected in one training session. Its size affects the optimization degree and speed of the model. A larger batch size can lead to memory overflow, while a smaller value will significantly prolong the training time. The changes in average loss under different batch sizes are shown in Figure 6. The figure shows that when the batch size is set to 32, the corresponding loss curve has a moderate descent rate in the early stage and small fluctuations in the later stage, indicating that the gradient update is relatively stable.

[0104] (2) Learning rate: This is the magnitude by which the model adjusts its parameters based on prediction errors. A learning rate that is too high will skip optimal results, while a learning rate that is too low will lead to inefficient training. The variation of average loss under different learning rates is shown below. Figure 7As shown in the chart, the data analysis reveals significant differences in the impact of different learning rates on model training. According to the chart, a learning rate of 0.001 initially results in a rapid decrease in loss but later exhibits significant fluctuations; 0.0005 leads to stable convergence but ultimately a higher loss; 0.0001 shows a continuous decrease in loss that stabilizes at its lowest value later, balancing speed and accuracy; 0.00005 results in a slow decrease and low efficiency. Comprehensive analysis indicates that setting the learning rate to 0.0001 yields the best model convergence and the highest final accuracy.

[0105] (3) Number of hidden layer units in BiLSTM (cell_units): cell_units is a key parameter that determines the model's capacity and generalization ability, such as... Figure 8 The figure shows the loss variation under different numbers of units. Analysis of the figure reveals that when cell_units is set to 64, the model's capacity is insufficient, leading to underfitting and an inability to fully learn data features. When cell_units is set to 128, the loss curve continuously decreases and converges stably, indicating that the model complexity is moderate, efficiently capturing data patterns while avoiding overfitting, resulting in optimal overall performance. When cell_units are set to 256 and 300, the initial loss decreases rapidly, but later rebounds to 0.55 and 0.58 respectively, indicating overfitting due to excessive unit numbers and a significant decrease in generalization ability.

[0106] Through systematic hyperparameter optimization experiments, the optimal parameter configuration of the model was determined, and the specific settings are detailed in Table 4. During parameter tuning, the impact of different parameter combinations on model performance was examined. Experimental results show that the parameter settings listed in Table 4 can achieve optimal classification performance while ensuring the model's convergence speed. The selection of these parameters considers both theoretical basis and empirical results on the validation set, ensuring that the model achieves the best balance between accuracy and generalization ability.

[0107] Table 4 Model Parameters

[0108] Parameter name Parameter description Parameter setting sg Training model Skip-gram size Word vector dimension 50 maxlen Maximum length of text input 100 window Sliding window size 5 iter Number of iterations of stochastic gradient descent 5 Mini_count Minimum word frequency 10 batch_size Batch size 32 learning_rate Learning rate 0.0001 lstm_units Number of BiLSTM units 128 dropout Dropout rate 0.5

[0109] This application employs a controlled variable approach to design a parameter comparison scheme for the system, using the same data preprocessing workflow and training and validation sets. The experiment selected seven representative benchmark models for system comparison, covering various model architectures from traditional machine learning to deep learning, as detailed below:

[0110] 1) SVM, a machine learning algorithm commonly used for text classification, uses a simple weighted sum of text vectors as input to the model;

[0111] 2) FastText is an efficient open-source tool for text classification and word vectorization. It uses a shallow neural network structure to quickly represent and classify text.

[0112] 3) Text Convolutional Neural Network (TextCNN): A text classification model based on convolutional neural networks. It extracts local features through multiple convolutional kernels and performs classification in fully connected layers.

[0113] 4) BiLSTM, which uses text word vectors as model input;

[0114] 5) emoji2vec: This uses pre-trained emoji vectors concatenated with text vectors as input to the model.

[0115] 6) BiLSTM+emoji vectors: Based on BiLSTM, emoji vectors and text vectors are directly concatenated as input to the model.

[0116] 7) ET-BiLSTM uses an attention mechanism to weight emojis and integrates text representations and emoji embedding vectors into BiLSTM by concatenation.

[0117] All experiments were repeated three times and the average value was taken to eliminate the influence of randomness and ensure the reliability of the experimental results. The specific experimental setup and results are shown in Table 5.

[0118] Table 5 Comparison of Experimental Results (Unit: %)

[0119] Model F1 Precision Recall SVM 70.12 70.42 69.76 FastText 73.52 74.13 72.89 TextCNN 74.91 75.28 74.69 BiLSTM 77.05 77.45 76.61 emoji2vec 76.01 75.79 76.25 BiLSTM + emoji vectors 76.96 76.99 76.95 ET-BiLSTM 79.38 79.84 78.96 ET-BiLSTM-CNN 82.98 83.40 82.61

[0120] Furthermore, to verify the contribution of each component in the constructed model to the final performance, this study designed systematic ablation experiments. All ablation experiments were conducted under the same data partitioning and parameter settings to ensure the comparability of the results. The experimental results are as follows. Figure 9 As shown, by progressively removing key modules, including the attention mechanism, CNN feature extraction, and text representation, the impact of each component on model performance was quantitatively analyzed. Experimental results show that the complete ET-BiLSTM-CNN model achieves optimal performance through the collaboration of its modules, with an F1 score of 82.98%, precision of 83.40%, and recall of 82.61%. Removing the emoji attention mechanism resulted in a 5.74% decrease in F1 score, a 6.15% decrease in precision, and a 6.04% decrease in recall. Removing the CNN module reduced precision to 73.70%. Removing the text representation module decreased the F1 score by 15.07 percentage points.

[0121] Experiments demonstrate that each component plays an irreplaceable role, with each module contributing uniquely to semantic representation, local feature enhancement, and multimodal interaction; none can be omitted. The absence of the emoji attention mechanism significantly reduces the model's classification accuracy for text containing emojis. Text representation is fundamental to the model; its removal leads to performance degradation, indicating the model's high dependence on textual semantic information. This experimental result strongly validates the necessity of the proposed multimodal feature fusion method.

[0122] The specific method primarily proposes an emoji vectorization approach, addressing the shortcomings of existing research in emoji vectorization and providing richer semantic information for short text sentiment analysis. A sentiment analysis model based on ET-BiLSTM-CNN is designed, integrating emojis to improve the accuracy of sentiment classification and validating its effectiveness in multiple scenarios. Contextual information is combined to optimize the semantic representation of emojis, and an attention mechanism is introduced to strengthen emoji features, which are then fused with text word vectors. Finally, sentiment classification is performed, and comparative and ablation experiments are designed to verify the contribution of each module to the model's performance. Experimental results show that the model exhibits a significant advantage in processing text containing emojis, with an accuracy improvement of 5.95% compared to methods using only text word vectors as input to the BiLSTM model.

[0123] The trained model was used to classify the sentiment polarity of comment text from another well area. For example, the comment "Nice scenery, lots of bamboo!" "The staff were very friendly and helpful, and the scenic area is very large." The sentiment was categorized as positive. "Too many people, poor service, too few cars. Chaotic order." "It's so congested, there are too few cars!" "The sentiment is categorized as negative. Using this model allows for more accurate sentiment classification of scenic area reviews, helping management teams better understand visitors' feelings and grasp changes in their emotions, thus more effectively meeting their needs. Furthermore, by providing more accurate sentiment classifications, management teams can make targeted improvements to the scenic area's services and facilities. If consistent negative sentiment emerges in the reviews, management teams can quickly take action to address the issue, thereby increasing visitor satisfaction. Additionally, by analyzing the sentiment classification results of a large number of scenic area reviews, management teams can gain deeper insights into visitor trends and needs, helping them predict future demands and take adaptive measures. These initiatives contribute to enhancing the competitiveness and sustainable development of the scenic area."

[0124] The above are merely preferred embodiments of the present invention. It should be noted that any modifications and improvements made by those skilled in the art without departing from the present technical solution should also be considered to fall within the scope of protection claimed in this claim.

Claims

1. A multimodal sentiment analysis method for short texts with emojis, characterized in that, Includes the following steps: S1: Collect short text datasets and perform data preprocessing; S2, Construct an emoji sample set; S3, perform feature vectorization; S4. Construct an ET-BiLSTM-CNN hybrid analysis model to perform collaborative modeling of global and local semantics. The CNN module is used to extract local semantic features, while the ET-BiLSTM module encodes the global contextual dependency of the text through bidirectional LSTM, dynamically fuses the forward and backward hidden states, and combines an attention mechanism to weight the obtained emoji vectors. S5: Train the hybrid analysis model using the training set; S6: Use a trained hybrid analysis model for short text sentiment classification.

2. The multimodal sentiment analysis method for short texts with emojis according to claim 1, characterized in that: The data preprocessing in step S1 includes at least punctuation removal, case conversion, stop word removal, word segmentation, and special character cleanup.

3. The multimodal sentiment analysis method for short texts with emojis according to claim 1 or 2, characterized in that, Step S3, feature vectorization, includes text vectorization and emoji vectorization, wherein emoji vectorization includes the following steps: S3.1, Based on the names and their meanings in the constructed emoji sample set, perform mathematical modeling to extract relevant features; S3.2, map each emoji name to a dense vector xi, and extract features using the Word2Vec word vector tool; S3.3: After segmenting the emoji's meaning into words and vectorizing them, a sequence of word vectors is obtained. These vectors are then summed to obtain the meaning vector vj. Next, the Sigmoid function is used to convert the vector dot product into a probability output, and the dot product result is mapped to a probability value between 0 and 1, representing the probability that the emoji belongs to a specific emotion category. S3.4 optimizes the vector representation by maximizing the likelihood function, enabling the model to accurately distinguish the emotional polarity of emojis; S3.5, Define the loss function, quantifying the difference between the probability and the true label; S3.6, based on the frequency of emoji occurrence, uses one-hot encoding to generate fixed-dimensional vectors. The emoji vector is represented as an integration of the Xi vector and the quantitative features.

4. The multimodal sentiment analysis method for short texts with emojis according to claim 1, characterized in that: The ET-BiLSTM-CNN hybrid analysis model includes a vector representation layer, a feature extraction layer, a semantic feature fusion layer, and a classification layer. The feature extraction layer includes an ET-BiLSTM feature extraction layer for contextual feature extraction and a CNN feature extraction layer for local feature extraction. The ET-BiLSTM feature extraction steps include bidirectional context encoding, emoji attention mechanism, and multimodal feature fusion.

5. The multimodal sentiment analysis method for short texts with emojis according to claim 4, characterized in that: The emoji attention mechanism is based on traditional attention mechanisms. It assigns weights to each word in the text based on emoji features, quantifies the influence of emojis on the emotional polarity of the text, and concatenates the text representation h, the emoji vector e, and the emoji-weighted text representation s into a joint feature vector; including The following steps are required: First, define the score function to measure the word h. i The correlation with the emoji 'e' is score(h i ,e)=v T tanh(W h h i +W E e+b) Among them, W h ∈R a×d W E ∈R a×d Let v ∈ R be the projection matrix. d These are the weighted moments; Secondly, the scores of all words are normalized using Softmax, and the attention weight 'a' for each hidden state is... i Defined as Where a i ∈[0,1]; Finally, by weighted aggregation of contextual information, the sentence representation s is defined as the hidden state h of each word. i The weighted sum is calculated using the following formula:

6. The multimodal sentiment analysis method for short texts with emojis according to claim 4 or 5, characterized in that: The semantic feature fusion layer uses vector concatenation operations to combine the local features h extracted by the CNN. cnn Contextual features h generated by ET-BiLSTM ET-BiLSTM Perform joint representation as

Images