Public opinion emotion detection method, device and equipment based on large model fine tuning

Abstract

The invention relates to the field of network public opinions, and discloses a public opinion emotion detection method, device and equipment based on large model fine tuning. A public opinion text is subjected to sentence segmentation, denoising and word segmentation, a text sequence and a timestamp sequence are generated and input into a public opinion emotion detection model, and the model is fused with time information, semantic features and an emotion dictionary for joint modeling. According to the method, a time sequence emotion joint embedding module is constructed on a large model embedding layer, the information timeliness is measured by a time decay function, and emotion features are extracted in combination with an emotion dictionary; a multi-dimensional attention fusion module is introduced into a Transform trunk, and multi-dimensional attention modeling of domain knowledge, emotional intensity and time sequence attenuation is achieved; and a multi-stage fine tuning strategy is adopted to realize time sequence adaptation and domain self-learning of the model. According to the method, the accuracy and robustness of public opinion emotion recognition are remarkably improved, and the method has better time sensitivity and field generalization ability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network public opinion, in particular to a public opinion sentiment detection method, device and equipment based on large model fine-tuning. BACKGROUND

[0002] With the rapid development of the Internet, microblogs, WeChat public accounts, short video platforms, news comment sections and other media have become important channels for the public to express their views and emotions. Public opinion data is usually represented as unstructured natural language text, with characteristics such as colloquialization, implicitness, emoticonization, fragmentation and strong timeliness. Public opinion sentiment analysis aims to identify the emotional tendencies (such as positive, negative or neutral) hidden in the text, and has important application value in government public opinion monitoring, enterprise brand management, emergency warning and other scenarios.

[0003] Current common sentiment analysis methods mainly include rule-based sentiment analysis, which relies on sentiment dictionaries and matching rules, and is difficult to understand complex semantics and has poor generalization ability. Deep learning-based analysis methods, such as BERT, have semantic modeling capabilities, but their knowledge reserves are limited, and they are not good at recognizing domain-specific vocabulary and obscure knowledge, and they do not consider the time sequence variation characteristics of public opinion data.

[0004] In recent years, large language models (LLM) have strong semantic understanding and generation capabilities, but direct application to public opinion sentiment analysis still has the technical problems of lack of understanding of domain knowledge and professional terminology, high fine-tuning cost and easy to forget original knowledge, and ignoring the time sequence and transmission dynamics of public opinion data. SUMMARY

[0005] Therefore, the present application provides a public opinion sentiment detection method based on large model fine-tuning, which solves the technical problems of lack of understanding of domain knowledge and professional terminology, high fine-tuning cost and easy to forget original knowledge, and ignoring the time sequence and transmission dynamics of public opinion data in the prior art.

[0006] According to a first aspect of the present application, a public opinion sentiment detection method based on large model fine-tuning is provided, comprising:

[0007] Obtaining the public opinion text to be detected, and performing sentence division, denoising and word segmentation processing on the public opinion text to be detected to generate a text sequence and a corresponding timestamp sequence wherein each word x i is the i-th word, and the corresponding time identifier of the text x i is t i ;

[0008] The text sequence and its corresponding timestamp sequence are input into the public opinion sentiment detection model to obtain the sentiment tags of the public opinion text. The public opinion sentiment detection model is used to generate dynamic word embeddings that fuse time decay features and sentiment semantics based on the text sequence X, timestamp sequence T, and sentiment dictionary C. The dynamic word embeddings are input into the multi-dimensional attention fusion module in the Transformer architecture to obtain fusion features. The multi-dimensional attention fusion module includes domain knowledge attention oriented towards the domain dictionary, sentiment intensity attention oriented towards polarity intensity, and time decay attention oriented towards time factors. The sentiment dictionary contains a domain vocabulary and sentiment polarity and sentiment intensity labels corresponding to each word.

[0009] Output the sentiment tags of the text to be detected.

[0010] In one possible implementation,

[0011] Based on the text sequence, timestamp sequence, and sentiment dictionary, dynamic word embeddings that integrate time decay features and sentiment semantics are generated, including:

[0012] The word embedding layer based on the public opinion sentiment detection model obtains the original word embedding E(x). i )∈R d ;

[0013] Calculate the monotonically decreasing time decay coefficient α for each word's timestamp. i , where α i Let α be the event decay coefficient for the i-th word. i α is used to measure the temporal impact weight of a token on overall sentiment assessment. i =exp(-μ·Δt) i ), μ is the time decay coefficient, Δt i For the standardized time interval, t i The timestamp of the text or sentence containing the token, t max With t min These represent the latest and earliest timestamps in the text of public opinion to be detected, respectively, and ε is a small constant to prevent the denominator from being zero;

[0014] Based on the sentiment dictionary C={C k}Calculation word x i semantic similarity S(x) i C k ),

[0015] Among them, C k Let k be the k-th sentiment center, mean(E(C) k )) represents the mean of the vector of entries in the sentiment dictionary, and ||·||2 represents the L2 norm of the vector;

[0016] Embed the original word into E(x) i Time decay coefficient α i , The words are merged to form a dynamic word embedding E′(x). i ),in, For word x i With the various categories of emotion centers C in the emotion dictionary k Semantic similarity by weight w k Weighted summation, where m is the number of sentiment categories.

[0017] In another possible implementation, dynamic words are embedded into the multi-dimensional attention fusion module of the input Transformer architecture to obtain fused features, including:

[0018] Embed dynamic words into E′(x) i The mean vector of all word embeddings in the industry sentiment dictionary D. Through isomorphic linear mapping matrix W d The first weight is obtained by scoring the dot product after projection and calculating the softmax. The first weight reflects the degree of semantic matching between the input word and the domain sentiment dictionary, where, As a feature dimension normalization term, the industry sentiment dictionary D is a domain-specific sentiment knowledge base built for a specific public opinion application field. The content of the industry sentiment dictionary D includes common emotional expression words, industry terms and their corresponding sentiment polarity and intensity annotations in the field.

[0019] The second weight is generated based on the sentiment polarity / intensity dictionary P. Where, E′(x i W represents the dynamic word embedding vector of the i-th word. p b is a learnable weight matrix for emotion intensity and attention. p Here, δ(·) is the bias term, and E(p(x) is the nonlinear activation function. i ) indicates word x i The emotional intensity embedding, p(x) i ) represents the corresponding entry of the word in the sentiment polarity / intensity dictionary P, which is a set containing positive, negative, and neutral sentiment words and their intensity labels;

[0020] A third weight is generated based on the time decay factor and the first and second weights. Where μ is the time decay coefficient, rank(t) i () represents the relative sorting position of the timestamp corresponding to the i-th word in the entire text sequence. As the first weight, As the second weight;

[0021] By using a third weight, the global context features obtained from the Transformer encoder in the public opinion sentiment detection model network architecture are weighted and fused together. i Global context features of the Long Short-Term Memory (LSTM) network in the network architecture of the public opinion sentiment detection model (E(x)). i )) to obtain fusion features As the third weight, E(x) i () is the original word embedding.

[0022] In another possible implementation, a multi-stage fine-tuning strategy is employed to train the public opinion sentiment detection model, including temporal sentiment feature pre-training, domain knowledge fine-tuning, and end-to-end joint optimization. The temporal sentiment feature pre-training minimizes the temporal sentiment alignment loss L. time-sent This makes the weights or attention of the sentiment detection model sensitive to time decay and sentiment intensity; domain knowledge fine-tuning is achieved by introducing domain knowledge regularization L... domain To improve industry adaptability, we inject domain prior distributions; we perform end-to-end joint optimization to achieve joint loss convergence; and we apply a temperature annealing strategy at the classification head / attention score to optimize the classification boundary.

[0023] In another possible implementation, temporal sentiment feature pre-training includes:

[0024] Provide a time-sentiment weighting benchmark for each word Used to supervise the alignment of the model with temporal decay and sentiment semantics;

[0025] Minimize temporal sentiment alignment loss in, The weight distribution is derived from dynamic embedding / attention, where N is the total number of samples and n i Let P be the sequence length of the i-th sample. θ (x i,t |x i,1:t-1 This indicates that, with model parameters θ, the prediction of word x at the current position t is calculated. i,t The conditional probability, x i,1:t-1 This represents the sequence of the i-th sample from the first word to the (t-1)-th word, which is used to predict the current word x. i,t The preceding context upon which the time model depends, -logP θ Used to measure language modeling error The time decay weight of the i-th sample at time t calculated by the model. The target time decay reference value is generated by manual annotation or rules, and λ1 is a balance coefficient used to control the relative weight of language modeling error and time alignment error. This represents the Euclidean distance between the model's time decay prediction and the target.

[0026] The preferred approach is to first freeze the backbone network of the public opinion sentiment detection model and then train only the dynamically embedded relevant parameters to stably introduce temporal sentiment priors.

[0027] In another possible implementation, domain knowledge fine-tuning includes:

[0028] Differences between word domain matching distribution and domain prior using the Kullback–Leibler divergence constraint model Among them, W d The isomorphic linear mapping matrix is ​​used to map word embeddings and neighborhood centers to the same space for comparability, and KL(·) is the Kullback–Leibler divergence. The mean vector of all word embeddings in the industry sentiment dictionary D is used as the overall representation of the domain sentiment.

[0029] Minimize knowledge fine-tuning L finetunine =L ce +αL domain , where L ce Cross-entropy loss;

[0030] The parameters of the dynamically embedded layer that have been trained in the pre-training of temporal features are frozen, and some Transformer layers and fusion modules are unfrozen so that the regularization terms continue to influence attention allocation and intermediate representation.

[0031] In another possible implementation, end-to-end joint optimization includes:

[0032] Minimize the joint loss of knowledge fine-tuning L = L ce +αL domain +βL time-sent , where L ce For cross-entropy loss, L domain To illustrate the difference between the word domain matching distribution and the domain prior in the Kullback–Leibler divergence-constrained model, L time-sent Loss of time-emotional alignment;

[0033] Temperature scaling is applied at the classification layer or attention score, with the temperature decreasing with each training epoch, where T... t =T0·e (-γ·epoch) T t γ is the current temperature value at the t-th training round, T0 is the initial temperature value, γ is the annealing rate constant, and epoch is the current training round.

[0034] Output the class probability of the classification With smooth boundaries and stable confidence, p(y|F) represents the probability that an input sample is classified as sentiment category y under feature F, where F... y F represents the model output score for the corresponding category y. k The output score is for the k-th category, where K is the total number of emotion categories and T is the temperature adjustment parameter.

[0035] Simultaneously, the dynamic embedding, multi-dimensional attention fusion module, and Transformer backbone and classification head parameters are updated to achieve unified convergence of semantic-temporal-domain three-dimensional features.

[0036] According to a second aspect of the present invention, a public opinion sentiment detection device based on large model fine-tuning is provided, comprising:

[0037] The acquisition module is used to acquire the public opinion text to be detected, and to perform sentence segmentation, noise reduction, and word segmentation on the text to generate a text sequence. and the corresponding timestamp sequence Where each word x i For the i-th word, text x i The corresponding time identifier is t i ;

[0038] The detection module is used to input the text sequence and its corresponding timestamp sequence into the public opinion sentiment detection model to obtain the sentiment tags of the public opinion text. The public opinion sentiment detection model is used to generate dynamic word embeddings that fuse time decay features and sentiment semantics based on the text sequence X, timestamp sequence T, and sentiment dictionary C. The dynamic word embeddings are input into the multi-dimensional attention fusion module in the Transformer architecture to obtain fusion features. The multi-dimensional attention fusion module includes domain knowledge attention oriented towards the domain dictionary, sentiment intensity attention oriented towards polarity intensity, and time decay attention oriented towards time factors. The sentiment dictionary contains a domain vocabulary and sentiment polarity and sentiment intensity labels for each word.

[0039] The output module is used to output the sentiment tags of the public opinion text to be detected.

[0040] According to a third aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described public opinion sentiment detection method based on large model fine-tuning.

[0041] By employing the aforementioned technical solutions, this invention provides a public opinion sentiment detection method based on large-scale model fine-tuning. This method involves segmenting the public opinion text to be detected into sentences, denoising, and word segmentation to generate text sequences and timestamp sequences. These sequences are then input into a public opinion sentiment detection model to obtain the sentiment tag output for the text. The model combines temporal information, text semantics, and a sentiment dictionary for multi-dimensional modeling. This method constructs a temporal sentiment joint embedding module on top of the original embedding layer of the large model. It uses a time decay function to quantify the importance of information changing over time and obtains sentiment features by calculating semantic similarity through a sentiment dictionary. A multi-dimensional attention fusion module is inserted into the Transformer backbone structure to achieve joint modeling of domain knowledge attention, sentiment intensity attention, and temporal decay attention. Through a multi-stage fine-tuning strategy, including temporal sentiment feature pre-training, domain knowledge regularization fine-tuning, and end-to-end joint optimization, the model takes into account temporal evolution, semantic understanding, and domain adaptation. This invention effectively improves the accuracy and timeliness of public opinion sentiment analysis, enhances the model's ability to identify obscure knowledge and industry-specific emotional expressions, and exhibits strong generalization and robustness.

[0042] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0044] Figure 1 A schematic diagram of an architecture based on large model fine-tuning provided in an embodiment of the present invention is shown;

[0045] Figure 2 The diagram illustrates a flowchart of a public opinion sentiment detection method based on large model fine-tuning provided in an embodiment of the present invention.

[0046] Figure 3 A schematic diagram of the multidimensional attention fusion mechanism in an embodiment of the present invention is shown;

[0047] Figure 4 A schematic diagram of the multi-stage fine-tuning strategy training mechanism in an embodiment of the present invention is shown;

[0048] Figure 5 The diagram shows a structural schematic of a public opinion sentiment detection device based on large model fine-tuning provided in an embodiment of the present invention. Detailed Implementation

[0049] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0050] The public opinion sentiment detection model architecture based on large model fine-tuning provided in this embodiment of the invention, such as... Figure 1 As shown, the model comprises two key modules: dynamic word embedding and multi-dimensional attention fusion. First, the public opinion text is segmented into sentences, denoised, and word-segmented to generate text sequences and timestamp sequences. Then, a temporal-sentiment joint embedding module is introduced in the word embedding stage, fusing temporal decay features with sentiment semantic features to form a dynamic embedding input. This embedding uses a Transformer architecture optimized after positional encoding, inserting a multi-dimensional attention fusion sublayer into the original encoder structure. This sublayer integrates domain knowledge attention, sentiment intensity attention, and temporal decay attention, achieving unified modeling of semantic, emotional, and temporal features. After multi-layer normalization, a feedforward network, and linear compression, the output, after temperature-controlled Softmax layer, yields the sentiment classification result, thus achieving temporal awareness and domain-adaptive public opinion sentiment recognition.

[0051] This invention can be widely applied to scenarios such as government public opinion monitoring, social event public opinion analysis, corporate brand reputation tracking, and public opinion risk early warning. Its application process is as follows: Collect public opinion text data and annotate with time information → Input into the preprocessing module to complete text cleaning and word segmentation → Input the text sequence and timestamp sequence into the sentiment detection model → The model extracts time and sentiment features through a time-sentiment embedding layer → Achieve three-dimensional feature modeling of domain, emotion, and time sequence through a multi-dimensional attention fusion module → Complete feature encoding and global semantic understanding through the Transformer backbone network → Output the sentiment label (e.g., positive, neutral, or negative) and its confidence level of the text through temperature Softmax. This process achieves high-precision, interpretable, and timely analysis of public opinion sentiment recognition, and can assist government agencies or enterprises in real-time public opinion monitoring and sentiment-based decision support.

[0052] The present invention will now be described in detail through specific embodiments.

[0053] Example 1:

[0054] like Figure 2 As shown in the figure, this is a public opinion sentiment detection method based on large model fine-tuning provided in an embodiment of the present invention, including:

[0055] Step 201: Obtain the text of public opinion to be detected, and perform sentence segmentation, noise reduction, and word segmentation on the text to generate a text sequence. and the corresponding timestamp sequence

[0056] Where each word x i For the i-th word, text x i The corresponding time identifier is t i ;

[0057] Step 202: Input the text sequence and the corresponding timestamp sequence into the public opinion sentiment detection model to obtain the sentiment tags of the public opinion text;

[0058] The sentiment detection model generates dynamic word embeddings that fuse time decay features and sentiment semantics based on text sequence X, timestamp sequence T, and sentiment dictionary C. The dynamic word embeddings are then input into the multi-dimensional attention fusion module in the Transformer architecture to obtain fused features. The multi-dimensional attention fusion module includes domain knowledge attention oriented towards the domain dictionary, sentiment intensity attention oriented towards polarity intensity, and time decay attention oriented towards time factors. The sentiment dictionary contains a domain vocabulary and sentiment polarity and sentiment intensity labels for each word.

[0059] The steps for generating dynamic word embeddings include:

[0060] Step 202-1: Based on the word embedding layer of the public opinion sentiment detection model, obtain the original word embedding E(x) i )∈R d ;

[0061] Step 202-2: Calculate the monotonically decreasing time decay coefficient α for the timestamp of each word. i , where α i Let α be the event decay coefficient for the i-th word. i α is used to measure the temporal impact weight of a token on overall sentiment assessment. i =exp(-μ·Δt) i ), μ is the time decay coefficient, Δt i For the standardized time interval, t i The timestamp of the text or sentence containing the token, t max With t min ε represents the latest and earliest timestamps in the text of public opinion to be detected, respectively. ε is a small constant to prevent the denominator from being zero. The time decay coefficient reflects the timeliness law that "the newer the information, the greater the weight, and the older the information, the smaller the weight", so that the model automatically emphasizes the dominant role of recent public opinion changes in sentiment judgment in sentiment analysis.

[0062] Step 202-1: Based on the sentiment dictionary C = {C k}Calculation word x i semantic similarity S(x) i C k ), Among them, Ck Let k be the k-th sentiment center, mean(E(C) k Let C be the mean vector of the sentiment dictionary entries, and ||·||2 be the L2 norm of the vector. The sentiment dictionary C is obtained by combining the expansion of a general sentiment lexicon with adaptive construction of domain corpora. First, existing open-source sentiment dictionaries (such as NTUSD, HowNet, and Sentiment Knowledge Network) are selected as the basic vocabulary set to provide initial sentiment polarity and intensity annotations. Second, combined with historical corpora of the target public opinion domain (such as government affairs, finance, and energy), domain-specific sentiment expression words and implicit emotion words are mined through co-occurrence analysis, word vector clustering, and sentiment transfer learning methods. Then, expert annotation and statistical learning methods are used to correct the polarity and intensity of these words to form a high-quality domain-extended dictionary. Finally, the general and domain words are uniformly encoded into vector form, and their mean vector C is calculated. k Serving as the semantic center for each emotion category, the constructed emotion dictionary C not only contains general emotion expressions but also accurately reflects industry-specific semantic features, providing the model with rich prior knowledge of emotional semantics.

[0063] Step 204-1: Embed the original words into E(x) i Time decay coefficient α i , The words are merged to form a dynamic word embedding E′(x). i ),in, For word x i With the various categories of emotion centers C in the emotion dictionary k Semantic similarity by weight w k Weighted summation, where m is the number of sentiment categories.

[0064]

[0065] Step 203: Output the sentiment tags of the public opinion text to be detected.

[0066] Taking government public opinion monitoring as an example, when the input text to be detected is "Citizens in a certain area reported severe road flooding, and multiple complaints have not been effectively addressed," the model, after sentence segmentation, word segmentation, and timestamp annotation, first extracts the semantic and temporal features of the text, and then analyzes them in conjunction with the intensity features of negative sentiment words such as "complaint" and "unresolved" in the domain sentiment lexicon. After calculation by the Transformer backbone network and the multi-dimensional attention fusion module, the model outputs the following sentiment tag structure:

[0067] Output = {label:'negative',confidence:0.94,timestamp:'2025-10-22T14:36:00',domain:'government_public_service'}, where "label" represents the sentiment polarity category (e.g., positive, negative, neutral), "confidence" represents the model's prediction confidence, "timestamp" indicates the time information of the corresponding public opinion, and "domain" represents the public opinion domain. This structured output can be directly used for sentiment statistical analysis, trend monitoring, and public opinion risk early warning systems to achieve efficient and interpretable sentiment recognition results.

[0068] This invention provides a public opinion sentiment detection method based on large model fine-tuning. The method involves segmenting the public opinion text to be detected into sentences, denoising, and word segmentation to generate text sequences and timestamp sequences. These sequences are then input into a public opinion sentiment detection model to obtain the sentiment tag output. The model combines time information, text semantics, and a sentiment dictionary for multi-dimensional modeling. This method constructs a temporal sentiment joint embedding module on top of the original embedding layer of the large model. It uses a time decay function to quantify the importance of information changing over time and obtains sentiment features by calculating semantic similarity through a sentiment dictionary. A multi-dimensional attention fusion module is inserted into the Transformer backbone structure to achieve joint modeling of domain knowledge attention, sentiment intensity attention, and temporal decay attention. Through a multi-stage fine-tuning strategy, including temporal sentiment feature pre-training, domain knowledge regularization fine-tuning, and end-to-end joint optimization, the model takes into account temporal evolution, semantic understanding, and domain adaptation. This invention effectively improves the accuracy and timeliness of public opinion sentiment analysis, enhances the model's ability to identify obscure knowledge and industry-specific emotional expressions, and has strong generalization and robustness.

[0069] Example 2:

[0070] The dynamic words are embedded into the multi-dimensional attention fusion module of the Transformer architecture to obtain fused features. The process is as follows: Figure 3 As shown, it includes:

[0071] Embed dynamic words into E′(x) i The mean vector of all word embeddings in the industry sentiment dictionary D. Through isomorphic linear mapping matrix W d The first weight is obtained by scoring the dot product after projection and calculating the softmax. The first weight reflects the degree of semantic matching between the input word and the domain sentiment dictionary, where, As a feature dimension normalization term, the Industry Sentiment Dictionary is a domain-specific sentiment knowledge base built for specific public opinion application areas. The Industry Sentiment Dictionary includes common emotional expressions, industry terms, and their corresponding sentiment polarity and intensity annotations. Unlike general sentiment dictionaries, the Industry Sentiment Dictionary focuses on reflecting the specific emotional meaning of words in the industry context. For example, "limit up" and "positive news" are positive words in the financial field, while "complaint" and "inaction" are negative words in the government field. Obtaining this dictionary typically involves three steps:

[0072] ① Candidate words are automatically selected from a large amount of domain-specific corpus using keyword extraction and sentiment co-occurrence analysis;

[0073] ② Polarity transfer annotation and manual review are performed using existing general sentiment dictionaries;

[0074] ③ Calculate the sentiment center vector and mean vector for each word through clustering, sentiment scoring, or word vector similarity analysis to form an embedding representation that can be used as model input. The introduction of an industry-specific sentiment dictionary enables the model to possess domain-prior knowledge during sentiment recognition, thereby more accurately capturing industry-specific emotional tendencies;

[0075] The dot product, also known as the inner product or scalar product, is a mathematical operation used to calculate the similarity or alignment between two vectors. The dot product is calculated by multiplying the corresponding dimensions of the two vectors and then summing all the products. The result of the dot product is a scalar, i.e., a single numerical value. The magnitude of this value (usually in unnormalized form) reflects how much the current contextual meaning of the dynamic word "points in the same direction" as the vector representing the overall sentiment of the industry.

[0076] In this step, the softmax operation is used to normalize the attention scores of all words, obtaining weights in the form of a probability distribution. Through this calculation, the model can automatically assign attention weights based on the semantic similarity between words and the domain sentiment center, giving higher weights to words more relevant to the domain sentiment, thereby strengthening the domain-specific sentiment recognition capability.

[0077] The second weight is generated based on the sentiment polarity / intensity dictionary P. Where, E′(x i W represents the dynamic word embedding vector of the i-th word. p b is a learnable weight matrix for emotion intensity and attention. p Here, δ(·) is the bias term, and E(p(x) is the nonlinear activation function. i ) indicates word x i The emotional intensity embedding, p(x) iLet be the corresponding entry of the word in the sentiment polarity / intensity dictionary P. The sentiment polarity / intensity dictionary P is a set containing positive, negative, and neutral sentiment words and their intensity annotations. It is usually constructed by manual annotation, sentiment corpus statistics, and word vector clustering methods. It can be combined with existing sentiment dictionaries (such as Hownet, NTUSD, and Sentiment Knowledge Network) and expanded based on the target public opinion domain corpus. Through the second weight formula, the model can learn the sentiment intensity weight corresponding to each word in the semantic context, thereby highlighting the contribution of sentiment-sensitive words to sentiment classification.

[0078] A third weight is generated based on the time decay factor and the first and second weights. Where μ is the time decay coefficient, rank(t) i The rank value represents the relative ranking position of the timestamp corresponding to the i-th word within the entire text sequence (the earlier the timestamp, the higher the rank value). The first weight is used to reflect the relevance of the word to the domain dictionary. The second weight represents the intensity of the emotional polarity of a word;

[0079] By using a third weight, the global context features obtained from the Transformer encoder in the public opinion sentiment detection model network architecture are weighted and fused together. i Global context features of the Long Short-Term Memory (LSTM) network in the network architecture of the public opinion sentiment detection model (E(x)). i )) to obtain fusion features As the third weight, E(x) i () is the original word embedding.

[0080] Final fusion feature h i The role of the LSTM in the multidimensional attention sublayer is to serve as the contextual semantic expression input after integrating temporal, sentiment, and semantic information, guiding subsequent attention weight allocation and sentiment classification decisions. In terms of network architecture, LSTM and Transformer coexist in parallel: LSTM is responsible for capturing the sequential dependencies and temporal evolution patterns of the text, modeling the temporal changes in public sentiment, and extracting temporal contextual features that can capture the sequential relationships of sentiment changes. This formula achieves a dynamic balance between temporal perception and global semantic features at the semantic level by weighted fusion of the two features, allowing recent public sentiment information with higher time decay weights to rely more on the semantic modeling of Transformer, while earlier information retains more of the temporal sequence memory of LSTM. This improves the model's ability to express and stabilize the emotional trends of public sentiment evolution at the semantic level. Transformer is responsible for extracting the global semantics and cross-sentence dependency information of the text, strengthening the semantic connections between multiple words. The outputs of both are processed through a third weight. Weighted fusion is performed to form the final h. i In the multidimensional attention sublayer, the comprehensive feature vector is fed into the subsequent domain knowledge attention, sentiment intensity attention, and time-series decay attention modules. This enables the attention mechanism to simultaneously focus on timeliness features, sentiment polarity features, and domain semantic features, achieving dynamic synergy and information complementarity among features of different dimensions. This enhances the model's sentiment recognition accuracy and robustness in complex public opinion scenarios.

[0081] Example 3:

[0082] The public opinion sentiment detection model in this invention directly addresses technical problems in public opinion sentiment analysis, such as poor domain adaptability, insufficient temporal awareness, and weak sentiment expression. Addressing the characteristics of public opinion data—its temporal dynamism, semantic complexity, and domain diversity—this invention employs a multi-stage fine-tuning strategy to train the public opinion sentiment detection model. First, time-sentiment feature pre-training establishes the model's perception of temporal and emotional correlations. Then, domain knowledge is introduced for fine-tuning to inject industry-specific semantic priors. Finally, end-to-end joint optimization achieves global parameter convergence and multi-source feature fusion. This strategy enables the model to maintain its general semantic understanding capabilities while gradually mastering the temporal evolution and sentiment polarity expression of public opinion texts, fundamentally improving the accuracy, stability, and domain generalization ability of sentiment recognition. The multi-stage fine-tuning strategy for training the public opinion sentiment detection model includes time-sentiment feature pre-training, domain knowledge fine-tuning, and end-to-end joint optimization. The time-sentiment feature pre-training minimizes the time-sentiment alignment loss L. time-sent This makes the weights or attention of the sentiment detection model sensitive to time decay and sentiment intensity; domain knowledge fine-tuning is achieved by introducing domain knowledge regularization L... domain To improve industry adaptability, we inject domain prior distributions; we perform end-to-end joint optimization to achieve joint loss convergence; and we apply a temperature annealing strategy at the classification head / attention score to optimize the classification boundary.

[0083] The pre-training of time-based sentiment features includes:

[0084] Provide a time-sentiment weighting benchmark for each word Used to supervise the alignment of the model with temporal decay and sentiment semantics;

[0085] Minimize temporal sentiment alignment loss in, The weight distribution is derived from dynamic embedding / attention, where N is the total number of samples and n i Let P be the sequence length of the i-th sample. θ (x i,t |x i,1:t-1 This indicates that, with model parameters θ, the prediction of word x at the current position t is calculated.i,t The conditional probability, x i,1:t-1 This represents the sequence of the i-th sample from the first word to the (t-1)-th word, which is used to predict the current word x. i,t The preceding context upon which the time model depends, -logP θ Used to measure language modeling error The time decay weight of the i-th sample at time t calculated by the model. The target time decay reference value is generated by manual annotation or rules, and λ1 is a balance coefficient used to control the relative weight of language modeling error and time alignment error. This represents the Euclidean distance between the model's time decay prediction and the target.

[0086] This represents the time-sentiment weight distribution calculated jointly by the dynamic embedding layer and the attention mechanism during model training. It quantifies the comprehensive impact of each word on the overall sentiment judgment in the current context. Specifically, this weight is derived from two parts: first, the time decay weight from the dynamic word embedding, reflecting the importance of the freshness of information to the sentiment judgment; and second, the attention weight from the multi-dimensional attention fusion module, including domain knowledge attention and sentiment intensity attention, used to reflect the salience of words in semantic and sentiment dimensions. The model automatically learns the distribution of these weights during training, giving higher attention values ​​to semantically relevant, emotionally strong, and more timely words. This achieves unified modeling of time, sentiment, and semantic features, providing a dynamic and interpretable weight basis for sentiment classification. For example, in the text "Netizens say that the city's road construction delays have caused travel difficulties," the model calculates higher time weights for the more recent words "delay" and "difficulty" through dynamic embedding, and identifies their high similarity to negative sentiment dictionary entries through the attention mechanism, thus assigning these two words higher weights. Values ​​are assigned accordingly; conversely, words with earlier dates or semantically neutral meanings, such as "road construction," receive lower weights. This results in... The value not only guides the model to focus on core emotional words and recent public opinion information during training, but also provides a quantitative basis for time-sentiment alignment in the subsequent loss function, improving the model's ability to perceive the trend of emotional evolution.

[0087] The preferred approach is to first freeze the backbone network of the public opinion sentiment detection model and then train only the dynamically embedded relevant parameters to stably introduce temporal sentiment priors.

[0088] Domain knowledge fine-tuning includes:

[0089] Differences between word domain matching distribution and domain prior using the Kullback–Leibler divergence constraint model Among them, W dThe isomorphic linear mapping matrix is ​​used to map word embeddings and neighborhood centers to the same space for comparability, and KL(·) is the Kullback–Leibler divergence. The mean vector of all word embeddings in the industry sentiment dictionary D is used as the overall representation of the domain sentiment.

[0090] Minimize knowledge fine-tuning L finetunine =L ce +αL domain , where L ce Cross-entropy loss;

[0091] The parameters of the dynamically embedded layer that have been trained in the pre-training of temporal features are frozen, and some Transformer layers and fusion modules are unfrozen so that the regularization terms continue to influence attention allocation and intermediate representation.

[0092] End-to-end joint optimization includes:

[0093] Minimize the joint loss of knowledge fine-tuning L = L ce +αL domain +βL time-sent , where L ce For cross-entropy loss, L domain To illustrate the difference between the word domain matching distribution and the domain prior in the Kullback–Leibler divergence-constrained model, L time-sent Loss of time-emotional alignment;

[0094] Temperature scaling is applied at the classification layer or attention score, with the temperature decreasing with each training epoch, where T... t =T0·e (-γ·epoch) T t Let T0 be the current temperature value during the t-th training round, T0 be the initial temperature value, γ be the annealing rate constant, and epoch be the current training round. This strategy uses exponential decay to gradually reduce the temperature during training, thereby maintaining a relatively high temperature in the early stages to promote exploration and smooth probability distribution, and avoid the model getting stuck in local optima. In the later stages, as the temperature decreases, the determinism of the classification boundary is enhanced, making the class probabilities output by the model more concentrated and confident, thus realizing a dynamic optimization process from "global search" to "fine convergence".

[0095] Output the class probability of the classification With smooth boundaries and stable confidence, p(y|F) represents the probability that an input sample is classified as sentiment category y under feature F, where F... y F represents the model output score for the corresponding category y. k The output score is for the k-th category, where K is the total number of emotion categories and T is the temperature adjustment parameter.

[0096] Simultaneously, the dynamic embedding, multi-dimensional attention fusion module, and Transformer backbone and classification head parameters are updated to achieve unified convergence of semantic-temporal-domain three-dimensional features.

[0097] Furthermore, as Figures 2 to 4 In a specific implementation of the method, this invention provides a public opinion sentiment detection device based on large model fine-tuning, such as... Figure 5 As shown, the device includes:

[0098] The acquisition module 510 is used to acquire the public opinion text to be detected, and to perform sentence segmentation, noise reduction, and word segmentation on the public opinion text to be detected, generating a text sequence. and the corresponding timestamp sequence Where each word x i For the i-th word, text x i The corresponding time identifier is t i ;

[0099] The detection module 520 is used to input the text sequence and the corresponding timestamp sequence into the public opinion sentiment detection model to obtain the sentiment tags of the public opinion text. The public opinion sentiment detection model is used to generate dynamic word embeddings that fuse time decay features and sentiment semantics based on the text sequence X, the timestamp sequence T, and the sentiment dictionary C. The dynamic word embeddings are input into the multi-dimensional attention fusion module in the Transformer architecture to obtain fusion features. The multi-dimensional attention fusion module includes domain knowledge attention oriented towards the domain dictionary, sentiment intensity attention oriented towards polarity intensity, and time decay attention oriented towards time factors. The sentiment dictionary contains a domain vocabulary and sentiment polarity and sentiment intensity labels corresponding to each word.

[0100] Output module 530 is used to output the sentiment tags of the public opinion text to be detected.

[0101] This invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a public opinion sentiment detection method based on large model fine-tuning, including:

[0102] The system acquires the text of public opinion to be detected, and performs sentence segmentation, noise reduction, and word segmentation on the text to generate a text sequence. and the corresponding timestamp sequence Where each word x i For the i-th word, text x i The corresponding time identifier is t i ;

[0103] The text sequence and its corresponding timestamp sequence are input into the public opinion sentiment detection model to obtain the sentiment tags of the public opinion text. The public opinion sentiment detection model is used to generate dynamic word embeddings that fuse time decay features and sentiment semantics based on the text sequence X, timestamp sequence T, and sentiment dictionary C. The dynamic word embeddings are input into the multi-dimensional attention fusion module in the Transformer architecture to obtain fusion features. The multi-dimensional attention fusion module includes domain knowledge attention oriented towards the domain dictionary, sentiment intensity attention oriented towards polarity intensity, and time decay attention oriented towards time factors. The sentiment dictionary contains a domain vocabulary and sentiment polarity and sentiment intensity labels corresponding to each word.

[0104] Output the sentiment tags of the text to be detected.

[0105] It should be noted that the above embodiments only use sentiment analysis of public opinion text as an example to illustrate the principles and implementation steps of the present invention, and do not limit specific application scenarios. The present invention is also applicable to other fields that require text sentiment recognition, sentiment trend monitoring, user sentiment profiling, event-based public opinion dissemination analysis, and intelligent question-and-answer sentiment understanding based on large models. By adjusting the sentiment dictionary, domain knowledge base, and time decay parameters, it can be flexibly extended to various vertical scenarios such as government affairs, finance, energy, education, and healthcare, achieving temporal-sentiment joint modeling of different types of text. For details regarding the functions or steps achievable by computer-readable storage media or computer devices, please refer to the descriptions in the foregoing method embodiments; they will not be repeated here.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0108] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A public opinion sentiment detection method based on large model fine-tuning, characterized in that, The method comprises: Acquire a to-be-detected public opinion text, and perform sentence division, denoising and word segmentation processing on the to-be-detected public opinion text to generate a text sequence And a corresponding timestamp sequence Wherein each word x i Is the i th word, and the text x i Corresponding time identifier t i ; inputting the text sequence and the corresponding timestamp sequence into an opinion emotion detection model to obtain an emotion label of the opinion text, wherein the opinion emotion detection model is used to generate dynamic word embedding fusing time decay feature and emotion semantics according to the text sequence X, the timestamp sequence T and an emotion dictionary C; the dynamic word embedding is input into a multi-dimensional attention fusion module in a Transformer architecture to obtain fusion features, the multi-dimensional attention fusion module comprises domain knowledge attention facing a domain dictionary, emotion intensity attention facing polarity intensity, and time sequence decay attention facing a time factor, wherein the emotion dictionary contains a domain vocabulary and emotion polarity and emotion intensity labels corresponding to each vocabulary; outputting the emotion label of the opinion text to be detected.

2. The public opinion sentiment detection method based on large model fine-tuning according to claim 1, characterized in that, According to the text sequence, the timestamp sequence and the emotion dictionary, the dynamic word embedding fusing time decay feature and emotion semantics is generated, which comprises: Based on the word embedding layer of the public opinion sentiment detection model, obtain the original word embedding E(x i )∈R d ; The timestamp of each word is used to calculate a monotonically decreasing time decay coefficient a i where a i is the event decay coefficient for the ith word, a i is used to measure the time influence weight of token on the overall sentiment judgment, a i = exp(-μ·Δt i )·t μ is a time decay coefficient, Δt i is a normalized time interval, t i is a time stamp of the text or sentence where the token is located, t max and t min represent the latest and earliest time stamp in the text to be detected, respectively, and ε is a small constant to prevent the denominator from being zero. The semantic similarity S(x i ,C i ) of the word x k is calculated based on an emotion dictionary C={C k , k=1,2,…,K}, wherein C k is the kth emotion center, mean(E(C i )) is the mean of the emotion dictionary entry vector, and ||·||2 is the L2 norm of the vector. The original word embedding E(x i ) is fused with the time decay coefficient α i , to form a dynamic word embedding E'(x i ), wherein, is a weighted sum of the semantic similarity between the word x i and the center C k of each category of sentiment in the sentiment dictionary, weighted by w k , and m is the number of sentiment categories.

3. The public opinion sentiment detection method based on large model fine-tuning according to claim 1, characterized in that, The dynamic word embedding is input into a multi-dimensional attention fusion module in a Transformer architecture to obtain fusion features, which comprises: dynamic word embedding E'(x i ) is projected onto the mean vector of all word embeddings in the industry sentiment dictionary D is projected onto the mean vector of all word embeddings in the industry sentiment dictionary D d The first weight is obtained after the dot product scoring and softmax calculation after projection The first weight reflects the semantic matching degree of the input word and the domain sentiment dictionary, wherein, is a feature dimension normalization term, the industry sentiment dictionary D is a domain-specific sentiment knowledge base constructed for a specific public opinion application domain, and the content of the industry sentiment dictionary D includes common emotional expression words, industry terms, and corresponding sentiment polarity and intensity annotations in the domain. generating a second weight based on a sentiment polarity / intensity lexicon P where E'(x i ) denotes the dynamic word embedding vector of the i-th word, W p is a learnable weight matrix for sentiment intensity attention, b p is a bias term, δ(·) is a non-linear activation function, E(p(x i ) denotes the sentiment intensity embedding of the word x i , p(x i ) is the corresponding entry of the word in the sentiment polarity / intensity lexicon P, which is a set containing positive, negative and neutral sentiment words with their intensity annotations. generate the third weight based on a time decay factor and the first weight, the second weight wherein μ is a time decay coefficient, rank(t i ) represents the relative ranking position of the timestamp corresponding to the i-th word in the whole text sequence, is the first weight, is the second weight; The global context feature Transformer (E(x i )) obtained by the third weight is fused with the global context feature LSTM (E(x i )) of the long short-term memory network LSTM in the public opinion sentiment detection model network architecture, to obtain a fusion feature The third weight is E(x i ), and the original word embedding is E(x 4. The public opinion sentiment detection method based on large model fine-tuning according to claim 1, characterized in that, The public opinion sentiment detection model is trained by using a multi-stage fine-tuning strategy, including time sentiment feature pre-training, domain knowledge fine-tuning, and end-to-end joint optimization, wherein the time sentiment feature pre-training is performed by minimizing a time sentiment alignment loss L time-sent The weight or attention of the public opinion sentiment detection model is made to be sensitive to time decay and sentiment intensity; the domain knowledge fine-tuning is performed by introducing a domain knowledge regular L domain A domain prior distribution is injected to improve industry adaptability; and the end-to-end joint optimization is performed to converge in a joint loss, and a temperature annealing strategy is applied at a classification head / attention score to optimize a classification boundary.

5. The public opinion sentiment detection method based on large model fine-tuning according to claim 4, characterized in that, The time emotion feature pre-training comprises: providing a time sentiment weight baseline for each word for supervising alignment of time decay with sentiment semantics for a model minimizing temporal sentiment alignment loss wherein, is the weight distribution derived by dynamic embedding / attention, where N is the total number of samples, n i is the sequence length of the i-th sample, P θ (x i,t | x i,1:t-1 ) represents the conditional probability of predicting the word x i,t at the current position t under the model parameters θ, x i,1:t-1 represents the sequence of the i-th sample from the 1st word to the t-1th word, i.e., the preceding context on which the model relies when predicting the current word x i,t , -logP θ is used to measure the language modeling error, is the time decay weight of the i-th sample at time t calculated by the model, is the target time decay reference value generated by manual annotation or rules, λ1 is a balance coefficient used to control the relative weight of the language modeling error and the time alignment error; represents the Euclidean distance between the model time decay prediction and the target. Preferably, the opinion emotion detection model backbone network is first frozen, and only the dynamic embedding related parameters are trained to stabilize the introduction of time emotion prior.

6. The public opinion sentiment detection method based on large model fine-tuning according to claim 5, characterized in that, The domain knowledge fine-tuning comprises: Word domain matching distribution with the difference between the model constrained by Kullback-Leibler divergence and the domain prior where W d is an isomorphic linear mapping matrix used to map word embeddings and domain centers into the same space for comparability, KL(·) is the Kullback-Leibler divergence, is the mean vector of all word embeddings in the industry sentiment lexicon D as the overall representation of domain sentiment; Minimize knowledge fine-tuning L finetunine = L ce + aL domain where L ce is the cross-entropy loss; The dynamic embedding layer parameters trained in the time emotion feature pre-training are frozen, and part of the Transformer layer and the fusion module are unfrozen to enable the regular term to continuously affect attention allocation and intermediate representation.

7. The public opinion sentiment detection method based on large model fine-tuning according to claim 6, characterized in that, The end-to-end joint optimization comprises: minimizing the knowledge fine-tuning joint loss L = L ce + aL domain + bL time-sent where L ce is the cross-entropy loss, L domain is the Kullback-Leibler divergence constraining the difference between the word domain matching distribution of the model and the domain prior, and L time-sent is the temporal sentiment alignment loss. Applying temperature scaling at the classification layer or attention scores, with temperature decreasing with training epoch, where T t = T0· e (-γ·epoch) , T t is the current temperature value at the t-th training epoch, T0is the initial temperature value, γ is the annealing rate constant, and epoch is the current training epoch. Class probabilities of an output classification with smooth boundaries and stable confidence, where p(y|F) denotes the probability that the input sample is classified as sentiment class y under feature F, F y denotes the model output score for the corresponding class y, F k is the output score for the k-th class, K is the total number of sentiment classes, and T is a temperature regulation parameter; The dynamic embedding, the multi-dimensional attention fusion module, the Transformer backbone and the classification head parameters are updated at the same time to realize unified convergence of semantic-time-domain three-dimensional features.

8. An opinion sentiment detection device based on large model fine-tuning, characterized in that, The method comprises: An acquisition module is configured to acquire a to-be-detected public opinion text, and perform sentence division, noise removal and word segmentation on the to-be-detected public opinion text to generate a text sequence and a corresponding timestamp sequence wherein each word x i is an i-th word, and a text x i corresponds to a time identifier t i ; The detection module is used to input the text sequence and the corresponding timestamp sequence into an opinion emotion detection model to obtain an emotion label of the opinion text, wherein the opinion emotion detection model is used to generate dynamic word embedding fusing time decay feature and emotion semantics according to the text sequence X, the timestamp sequence T and an emotion dictionary C; the dynamic word embedding is input into a multi-dimensional attention fusion module in a Transformer architecture to obtain fusion features, the multi-dimensional attention fusion module comprises domain knowledge attention facing a domain dictionary, emotion intensity attention facing polarity intensity, and time sequence decay attention facing a time factor, wherein the emotion dictionary contains a domain vocabulary and emotion polarity and emotion intensity labels corresponding to each vocabulary; The output module is used to output the emotion label of the opinion text to be detected.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the opinion emotion detection steps based on large model fine-tuning in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to realize the opinion emotion detection steps based on large model fine-tuning in any one of claims 1 to 7.

Images