Generative AI emotion propagation prediction and guidance large model construction method and system

By constructing a grammatical, semantic and memory-driven emotion analysis model, combining multi-task learning and reinforcement learning, the accuracy and timeliness of traditional methods in social media sentiment analysis are solved, and efficient capture and implementation of complex emotional expressions are achieved. The prediction of dynamic emotional changes has improved the intelligence level of public opinion management.

CN119047512AActive Publication Date: 2024-11-29BEIJING UNIV OF POSTS & TELECOMM

Patent Information

Application Number
CN202411160140.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-29
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

Traditional emotions analysis methods lack accuracy and timeliness when facing massive social media data, and cannot effectively capture complex emotional expressions and emotional dynamic changes, especially in complex public opinion environments, which are difficult to meet demand.

Method used

Build a large-scale emotion analysis model based on grammar, semantics and memory-driven emotion analysis model, use deep pre-trained models and memory mechanisms, combine multi-task learning and reinforcement learning to generate personalized emotion guidance strategies, and use sparse gated hybrid expert network and DeepSpeed-MoE model Improve training efficiency and accuracy.

Benefits of technology

It realizes accurate analysis and efficient guidance of social media emotions, improves the intelligence level of public opinion management, provides stable and effective emotional guidance strategies and content, ensures the consistency and traceability of guidance effects, and enhances public opinion response capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119047512A_ABST
    Figure CN119047512A_ABST
Patent Text Reader

Abstract

The invention provides a generative AI-based emotion propagation prediction and guidance large model construction method and system, and the method comprises the steps: firstly constructing an emotion analysis large model, employing a large language model and multi-source social media data, fusing grammar, semantics and memory driving, employing a sparse gating hybrid expert training technology, and improving the training efficiency and performance; secondly, public opinion event propagation prediction is carried out based on an emotion analysis large model, a controlled product for multi-dimensional guidance of individual emotion and propagation content is generated, propagation prediction is carried out, and an emotion guidance strategy and content are generated at the same time; and finally, a comprehensive system is constructed for demonstration verification, network public opinion event propagation is analyzed based on spatial-temporal characteristics for diversified sudden public opinion events, empirical analysis is carried out, and the precision and efficiency of a prediction model are improved. The method breaks through the limitation of an existing method in the aspects of processing complexity, multi-modal emotional expression and dynamic change prediction guidance, and is helpful for finding and dredging social contradictions and maintaining social stability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of AI big models, sentiment analysis and prediction, and public opinion dissemination technology, and in particular to a method and system for building a generative AI sentiment dissemination prediction and guidance big model. Background Art

[0002] With the rapid development of the Internet and social media, the Internet has become an important platform for people to express emotions, exchange views, and participate in public opinion. Individual emotions in cyberspace show new characteristics of frequent expression, wide dissemination, and interactive influence. The analysis of individual emotions is not only related to the individual level, but also a micro-manifestation of the dynamic changes of public opinion at the national governance level, reflecting the dynamic changes of society and the tendency of public opinion, which provides an important basis for government departments to adjust policies and respond to public opinion in a timely manner. The "2024 Government Work Report" includes "improving the level of public security governance and promoting the transformation of governance models to pre-prevention" as a work task. Further analysis of individual emotions and guidance of negative emotions can help the government respond to social conflicts and emergencies in a timely and effective manner and maintain social stability and harmony. However, traditional sentiment analysis methods are stretched when faced with massive social media data, lack sufficient accuracy and timeliness, and are difficult to meet the needs of the current complex public opinion environment.

[0003] Traditional sentiment analysis methods usually rely on the following technical means: keyword matching and sentiment dictionary. This method uses predefined keywords and sentiment dictionary to identify sentiment tendencies in text. This method is simple and direct, but it is powerless for diverse and complex emotional expressions and cannot capture context and implicit emotions; machine learning classifiers. Based on feature engineering and machine learning classifiers (such as SVM, naive Bayes, etc.), text features are manually extracted and classifiers are trained to identify emotions. This method requires a lot of feature engineering and is highly dependent on the quality of feature extraction, and has limited generalization ability; shallow neural networks. Although shallow neural networks can automatically learn some features for sentiment analysis, they are still insufficient when facing long texts, complex contexts and sentiment evolution, and cannot handle long-range dependencies and multimodal data. More importantly, traditional methods cannot make full use of important factors such as information diffusion and user social relations in social networks, and it is difficult to deeply explore the polarization and contagion mechanisms of individual emotions in dynamic communication, as well as the heterogeneous evolution patterns of emotions of different user groups. At the same time, there is a lack of personalized emotion guidance strategies that are dynamically adjusted in combination with user feedback, which makes it difficult to provide strong support for the intelligent upgrade of public opinion management.

[0004] At present, the rapid development of information technology represented by big data and artificial intelligence has brought new opportunities for intelligent analysis and precise guidance of online public opinion. Large language models (such as GPT-4, Claude-3, etc.) have greatly improved the machine's ability to understand and generate natural language, laying the foundation for applications such as sentiment analysis, communication tracking, and intelligent question and answer. The large model can better understand emotional semantics and model the evolution of emotions through massive data pre-training. Using deep learning networks, large models can automatically learn emotional features, overcoming the complex problem of feature engineering in traditional methods. The introduction of memory mechanisms enables the model to capture the long-term dependence of emotional information, breaking through the bottleneck of the limited time scale of existing methods. The present invention intends to achieve organic integration at the three levels of semantic understanding, grammatical analysis, and memory drive, construct a large model of emotional analysis for individual expressions on social media, accurately analyze the emotional state of individuals, reveal their evolution laws and their influence mechanisms in the time and space dimensions, and accurately perceive the public's emotional situation. Related research is not only conducive to discovering and guiding social contradictions and maintaining social stability, but also provides a basis for scientific decision-making and precise policy implementation by the government.

[0005] In order to understand the mechanism of emotion generation and evolution, improve the accuracy of online public opinion analysis and prediction, reasonably guide individual emotions in social media, and then improve the social governance system to meet the strategic needs of national cyberspace security and social governance modernization, the present invention will construct a semantic, grammatical and memory-driven emotion analysis model, generate multi-dimensional controlled products guided by emotions, and build an evaluation and prediction index system for the spread of online public opinion events. The present invention will be based on the core scientific issues of intelligent analysis and guidance optimization of individual emotions in a complex social network environment, break through the bottleneck of semantic understanding in emotion representation learning, consider the information diffusion of social networks and user social relations, explore the polarization and contagion mechanism of individual emotions in dynamic propagation, reveal the heterogeneous evolution mode and mutual influence of emotions of different user groups, characterize the spatiotemporal cognitive mode of individual and group emotions in social networks, explore personalized emotion guidance strategies dynamically adjusted in combination with user feedback, guide the public opinion situation perception, propagation path tracing, risk assessment and early warning of Internet platforms, promote the intelligent upgrading of public opinion management methods, and provide key technical support for strengthening the comprehensive governance of cyberspace and building a harmonious and clear network ecology. Summary of the invention

[0006] In view of this, the embodiment of the present invention provides a generative AI emotion propagation prediction and guidance large model construction method and system. The present invention is based on big data and artificial intelligence technology, breaks through the bottleneck of traditional methods, and has the following innovations and advantages:

[0007] Deep pre-training model: Using large language models (such as GPT-4 and Claude-3) for massive data pre-training can deeply understand emotional semantics, automatically learn emotional features, and avoid tedious feature engineering;

[0008] Memory mechanism: The memory mechanism is introduced to enable the model to capture the long-term dependency of emotional information, breaking through the bottleneck of the time scale of existing methods;

[0009] Combination of multi-task learning and reinforcement learning: In the multi-task learning framework, sentiment analysis and text generation tasks are combined, and multi-task fine-tuning is achieved by weighted combination of the loss functions of multiple sub-tasks. Reinforcement learning (RLHF) is combined with human feedback, and the reward model is fine-tuned to make the emotion-guided content generated by the model more in line with user needs and actual scenarios;

[0010] Personalized emotion counseling strategies: By exploring the polarization and contagion mechanisms of individual emotions in dynamic communication, and dynamically adjusting personalized emotion counseling strategies based on user feedback, we can guide the Internet platform's public opinion situation awareness, communication path tracing, and risk assessment and early warning, thereby improving the level of intelligent public opinion management.

[0011] The present invention proposes to construct a semantic, grammatical and memory-driven generative sentiment analysis large model to address the limitations of existing sentiment analysis and communication prediction methods, such as insufficient context understanding, difficulty in capturing implicit emotions, poor flexibility, and inability to process long texts, especially the insufficient processing of complex and multimodal emotional expressions on social media and inaccurate prediction and guidance of dynamic changes in emotions.

[0012] The present invention is both innovative and practical. For the first time, the present invention applies the fusion of grammatical, semantic and memory multi-level information to social media sentiment analysis, and adopts the improved DeepSpeed-MoE model training technology with multi-expert and multi-data parallelism, simplified routing, load balancing, gradient quantization and asynchronous training to significantly improve the training efficiency and accuracy of large models. By constructing a multi-dimensional guidance strategy and sentiment propagation prediction model, the ability to guide and respond to social media public opinion is effectively improved, which has important practical application value.

[0013] The present invention provides a generative AI emotion propagation prediction and guidance large model construction method, characterized in that the construction of the large model and system includes the following steps:

[0014] Step 1: Build a sentiment analysis model based on grammatical semantics and memory drive for social media. Use the language model to model grammatical structure and meaning, and combine the storage and traceability of historical information. Use the open source Chinese language model as the basis, combine social media data, and use sparse gated hybrid expert training technology to build a sentiment analysis model to understand the emotions expressed by individuals on social media and generate responses.

[0015] Step 2: Carry out the propagation prediction of public opinion events based on the sentiment analysis model, collect and preprocess social media data, extract sentiment features and propagation paths, build a propagation model based on graph theory and complex network theory, simulate the process of information propagation in social networks, use LSTM and random forest to train and optimize the model to improve the prediction accuracy, and finally, combine the sentiment analysis results to predict the propagation path and impact range of public opinion events;

[0016] Step 3: Generate a multi-dimensional guidance strategy for individual emotions and communication content, trace the individual emotion communication path, use the target emotion controlled generation algorithm, generate a multi-dimensional guidance strategy, achieve effective guidance of individual emotions and communication content, and enhance the group's ability to identify negative emotions and respond to public opinion;

[0017] Step 4: Build an emotion propagation prediction model and demonstrate and verify the large model system, establish a network public opinion event propagation prediction model based on spatiotemporal feature analysis, select typical public opinion cases for empirical analysis, verify the accuracy and efficiency of the prediction model, and achieve efficient prediction of the scope and effect of public opinion event propagation.

[0018] Collect and process multi-source social media individual expression data, combine 12345 historical and real-time data, use crawler technology and API interface to obtain representative and diverse individual expression data from major social media platforms. The goal is to obtain data covering social media and conversation content in different topics, fields and contexts, and provide data support for the generalization ability of sentiment analysis models for various emotional expressions. The data preprocessing stage will include text cleaning, word segmentation, entity recognition and denoising. Shallow noise removes punctuation, redundant spaces and HTML tags, and uses regular expression-based denoising:

[0019] T ′ =regex.sub(pattern,replacement,T)

[0020] Using T ′ =regex.sub('<.*?>',",T) can remove the crawled HTML tags, and regular expressions can also be used as dictionary mappings to replace noise;

[0021] For deep-level grammatical errors, spelling errors, irrelevant words, repeated words, semantic noise, and ambiguous noise, the BERT pre-trained language model is used to monitor and remove deep-level noise:

[0022] First, convert the input text into an embedded representation. For an input text, obtain its embedded representation through the BERT pre-trained language model:

[0023] F=BERT(T)

[0024] The classification layer is used to determine whether a word or sentence is noise. Since the specific type of noise is not important, in order to reduce the output categories, only a binary classification task of noise and non-noise is defined. For each embedded vector, its classification probability is calculated by the following formula:

[0025] p i =softmax(We i +b)

[0026] Among them, W is the weight matrix of the classification layer, b is the bias vector, softmax(·) is the activation function for binary classification, and the output p i Indicates the probability that the i-th word is noise:

[0027] T′={w i ∣p i ≤τ}

[0028] According to the probability of detected noise, a threshold τ is set. If p i ≤τ, the word or sentence is considered as noise and removed, otherwise it is retained, T ′ Represents the denoised text. The pre-processed text data is manually added with emotional tags and statistically classified into basic events. Based on these tags and classifications, the GPT generative language model generates preliminary answers. The generated answers are manually reviewed to determine whether they are consistent with the actual situation and context, and necessary adjustments are made. The opinions of psychological experts are further introduced to evaluate and adjust the answers generated by the model to ensure professionalism and accuracy. The answers to these individual expression data are obtained to provide the training data required for supervised learning for the large emotional analysis model.

[0029] Construct a generative multimodal and multiscale sentiment analysis model. Multimodality refers to text, image, voice, and video modes. Multiscale refers to pixel-level scale and regional-level scale. The open source Chinese language model is used as the basis of the generative sentiment analysis model. The function of the generative multimodal and multiscale sentiment analysis model is to generate text replies with emotional colors. At the same time, it supports the access of multimodal generative models of images, voices, and videos. For different types of data, multimodal feature fusion is performed. Multimodal fusion is performed from three aspects: feature level, attention level, and decision level. At the same time, a multiscale fusion method based on the attention mechanism is used to improve the contribution of each image modality feature. The construction of the generative multimodal and multiscale sentiment analysis model is based on training data. The supervised learning and reinforcement learning methods are used to train and fine-tune the model to reduce hallucinations and increase consistency. Specifically, keep the pre-trained model parameters unchanged, add an output layer with an output size of the number of emotion categories to the target model, and randomly initialize the model parameters of this layer. The output layer of the target model is trained from scratch on the target dataset, and the model output f is calculated.θ (x i ) and calculate the loss The gradient of the original model is obtained by back propagation, and the parameters of the remaining layers are fine-tuned based on the parameters of the source model. The key to fine-tuning is to let the model learn the specific context and expression in the field of sentiment analysis so that the model can adapt to different emotional expressions more flexibly, thereby improving the overall performance of sentiment analysis and generating emotional content that is more in line with the individual expressions on social media.

[0030] In order to cope with the problems of high computing power requirements and multi-card reasoning difficulties during multi-task reinforcement learning training, a sparse gated hybrid expert network is used for multi-task training. An improved DeepSpeed-MoE model training technology with multi-expert and multi-data parallelism, simplified routing, load balancing, gradient quantization and asynchronous training is proposed. The pre-trained parameter amount is maximized in a simple and computationally efficient way. At the same time, the expert model allocation strategy is dynamically adjusted according to the load and performance of each expert model to efficiently utilize computing resources. The load status of each expert model, including computing load and memory usage, is monitored in real time. The model is distributed in parallel on multiple computing nodes, and each node is responsible for a part of the model parameters, reducing the computing load and memory usage of a single node and improving the reasoning speed. Compared with the traditional MoE routing strategy, the gated network sends the input x of each token to the top-k expert models. In order to reduce communication and computing costs, a simplified strategy is adopted, and each input x is only sent to one expert model. In order to make the load of the expert model more balanced, an auxiliary loss is added:

[0031]

[0032] The number of experts is N, f i is the proportion of tokens allocated to expert i, P i is the sum of the probabilities of all tokens in a batch being assigned to expert i. The gradient quantization and compression formula is:

[0033]

[0034] in is the gradient vector, quantized scaling factor It is used to control the quantization accuracy, reduce the amount of gradient data transmitted over the network, and reduce the communication bandwidth requirements, where n is the quantization bit width; using an asynchronous training strategy, each computing node independently calculates and updates gradients, reduces synchronization overhead, and increases training speed. Each computing node independently calculates gradients and updates parameters, reducing synchronization waiting time. Each node calculates gradients and updates parameters at its own speed to adapt to different computing resources and data distributions. A two-stage fine-tuning strategy is used in the multi-task training phase to improve the training efficiency and performance of the DeepSpeed-MoE model:

[0035] In the first stage, a large-scale corpus is used to pre-train the sparse gated DeepSpeed-MoE network. Assuming there are E expert networks and an input sample x, the output of the router can be expressed as g(x) = (g1(x), g2(x), …, g E (x)), and use the softmax function to convert these scores into probabilities, After probability sorting and mask selection, a mask vector is created, whose element value is 1 for selected experts and 0 for unselected experts. The expert with the highest probability is selected for dynamic activation to achieve dynamic allocation of the expert network for effective information integration and learning. Sparsity constraints are used to limit the number of parameters in the gating network, thereby reducing the computational complexity and memory usage of the DeepSpeed-MoE model.

[0036] In the second stage, the data of the two target tasks, sentiment analysis and text generation, are used to fine-tune the entire DeepSpeed-MoE model. The layers close to the input layer are frozen, and only the later layers and the output layer are trained. This is because the earlier layers capture lower-level features, while the later layers and the output layer can recognize higher-level features and ultimately complete the task. The knowledge learned in the first stage is fine-tuned to be closer to the tasks of sentiment analysis and text generation. By fine-tuning on the target tasks, the entire DeepSpeed-MoE model is made to better fit the data distribution and characteristics of the specific task, thereby improving the performance and generalization ability of the DeepSpeed-MoE model.

[0037] The sentiment analysis library SnowNLP is used to perform sentiment polarity analysis, extract sentiment features, sentiment polarity and sentiment intensity. In addition, based on the interactive relationship between users, the NetworkX library is used to build a propagation network diagram. The formula for extracting the propagation path is:

[0038]

[0039] Among them, P(u,v) represents the probability of user u spreading information to user v, W(u,v) represents the interaction weight between user u and user v, and D(u) is the out-degree of user u. Based on graph theory, the classic SIR (Susceptible-Infected-Recovered) model is used to simulate the spread of public opinion in social networks. The specific model is defined as follows: S(t) is the proportion of infected people at time t, I(t) is the proportion of susceptible people at time t, and R(t) is the proportion of recovered people at time t. The dynamic equation of the model is:

[0040]

[0041]

[0042]

[0043] Among them, β is the propagation rate and γ is the recovery rate. In addition, using complex network theory, a social media public opinion propagation model is constructed. The specific method is to use the Barabasi-Albert (BA) model to generate a scale-free network, simulate the interactive relationship between social media users, use the PageRank algorithm to calculate the importance of user nodes, and identify key propagation nodes:

[0044]

[0045] Among them, PR(u) is the PageRank value of node u, d is the damping factor, N is the total number of nodes, M(u) is the set of all incoming link nodes of node u, L(v) is the out-degree of node v, and LSTM network is used for time series prediction. The input of LSTM is the emotional feature sequence, and the output is the emotional state of the next time step. The calculation formula of LSTM is as follows:

[0046] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0047] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0048]

[0049]

[0050] o t =σ(W o ·[ht-1 ,x t ]+b o )

[0051] h t =o t tanh(C t )

[0052] Among them, f t 、i t , C t , o t and h t They are forget gate, input gate, candidate cell state, cell state, output gate and hidden state. Finally, the random forest algorithm is used to classify and predict the propagation path. The calculation formula of random forest is:

[0053]

[0054] Among them, h(x) is the prediction result of random forest, N is the number of trees, and h i (x) is the prediction result of the ith tree. Combined with the sentiment analysis results, the trained LSTM and random forest models are used to predict the propagation path and impact range of the public opinion event, calculate the propagation probability and propagation speed of each node, and draw a propagation path diagram. Using the results of PageRank and LSTM models, key propagation nodes are identified, focusing on the emotional state and interactive behavior of these nodes, and predicting their impact on the overall propagation.

[0055] Construct an emotion-oriented content generation framework and method. Aiming at the existing generative language model's generation form and single content problems of emotion-oriented content for an individual's own emotional state, a guidance strategy and content generation model optimization scheme based on technologies such as generative adversarial networks and variational autoencoders is proposed to generate text, images, audio and other communication content that conforms to the target emotion. Memory-driven guidance effect evaluation, correction, follow-up and other key technologies are used. Using opposition state tracking technology, natural language processing and named entity recognition are combined to establish a dialogue state tracking module, evaluate the topic and viewpoint in the dialogue and complete the correction. A bidirectional long short-term memory network is used in combination with conditional random fields to construct a named entity recognition model. According to the dialogue history and user feedback, the dialogue state is updated, and key correction information is recorded and fed back to the generative model. The output of the bidirectional long short-term memory network is expressed as:

[0056]

[0057] in and denote the hidden states of the forward and backward LSTM units at time step t, respectively, and h tis the connection between the forward and backward hidden states. The conditional random field is used to capture the dependency between labels, especially the label transfer relationship in the sequence labeling task. The transfer matrix A is defined, where A i,j represents the score of transferring from label i to label j. Given an input sequence x and its corresponding label sequence y = (y1, y2, ..., y n ), calculate the score function:

[0058]

[0059] Where P represents the output score matrix of the bidirectional long short-term memory network, Indicates that the label is y at time step t t The scoring function is the core tool for evaluating the quality of generated content. It can solve the difficulties in guiding the communication direction of target emotions, the lack of emotional guidance evaluation and follow-up, and construct a framework for the fusion generation of multi-dimensional features of individual emotions. According to specific emotional targets, it can generate communication content and emotional guidance program strategies and specific contents that meet individual emotions.

[0060] Form a communication content guidance strategy, aiming at the problem of different processing methods such as emotional intensity, emotional vocabulary, expression, social network relationship, etc. of the individual source of emotion, combined with the characteristics of the target individual, study the model generation mechanism of the emotional guidance content elements based on social network analysis and communication model, break through the key nodes and communication path confirmation and evaluation verification methods in emotional communication, and the multi-dimensional guidance strategy generation algorithm model that conforms to the target emotion. Starting from the emoticon embedding coding technology and model word segmentation technology, realize the formation of a multi-dimensional guidance strategy for communication content that adapts to the individual emotional characteristics, and provide technical support for the generation of multi-dimensional guidance solutions that adapt to the individual emotional characteristics. Add emoticon embedding coding technology, the multi-dimensional guidance strategy generation algorithm model will use emoticons as special tags, encode them together with other text tags, and then embed emoticons in the generated text to enhance emotional expression. Emoticons are represented as embedding vectors. Let E be the emoticon set, T be the text tag set, V = E ∪ T be the expanded vocabulary, each emoticon e∈E and text tag t∈T has corresponding embedding vectors e and t, represented by a one-hot vector:

[0061] e onehot ∈{0,1} |V|

[0062] t onehot ∈{0,1} |V|

[0063] The embedding matrix is ​​a selection of expressions and texts. Each row corresponds to the vector representation of a tag for a specific word or emoticon. The embedding matrix is ​​used to get the embedding vector:

[0064] e=We ·e onehot

[0065] t=W t ·t onehot

[0066] in is the embedding matrix corresponding to the expression, is the embedding matrix corresponding to the text, and d is the dimension of the embedding vector;

[0067] The model word segmentation technology regards word segmentation as a state transition process in a hidden Markov model (HMM), which helps to identify and segment proper nouns and terms and improve the accuracy of text processing. In the HMM model, each word is regarded as a hidden state. The HMM model calculates the probability of each state through the observed character sequence and determines the word segmentation position through these probabilities. The HMM model consists of five parts: state set S, observation set V, initial state probability distribution π, state transition probability matrix A, and observation probability matrix B:

[0068] S={s1,s2,…,s N}

[0069] V={v1,v2,…,v M}

[0070] π={π i}

[0071] A={a ij}

[0072] B = {b ij}

[0073] Among them, s i represents a hidden state, such as the beginning (B), middle (M), end (E), and single word (S) states of emotional words, and v in the observation set i Represents an observation value (character), π in the initial state probability distribution i Represents the initial state s i The probability of the state transition probability matrix a ij Represents the probability of transitioning from state i to state j. The observation probability matrix b ij Indicates that in state s i Generate observations v i Probability:

[0074] Given an observation sequence O = (o1, o2, ..., o T), the HMM model determines the word segmentation position by calculating the probability of each state sequence. When the forward algorithm calculates the time step t, it takes state s i The probability of the partial sequence at the end is α t (i)

[0075] α t (i)=P(o1,o2,…,o t ,q t =s i )

[0076] The recursive formula is:

[0077]

[0078] where α t (i) At time step t, in state s i The probability of the partial sequence at the end, a ij State transition probability, from state s i Transfer to state s j The probability of b j (o t+1 ) in state s j Generate observations o t+1 The probability of , calculated at time step t, from state s i The probability of generating a partial sequence is β t (i)

[0079] β t (i) = P(o t+1 ,o t+2 ,…,o T ∣q t =s i )

[0080] The recursive formula is:

[0081]

[0082] Among them, β t (i) is at time step t, with state s i The probability of the starting partial sequence, a ij State transition probability, from state s i Transfer to state s j The probability of b j (o t+1 ) in state s j Generate observations o t+1 The probability of using the Viterbi algorithm to find the most likely state sequence, that is, the position of word segmentation:

[0083]

[0084] The recursive formula is:

[0085]

[0086] Establish an emotion propagation prediction and guidance system, design corresponding systems and software architectures, and develop an emotion propagation prediction and guidance system for the application and evaluation of large emotion analysis models in the analysis and cognitive patterns of individual and group emotions on the Internet, the representation of multimodal emotion evolution, and the controlled generation of multi-dimensional guidance of individual emotions and communication content;

[0087] Establish a public opinion event propagation status and trend analysis and prediction model, aim at the spatiotemporal propagation evaluation and prediction needs of various public opinion events suddenly generated in cyberspace, build a public opinion event propagation scope and guidance effect prediction model, and establish a public opinion event propagation evaluation method based on BP neural network and multi-layer social network research:

[0088] The indicator system framework is constructed from the three perspectives of public opinion source, public opinion propagation, and public opinion audience. The BP neural network and graph convolutional network are used to perform multi-factor input modeling and multi-layer social network modeling of the three indicators of public opinion source, public opinion propagation, and public opinion audience. The best public opinion event propagation prediction model is trained. 16 third-level indicators affect the public opinion event propagation evaluation index. Therefore, the number of input layer neurons in the BP neural network model is 16. Only the public opinion event propagation evaluation index needs to be obtained, so the number of output layer neurons is 1. There is a direct correlation between the number of hidden layers of the BP neural network and its learning efficiency. Theoretically, a BP neural network with at least 3 layers can approximate any Continuous function, that is, using a BP network with only one hidden layer can relatively simply realize the required function and model almost all nonlinear systems. With the increase of the number of hidden layers, the network error gradually decreases, but the network structure becomes more complex, which is easy to cause longer network training time or overfitting. Therefore, a 3-layer BP neural network is used, and the number of neurons in the hidden layer is 6. If the activation function is not selected, each layer of the neural network is just a linear transformation of the previous layer. After multiple layers are superimposed, the final output is still a linear transformation of the original input. Therefore, a nonlinear factor is introduced and the Sigmod function is used as the activation function, as shown in the following formula:

[0089]

[0090] The modeling of multi-layer social networks involves processing network data at multiple relationship levels. These data contain multiple types of nodes and edges. Nodes represent users and posts, and edges represent friend-like and follow-up relationships. Combined with BP neural networks, complex multi-factor input modeling can be achieved. Multi-layer social network modeling: GNN graph neural network is used to encode multi-layer social networks, and node and edge features in each layer of the network are extracted. Assuming there are L layers of social networks, the node feature matrix of each layer of the network is H (l) , the edge feature matrix is ​​A (l) , where l = 1, 2, ..., L, the GCN graph convolutional network processes each layer of the social network and generates node embeddings, H (l+1) =σ(A (l) H (l) W (l) ), where σ is the activation function, W (l) is the weight matrix of the lth layer, which combines the original input feature x with the nodes H generated by each layer of the social network (l) Splice to form a comprehensive feature vector z = [x; H (1) ;H (2) ;…;H (L) ], combining the multi-layer social network with the BP network, inputting the comprehensive feature vector z into the BP neural network for training, and adjusting the weight matrix W by the back propagation algorithm to minimize the loss function, and finally obtaining the prediction model Based on the combination of BP neural network and multi-layer social network, the propagation characteristics of public opinion events in space and time dimensions are extracted, and then the propagation status and evolution trend of public opinion events are analyzed, which provides strong technical support for improving the performance and accuracy of public opinion event prediction models, and the management and guidance of public opinion events;

[0091] Demonstrate and verify personal analysis and prediction and guidance of public opinion events. Aiming at the needs of public opinion guidance for typical hot events, carry out demonstration and verification of individual and group emotion propagation status analysis and trend prediction and guidance. Through the corresponding emotion propagation prediction and guidance large models and systems in previous research contents, analyze and predict and guide historical and current hot public opinion events, verify and confirm the feasibility of the path of emotion propagation of public opinion events and the effectiveness of the guidance methods, realize the iterative optimization of public opinion event management and guidance, and evaluate the performance and application effect of the research results on the analysis and guidance of public opinion event propagation.

[0092] A computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 4 are implemented.

[0093] The beneficial effects of the present invention are at least:

[0094] 1. Comprehensive sentiment analysis: On social media, people’s emotional expressions are often complex and varied, and may even contain multiple emotions at the same time. By building a large sentiment analysis model based on grammar, semantics, and memory drive, we can deeply understand these complex emotions and thus more accurately capture the emotional state of individuals. Traditional sentiment analysis methods often only focus on the surface information of the text, while ignoring the deep meaning and contextual relationships behind the text. The sentiment analysis large model of the present invention can comprehensively consider the grammatical structure, semantic information, and memory factors of the text, thereby greatly improving the accuracy of sentiment analysis.

[0095] 2. Efficient emotional guidance: Generate controlled products for multi-dimensional guidance of individual emotions and communication content, which can provide stable and effective emotional guidance strategies and specific guidance content, improve the consistency and traceability of emotional guidance, and help to evaluate and follow up the subsequent guidance effect. By generating controlled products for multi-dimensional guidance of individual emotions and communication content, the present invention can provide stable and effective emotional guidance strategies. These strategies not only take into account the individual emotional state of the target audience, but also combine the characteristics of the communication content and the characteristics of the communication channel, so as to ensure the stability and consistency of the guidance effect. The emotional guidance method of the present invention has a high degree of traceability. During the guidance process, we can clearly track and record the guidance content and effect of each link, so as to facilitate subsequent effect evaluation and follow-up. This traceability helps to continuously optimize the guidance strategy and improve the guidance effect. Through efficient emotional guidance, information can have a more positive positive effect in the process of communication. This helps to improve the public's cognitive level, enhance social cohesion, and promote the harmonious and stable development of society.

[0096] 3. Improve the ability to respond to public opinion: Through comprehensive sentiment analysis and efficient emotion guidance, the present invention significantly improves the ability to respond to public opinion. Specifically reflected in: Rapid identification and response: Comprehensive sentiment analysis enables public opinion managers to quickly identify emotional changes and potential crises in the network and take timely response measures. Precise intervention and guidance: Efficient emotion guidance strategies can quickly stabilize the emotions of affected individuals when public opinion events occur, reduce the spread of negative emotions, promote the spread of positive emotions, and alleviate public opinion crises. Systematic public opinion management: Based on sentiment analysis and guidance models, a systematic public opinion response mechanism can be constructed to improve the scientificity and effectiveness of public opinion response.

[0097] 4. Optimized resource utilization: By adopting the sparse gated DeepSpeed-MoE training technology, the problem of large computing resources required for large model training is solved, the utilization of computing resources is optimized, and the efficiency and effect of DeepSpeed-MoE model training are improved. In large model training, computing resources are usually a huge expense. By adopting the sparse gated DeepSpeed-MoE training technology, the present invention can significantly reduce the consumption of computing resources, thereby reducing the training cost and making large model training more economical and efficient. The sparse gated DeepSpeed-MoE training technology can intelligently select and process key data and reduce redundant calculations. This can not only speed up the training speed, but also improve the resource utilization rate during the training process, making large model training more efficient. By optimizing the utilization of computing resources, the present invention can enable large models to better learn and adapt to data during the training process, thereby improving the performance of the model. This optimization is not only reflected in the accuracy of the model, but also in the robustness and generalization ability of the model.

[0098] 5. Wide application: In the face of diverse sudden public opinion events and the need to predict and verify the spread of public opinion events in a big data environment, the present invention has a wide range of application scenarios and applicability. Whether it is a public opinion event in the political, economic, social or cultural fields, the present invention can provide effective sentiment analysis and guidance strategies. By providing sentiment analysis and guidance strategies for different types of public opinion events, the present invention can help decision makers better understand public sentiment and grasp the direction of public opinion. This will help to improve the scientificity and effectiveness of decision-making and reduce decision-making risks. In sudden public opinion events, public sentiment is often prone to fluctuations and may even cause social unrest. Through the sentiment analysis and guidance strategies provided by the present invention, we can stabilize public sentiment in a timely and effective manner, avoid the spread and escalation of negative emotions, and thus enhance social stability and harmony.

[0099] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0100] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0102] Figure 1 Schematic diagram of the steps of the generative AI-based emotion propagation prediction and guidance large model and system in the present invention.

[0103] Figure 2 It is a schematic diagram of the technical process of collecting and processing multi-source social media individual expression data in the present invention.

[0104] Figure 3 Schematic diagram of the technical process of building a generative multimodal and multi-scale sentiment analysis large model in the present invention.

[0105] Figure 4 This is a schematic diagram of the sparse mixed expert model training in the present invention.

[0106] Figure 5 This is a schematic diagram of the fusion generation of multi-dimensional features of individual emotions in the present invention.

[0107] Figure 6 It is a schematic diagram of the process flow of the controlled generation mechanism of the multi-dimensional guidance strategy content in the present invention.

[0108] Figure 7 This is a diagram of the system layered architecture in the present invention.

[0109] Figure 8 This is a structural diagram of the multi-layer social network communication model in the present invention.

[0110] Fig. 9 This is a schematic diagram of the large model verification system for demonstrating emotion propagation prediction and guidance in the present invention. DETAILED DESCRIPTION

[0111] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0112] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0113] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0114] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0115] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0116] In order to understand the mechanism of emotion generation and evolution, improve the accuracy of online public opinion analysis and prediction, reasonably guide individual emotions in social media, and thus improve the social governance system, realize the strategic needs of national cyberspace security and social governance modernization, and solve the limitations of existing emotion analysis methods, especially the inadequate processing of complex and multimodal emotion expressions on social media and the inaccurate prediction and guidance of dynamic changes in emotions, the present invention provides a large model and system for emotion propagation prediction and guidance based on generative AI, such as Figure 1 As shown, the method includes the following steps S101 to S108:

[0117] Step S101: Multi-source social media individual expression data collection and processing. Combined with the 12345 dialogue text database, a large amount of network public opinion data is obtained through multi-source social media individual expression data collection, and text preprocessing is performed, and further artificial processing is used to complete the construction of the sentiment analysis data set.

[0118] Step S102: construct a generative multimodal and multiscale sentiment analysis model, with the open source Chinese language model as the base, which becomes the foundation of the generative sentiment analysis model. The function of the model is to generate text replies with emotional colors, and support the access of multimodal generative models of text, images, voice, and video. The model is constructed based on the training data, using supervised learning and reinforcement learning methods to train and fine-tune the model to reduce hallucinations and increase consistency. Specifically, keep the pre-trained model parameters unchanged, add an output layer with an output size of the number of emotion categories to the target model, and randomly initialize the model parameters of the layer. Train the target model on the target data set, train the output layer from scratch, and calculate the model output f θ (x i ) and calculate the loss The gradient of the source model is obtained by back propagation, and the parameters of the remaining layers are fine-tuned based on the parameters of the source model. The key to fine-tuning is to let the model learn the specific context and expression in the field of sentiment analysis so that the model can adapt to different emotional expressions more flexibly, thereby improving the overall performance of sentiment analysis. In terms of emotion propagation and public opinion evolution modeling, a graph model is used to simulate the propagation process of emotions in the network, and combined with traditional random walk and propagation algorithms, the path and trend of emotion diffusion are predicted. At the same time, the autoregressive integrated moving average model (ARIMA) is used to predict the time series of public opinion data to capture the trend of emotion propagation and changes in public opinion. In terms of multimodal feature fusion, weighted fusion and multi-layer perceptron (MLP) methods are used to fuse and process the features of various modalities such as text, image, and audio to capture the complex nonlinear relationship between features. In the study of the association between individual emotions and group emotions, social network analysis methods are used to study the relationship and influence between individuals. The status and influence of individuals in the network are determined by analyzing the node centrality index. The machine learning algorithm decision tree is applied to study the impact of individual characteristics on group emotions. The support vector machine method is used to establish a mathematical relationship between individual emotions and group emotions, and the degree of influence and changing trend of different characteristics on emotions are analyzed.

[0119] Step S103: Multi-task training using sparse gated hybrid experts. Based on the business model and subtasks supported by the large model, the improved DeepSpeed-MoE with multi-expert and multi-data parallelism, simplified routing, load balancing, gradient quantization and asynchronous training is used for multi-task training to obtain a basic large model for sentiment analysis that supports specific tasks and businesses. First, the input data is divided into multiple regions according to the task type and assigned to the corresponding GRU, and the input data is dynamically allocated by the gating unit. In order to reduce the amount of calculation and memory usage, the parameters of the gating unit are constrained by sparsity, which is achieved through L1 regularization and sparsity penalty. GRU uses a gating mechanism to control the flow of information and capture long-term and short-term dependencies in sequence data. In the multi-task fine-tuning process, subtasks such as sentiment analysis and text generation are combined, and the performance of the model between multiple subtasks is optimized by designing a joint loss function and using a multi-task learning framework. Specifically, the sentiment analysis task uses the cross entropy loss function, while the text generation task uses the negative log-likelihood loss function. During the fine-tuning process, a combination of supervised learning and reinforcement learning is used to gradually adjust the model parameters to make them more suitable for the data distribution and characteristics of specific tasks.

[0120] Step S104: Carry out prediction of the spread of public opinion events based on the sentiment analysis model. Use graph theory and complex network theory to build an information propagation model, determine the nodes, edges and propagation strength of information propagation, and thus simulate the process of information propagation in social networks. Use long short-term memory network (LSTM) and random forest algorithm to train and optimize the propagation model to improve prediction accuracy. Combined with the results of sentiment analysis, predict the propagation path and scope of influence of public opinion events, identify potential key propagation nodes and information influence, and support the management and intervention of public opinion events.

[0121] Step S105: Individual emotion multi-dimensional feature fusion generation framework. The requirements for multi-dimensional representation of individual emotion differences are sorted out. Emotional effects are affected by multiple features. Based on the guidance strategy and content generation model optimization scheme of generative adversarial networks, variational autoencoders and other technologies, text, images, audio and other communication content that conforms to the target emotion is generated. Memory-driven guidance effect evaluation, correction, follow-up and other key technologies are used. The dialogue state tracking module is established by combining natural language processing (NLP) and named entity recognition (NER) using opposition state tracking technology, evaluating the topics and opinions in the dialogue and completing the correction. The NER model is constructed by combining the bidirectional long short-term memory network (BiLSTM) with the conditional random field (CRF). According to the dialogue history and user feedback, the dialogue state is updated, and the key correction information is recorded and fed back to the generative model.

[0122] Step S106: Construct a controlled generation mechanism for multi-dimensional guidance strategy content. Use the multi-modal generation technology of emotion-oriented content based on emoticon embedding coding and model word segmentation to conduct multi-dimensional design of individual emotion characteristics, break through the memory-driven mechanism, and construct a multi-dimensional guidance content generation evaluation and verification mechanism with strong controllability and high reliability of results, solve the problem of lack of pertinence and interactivity of emotion guidance in complex network environments, and ensure the efficiency and stability of network public opinion guidance.

[0123] Step S107: Emotion propagation prediction and guidance system. The system consists of a data processing layer, a model integration layer, a demonstration and verification layer, and computing resources. Demonstration and verification of the emotion propagation prediction and guidance model is performed.

[0124] Step S108: Prediction model of the spread scope and guiding effect of public opinion events. Establishing a network public opinion detection index system from the three levels of network public opinion sources, public opinion spread, and public opinion audiences provides theoretical support for comprehensively evaluating the spread of public opinion events on the network, and establishing a BP neural network structure for public opinion event spread evaluation to improve the accuracy and precision of the comprehensive effect evaluation of public opinion event spread. The BP neural network structure can process multi-dimensional data, model complex nonlinear relationships, and realize real-time dynamic learning and adaptive adjustment, so as to better analyze the spread of public opinion events and provide reliable decision support and guidance strategies for public opinion control.

[0125] Step S109: Demonstration and verification system for emotion propagation status analysis and prediction guidance of typical public opinion events. From data collection and processing, emotion analysis model construction, emotion propagation path analysis to emotion trend prediction guidance, system verification and optimization, to real-time data monitoring and updating, visualization and report generation, as well as user feedback and system optimization, each step will provide key support and optimization for the establishment of the system. This complete technical route will ensure that the built system plays the greatest role in public opinion event management and provide users with comprehensive and accurate emotion propagation analysis and prediction guidance services.

[0126] Steps S101 to S103 are specific steps for constructing a large sentiment analysis model based on grammatical semantics and memory drive.

[0127] like Figure 2 In step S101, the multi-source social media individual expression data collection and processing are realized. First, the existing conversation data of the 12345 government service hotline is processed and textualized to obtain the corresponding conversation time and content of the conversation. Under the premise of complying with relevant laws and regulations, the intelligent crawler program and the data collection API provided by social media (such as Weibo, Baidu Tieba, Douyin, Xiaohongshu, Zhihu, etc.) are used to realize the targeted capture of public conversation content for different public opinion events on social media platforms. The collected social media conversation text needs to be preprocessed, including text cleaning, standardization, deduplication, desensitization, formatting, and denoising steps to ensure the accuracy of subsequent sentiment analysis and time series analysis.

[0128] The noise in the text can be divided into shallow and deep levels. Shallow noise includes extra symbols, HTML tags, punctuation marks, and other irrelevant characters. To remove punctuation marks, extra spaces, and HTML tags, use regular expression-based noise reduction:

[0129] T ′ =regex.sub(pattern,replacement,T)

[0130] For example, T ′ =regex.sub('<.*?>',",T) can remove the crawled HTML tags. Regular expressions can also be used as dictionary mappings to replace noise.

[0131] For deep-level noise, such as grammatical errors, spelling errors, stop words and irrelevant words such as advertising words, repeated words, semantic noise, and ambiguous noise, the BERT pre-trained language model is used to monitor and remove deep-level noise.

[0132] First, convert the input text into an embedded representation. For an input text, obtain its embedded representation through the BERT model:

[0133] F=BERT(T)

[0134] The classification layer is used to determine whether a word or sentence is noise. Since we do not care about the specific type of noise, in order to reduce the output categories, we only define a binary classification task (noise vs. non-noise). For each embedded vector, its classification probability is calculated using the following formula:

[0135] p i =softmax(We i +b)

[0136] Among them, W is the weight matrix of the classification layer, b is the bias vector, softmax(·) is the activation function for binary classification, and the output p i represents the probability that the i-th word is noise.

[0137] T′={w i ∣p i ≤τ}

[0138] According to the probability of detected noise, a threshold τ is set. If p i ≤τ, the word or sentence is considered as noise and removed, otherwise it is retained, and T′ represents the denoised text.

[0139] The cleaned conversation information is accurately sorted according to the timestamp. By establishing a time series data set, the evolution trend of user emotions over time can be captured, providing strong support for in-depth mining of the laws of emotional changes during the conversation process. Based on the collected data, the content classification and emotional categories obtained through statistical analysis are used as prompt words, and a large language model is used to generate a rough data set. The automatically generated training data is then screened and checked to select high-quality data as the training set.

[0140] like Figure 3 In step S102, a multimodal and multiscale generative sentiment analysis model is constructed. Combining multiple data sources such as text, image, and sound, a multimodal sentiment analysis model that can comprehensively analyze multiple information is constructed. The BERT model is used to extract the above processed text, and CNN is used to extract image and sound features.

[0141] H text =BERT(T ′ )

[0142] H video =CNN(X video )

[0143] H audio =CNN(X audio )

[0144] Feature-level fusion is performed at different stages, including attention in the training stage and decision-level fusion in the output stage. Feature-level fusion concatenates features of different modalities in the feature space.

[0145] H fusion =Concat(H text ,H video ,H audio )

[0146] On the basis of feature-level fusion, the attention mechanism is introduced to further focus on important features, and softmax is used to calculate the attention weights and weighted sum the weights:

[0147] α audio =softmax(W audio H fusion )

[0148] α image =softmax(W image H fusion )

[0149] α text =softmax(W text H fusion )

[0150] H attention =α audio H audio +α image H image +α text H text

[0151] Based on the feature level and attention fusion, the decision results of each modality are fused:

[0152] y audio =MLP(H audio )

[0153] y image =MLP(H image )

[0154] y text =MLP(H text )

[0155] y final =softmax(w audio y audio +w image y image +w text y text )

[0156] By effectively combining three multimodal fusion techniques, namely feature-level fusion, attention fusion and decision-level fusion, a more robust and accurate multimodal sentiment analysis model is constructed. This multi-level fusion method can fully utilize the advantages of each modality data and improve the overall performance of sentiment analysis.

[0157] Compared with audio data and text data, visual data contains rich and complex emotional information, including facial expressions and body movements. For example, wide eyes may indicate surprise, while slightly narrowed eyes may indicate suspicion or anger. The upturn or droop of the corners of the mouth and the degree of opening can convey emotions such as happiness, sadness, and surprise. Hunchback may indicate fatigue or frustration, while holding the head high may indicate confidence and positivity. Gestures and movements (such as clenching fists, pointing, waving, etc.) can also convey different emotions and intentions. This information is distributed at different scales, so it is necessary to obtain multi-scale information from visual data to achieve comprehensive judgment of emotions.

[0158] Using the multi-scale emotion feature fusion module, through multi-scale convolution fusion and multi-scale attention fusion, these complex information can be effectively captured. Small-scale convolution kernels (such as 3x3) can capture detail information features; medium-scale convolution kernels (such as 5x5) can capture textures and local patterns; large-scale convolution kernels (such as 16x16 and above) can capture global patterns and scene information. Multi-scale attention fusion aims to achieve selective learning between two different branches. The main idea is to let the network learn feature weights according to the loss, allowing the model to selectively fuse information from different scales. Multi-scale attention fusion is mainly divided into two parts: multi-scale attention module and multi-scale attention fusion module.

[0159] Multi-scale attention includes regional attention and pixel-level attention. Regional attention measures the importance of regions of different sizes in the feature map. Specifically, average pooling of different scales is used for the fused features, for example, one block is 1×1, 4 blocks are 2×2, and 16 blocks are 4×4. Channels are compressed and expanded through the convolution layer. In order to be compatible with pixel attention, it is restored to the size before pooling. Taking 4×4 as an example, the feature map is first averagely pooled. Fuse ∈R C×H×W We get fc4∈R C×4×4 , and then perform channel compression and expansion to obtain After restoration,

[0160]

[0161] Pixel attention measures the importance of each pixel. This module does not require pooling and reshaping, and directly performs channel compression and expansion to obtain The importance of each pixel can be measured at a microscopic scale.

[0162]

[0163] Multi-scale attention fusion obtains the weights of different positions in the feature map by superimposing pixel attention and regional attention:

[0164]

[0165] In the training and fine-tuning phase, supervised fine-tuning (SFT) is first performed on the pre-trained model of sentiment analysis, that is, instruction fine-tuning. The supervised fine-tuning phase aims to adapt the pre-trained model to the downstream task of sentiment analysis. Specifically, keep the parameters of the pre-trained model unchanged, add an output layer with an output size equal to the number of sentiment categories to the target model, and randomly initialize the model parameters of this layer. Train the target model on the target dataset, train the output layer from scratch, and calculate the model output f θ (x i ) and calculate the loss The gradient of the model is obtained by back propagation, and the parameters of the remaining layers are fine-tuned based on the parameters of the source model. This process uses the sentiment analysis dataset to adjust the model parameters to minimize the loss through forward propagation and back propagation. In this stage, the model fine-tunes the pre-trained language model through the prompt tuning method based on the query-answer pairs of the sentiment analysis dataset to cope with queries of various basic events and sentiment categories for sentiment analysis. The fine-tuning process includes forward propagation and back propagation on the training set, and adjusting the model parameters to minimize the loss on the validation set. While fine-tuning, different hyperparameters such as learning rate and batch size are used to adjust for better performance.

[0166] Pre-trained model f θ It is an open source large language model that has been pre-trained on a large scale. where x i is the input text, y i is the corresponding emotion label or emotional response. The fine-tuning goal is to minimize the cross entropy loss function:

[0167]

[0168] The cross entropy loss function is defined as:

[0169]

[0170] Among them, y i,k is the sentiment label of sample i in the kth category (usually one-hot encoded).θ (c i ) k is the model's predicted probability for the kth emotion. Use batch gradient descent to fine-tune the model parameters and minimize the cross entropy loss function:

[0171]

[0172] Where θ is the parameter vector to be optimized. α is the learning rate, which controls the step size of the parameter update. is the objective function Gradient with respect to the parameters θ. The learning rate and batch size are manually tuned hyperparameters.

[0173] The fine-tuning process includes calculating the model output f θ (c i ) and calculate the loss The key to fine-tuning is to let the model learn the specific context and expression in the field of sentiment analysis to improve the generalization ability of the model for specific sentiment analysis tasks. The best fine-tuning model can be selected by monitoring the performance on the validation set to ensure good performance on the sentiment analysis test set and finally obtain the SFT model.

[0174] Secondly, the language model is further fine-tuned through direct human feedback. There are two main steps: reward model fine-tuning and RLHF training. Reward model fine-tuning first requires reward modeling (RM). The goal of this step is to build a sentiment analysis text quality comparison model. For the same prompt word, the quality of multiple different output results given by the SFT model is ranked. By sampling the sentiment analysis dataset and generating multiple answers to the same query according to the dataset construction process, that is, query-multiple answer pairs, and manually scoring each answer, a scoring dataset is constructed. The goal of reward model fine-tuning is to evaluate the quality of the generated sentiment analysis text by establishing a reward model. This process involves multiple steps, including reward modeling (RM) and proximal policy optimization clip (PPO-Clip) based on the reward model. These steps and related formulas will be introduced in detail below.

[0175] The goal of reward modeling is to train a reward model R φ To evaluate the quality of generated texts. The model compares different generated texts and scores them based on human feedback. The training goal of the reward model is to make the predicted score close to the human score, which is formulated as:

[0176]

[0177] Use mean squared error as the loss function:

[0178]

[0179] Where R φ (x i ,y i ) is the predicted score of the reward model. s i It is the standard score given by humans.

[0180] Based on this dataset, an independent reward model is trained for subsequent RLHF training. During the RLHF training phase, the improved PPO-Clip algorithm is used. The traditional update strategy needs to ensure that the strategy change is within a trust region to prevent instability caused by excessive strategy updates. PPO-Clip approximates the effect of TRPO by introducing a simple and efficient clipping mechanism, and further fine-tunes the SFT model based on the reward feedback of the reward model. PPO-Clip is suitable for fine-tuning the generative model π θ Enable it to generate high-quality emotional text. Based on the reward model R φ The score of PPO-Clip is optimized by maximizing the expected reward. θ (y|x), θ is a parameter. The optimization goal of the proximal strategy is:

[0181]

[0182] Where r = R φ (x,y). To prevent the policy from updating too quickly and causing instability, a trust region constraint is used to limit the range of the policy update:

[0183]

[0184] in represents the probability distribution of the old strategy. clip(·,1-∈,1+∈) is the clipping function used to limit the strategy update amplitude, and the hyperparameter ∈ controls the clipping range. The reward model fine-tuning is: Initialize the strategy π θ and the reward model R φ ; Sample generated text, using the current strategy π θ Generate text y~π θ (x); Calculate the reward using the reward model R φ Score the generated text y and get the reward value r = R φ (x,y); Update strategy, optimize target L according to PPO-Clip CLIP (θ) Update the parameters θ of the generative model.

[0185] By using this reinforcement learning method based on human feedback, the model can be more flexible to adapt to different ways of expressing emotions, thereby improving the overall performance of sentiment analysis. In addition, this personalized fine-tuning can enable the model to capture emotional information more accurately and improve its performance in real application scenarios. Through this series of steps, a more powerful and intelligent generative sentiment analysis model is built to better cope with complex sentiment analysis and text generation tasks.

[0186] In terms of emotion propagation and public opinion evolution modeling, we use graph models to simulate the propagation process of emotions in the network, and combine traditional random walk and propagation algorithms to predict the path and trend of emotion diffusion. At the same time, the autoregressive integrated moving average model (ARIMA) is used to predict the time series of public opinion data to capture the trend of emotion propagation and changes in public opinion. The graph model is represented by G = (V, E), where V is a node set representing users, and E is an edge set representing the relationship between users. Each node v has an emotional state S v , the propagation can be regarded as the propagation of node status. Assume that the probability of node u infecting node v is p uv , during the propagation process:

[0187]

[0188] Among them, N(v) represents the set of neighbor nodes of node v. In the modeling of sentiment propagation and public opinion evolution, random walks can help us understand how sentiment spreads in social networks. In a simple random walk, the current node v selects a neighbor node u for the next move according to uniform probability. The random walk process can be expressed by the following formula:

[0189]

[0190] p t (v) represents the probability of emotional contagion at node v at time t, and d(u) represents the degree of node u.

[0191] Applying the Autoregressive Integrated Moving Average (ARIMA) model to predict the time series of public opinion data is an important method to capture the trend of sentiment propagation and changes in public opinion. This method uses the historical values ​​of the time series to predict the corresponding values ​​in the future, and is a typical method for time series prediction. The ARIMA model combines autoregression (AR), difference (I) and moving average (MA) to effectively analyze and predict the dynamic changes of time series data. The ARIMA model consists of three parts:

[0192] The autoregressive part predicts future values ​​through the past values ​​of the time series itself; the difference part stabilizes the time series by performing difference processing; the moving average part corrects the prediction by using the past prediction errors. The above three parts correspond to the three parameters p, d and q in ARIMA (p, d, q), where p is the order of the autoregressive term, d is the number of differences, and q is the number of moving average terms. Time series y t Represents the public opinion data value at a certain time point t. It can be one of the quantitative indicators of multiple emotional propagation and public opinion changes. Specifically, public opinion data includes the following business indicators: Sentiment Score, which represents the average or weighted value of the overall emotion at a certain time point. Sentiment score is usually calculated by sentiment analysis model, reflecting the positive and negative tendency of public emotions; Emotion Intensity, which represents the intensity or frequency of a certain emotion (such as anger, joy, sadness, etc.), can be obtained by classifying text data through sentiment classifier; Topic Popularity, which represents the discussion popularity or attention of a specific topic at a certain time point. It is usually quantified by indicators such as keyword frequency and topic tags; Information Diffusion Rate, which represents the speed or range of information propagation in social networks. It can be quantified by social interaction data such as the number of reposts, the number of comments, and the number of likes. Public Engagement, which represents the public's participation in an event or topic, is usually measured by indicators such as the number of user-generated content and the frequency of interaction.

[0193] Assume the current value y t It can be obtained by linear regression of the values ​​of the first p time steps, and the autoregressive part is obtained:

[0194] y t =φ1y t-1 +φ2y t-2 +…+φ p y t-p +∈ t

[0195] where φ i is the autoregressive coefficient. ∈ t It's white noise.

[0196] The difference part makes the time series stationary by taking d differences. The formula is:

[0197] y t ′ =(1-B) d y t

[0198] Where B is the backshift operator, defined as y t ′ is the time series after difference.

[0199] Assume the current value y t It can be obtained by linear regression of the error terms of the current and previous q time steps, and the moving average formula is:

[0200] y t =∈ t +θ1∈ t-1 +θ2∈ t-2 +…+θ q ∈ t-q

[0201] where θ i is the autoregressive coefficient. ∈ t is white noise. Combining the above three parts, the ARIMA (p, d, q) model is modeled as:

[0202] (1-φ1B-φ2B 2 -…-φ p B p )(1-B) d y t =(1+θ1B+θ2B 2 +…+θ q B q )∈ t

[0203] The ARIMA model is used to capture the trend of sentiment transmission and changes in public opinion. First, data preprocessing is performed to perform differential processing on the time series data to make it stable. The order model fitting of the autoregressive term, differential term, and moving average term is determined through the autocorrelation function and partial autocorrelation function graphs, and the ARIMA model is fitted according to the identified model structure to estimate the parameters φ and θ. The fitting goal is to minimize the mean square error of the residual sequence.

[0204] like Figure 4In step S103, sparse gated hybrid expert training is used to complete the construction of a large model for sentiment analysis of individual expressions on social media. The improved DeepSpeed-MoE model training technology with multi-expert and multi-data parallelism, simplified routing, load balancing, gradient quantization and asynchronous training is used to maximize the amount of pre-trained parameters in a simple and computationally efficient manner. At the same time, the allocation strategy of the expert model is dynamically adjusted according to the load and performance of each expert model to efficiently utilize computing resources, and the load status of each expert model, including computing load and memory usage, is monitored in real time. The model is distributed in parallel on multiple computing nodes, and each node is responsible for a part of the model's parameters, reducing the computing load and memory usage of a single node, and improving the inference speed. Compared with the traditional MoE routing strategy, the gated network sends the input x of each token to the top-k expert models. In order to reduce communication and computing costs, a simplified strategy is adopted, and each input x is only sent to one expert model. In order to make the load of the expert model more balanced, an auxiliary loss is added. To achieve this goal, the number of experts is N. i is the proportion of tokens allocated to expert i, P i is the sum of the probabilities of all tokens in a batch being assigned to expert i. The gradient quantization and compression formula is:

[0205]

[0206] in is the gradient vector, quantized scaling factor It is used to control the quantization accuracy, reduce the amount of gradient data transmitted over the network, and reduce the communication bandwidth requirements, where n is the quantization bit width; using an asynchronous training strategy, each computing node independently calculates and updates gradients, reduces synchronization overhead, and increases training speed. Each computing node independently calculates gradients and updates parameters, reducing synchronization waiting time. Each node calculates gradients and updates parameters at its own speed to adapt to different computing resources and data distributions. A two-stage fine-tuning strategy is used in the multi-task training phase to improve the training efficiency and performance of the model:

[0207] In the first stage, a large-scale corpus is used to pre-train the sparse gating network, which is responsible for dynamically allocating input data to different expert networks to achieve effective information integration and learning, and uses sparsity constraints to limit the number of parameters in the gating network, thereby reducing the computational complexity and memory usage of the model.

[0208] G(x)=softmax(W g ·x)

[0209] Among them, G(x) is the output of the gating network, which represents the probability of selecting each expert, and W gis the weight matrix of the gating network, and x is the input token. The gating model divides the input data into multiple regions according to the task type, and assigns one or more expert models to the data in each region. Each expert model focuses on processing this part of the input data, thereby improving the overall performance of the model. The gating unit is used to dynamically allocate input data to different expert networks to achieve information integration and learning. The expert network is GRU. In order to reduce the amount of computation and memory usage, the parameters of the gating unit are sparsely constrained. This is achieved through L1 regularization, sparsity penalty or the design of a gating mechanism. Limit the number of parameters in the gating unit so that only some neurons are activated in actual operation.

[0210] The L1 regularization loss function is:

[0211]

[0212] Loss data is the original loss function, λ is the regularization strength parameter, which controls the degree of sparsity penalty, and ω i is the parameter of the model. n is the number of parameters. In the study of network individual and group emotion analysis and cognitive patterns, the model needs to process a large amount of emotion data and perform calculations in complex neural networks. The sparsity constraints on the parameters can be used to enhance the performance of the model by reducing the parameter range and controlling the model size through L1 regularization, and also improve the generalization ability of the model. The number of parameters in the gated unit makes it activate only some neurons in actual operation.

[0213] The Gated Recurrent Unit (GRU) uses a gating mechanism to control the flow of information, allowing the network to effectively capture long-term and short-term dependencies in sequence data. The update gate of the GRU is:

[0214] z t =σ(W z ·[h t-1 ,x t ]+b z )

[0215] The update gate determines how much of the hidden emotional state of the current time step comes from the hidden emotional state of the previous time step and how much comes from the current emotional state input. Its function is similar to the combination of the forget gate and input gate in LSTM. t represents the output of the update gate, which determines the hidden emotional state h at the previous time step t-1 and candidate hidden emotional states σ represents the Sigmoid activation function, which maps the output value to (0,1) to reflect the probability of emotional state. z represents the weight matrix of the update gate. [ht-1 ,x t ] represents the hidden emotional state h of the previous time step t-1 and the current input x t splicing. z Represents the bias vector for the update gate.

[0216] The reset gate controls the influence of the hidden emotional state of the previous time step on the current candidate hidden emotional state. When the reset gate is close to 0, the network discards the hidden emotional state information of the previous time step; when the reset gate is close to 1, the hidden emotional state information of the previous time step is retained. The forget gate of GRU is:

[0217] r t =σ(W r ·[h t-1 ,x t ]+b r )

[0218] r t represents the output of the reset gate, which determines the hidden emotional state h at the previous time step t-1 In computing candidate hidden emotional states Similar to the update gate, σ represents the Sigmoid activation function, which maps the output value to (0,1) to reflect the probability of the emotional state. r Represents the weight matrix of the reset gate. [h t-1 ,x t ] represents the hidden emotional state h of the previous time step t-1 and the current input x t splicing. r Represents the bias vector for the update gate.

[0219] Candidate hidden emotional states It is calculated by the current input emotional state and the hidden emotional state of the previous time step (controlled by the reset gate). Its formula is:

[0220]

[0221] in, represents the candidate hidden emotional state, which combines the current input and the hidden emotional state of the previous time step (subject to the reset gate r tcontrol). tanh(·) represents the hyperbolic tangent activation function, which maps the output value to (-1,1). When the input value is close to 0, the gradient of tanh(·) is large; when the input value is large or small, the gradient of tanh(·) is farther away from 0 than Sigmoid, and it performs better in overcoming the gradient vanishing problem. It has the advantage that the mean of the output value is close to 0, which can help alleviate the offset problem of subsequent layers in the neural network, thereby accelerating convergence. Within the input value range, the gradient of tanh(·) is strong, which is conducive to faster convergence. W h The weight matrix representing the candidate hidden emotional state, [r t ⊙h t-1 ,x t ] represents the hidden emotional state h of the previous time step t-1 (Affected by the reset gate r t control) and the current input x t splicing. h A bias vector representing the candidate hidden emotional state.

[0222] Final hidden emotional state h t The update gate controls the mixture of the current candidate hidden emotional state and the hidden emotional state of the previous time step. Its formula is:

[0223]

[0224] Among them, h t represents the final hidden emotional state of the current time step. ⊙ represents the bitwise multiplication of elements. (1-z t ) represents the reverse output of the update gate, which controls the hidden emotional state h of the previous time step t-1 The retention ratio. t Represents the output of the update gate, controlling the candidate hidden emotional state The retention ratio.

[0225] In the second stage, subtasks such as sentiment analysis and text generation are selected as target tasks for multi-task fine-tuning. The sentiment analysis task aims to identify the emotional tendency in the text, while the text generation task aims to generate corresponding text based on the given sentiment. A joint loss function is designed to combine the goals of subtasks such as sentiment analysis and text generation. A multi-task learning framework is used to achieve the weights of the DeepSpeed-MoE large model between multiple subtasks by weighted combination of the loss functions of multiple subtasks to optimize the performance of the model.

[0226] The core of the multi-task learning framework is to train a unified model by weighted combination of loss functions of multiple subtasks. For sentiment analysis and text generation tasks, the following sentiment analysis loss function can be defined: And this generated loss function Loss Function

[0227]

[0228] Among them, α and β are hyperparameters used to control the loss weights of sentiment analysis and text generation tasks.

[0229] Sentiment analysis is a classification problem, where the goal is to predict the sentiment label y of the input text x. SA The loss function uses cross entropy loss:

[0230]

[0231] Where N is the number of samples, C is the number of sentiment categories, is the true sentiment label of sample i, is the emotion probability predicted by the model, and the parameter is θ.

[0232] The text generation task is a generative task whose goal is to generate emotional text y based on input x. TG The loss function uses the sequence-to-sequence negative log-likelihood (NLL) loss:

[0233]

[0234] Where T is the length of the generated text, y TG,i,t is the target word of sample i at time step t, y TG,i,<t are all the words of sample i before time step t.

[0235] During fine-tuning, the entire DeepSpeed-MoE model is fine-tuned using the data of the target task, freezing the earlier layers (those close to the input layer) and only training the later layers and the output layer. This is because the earlier layers capture lower-level features, while the later layers and the output layer are able to recognize higher-level features and ultimately complete the task. Freeze the early layers and freeze the parameters close to the input layer to keep them in the state learned during pre-training; fine-tune the later layers and only train the parameters close to the output layer to better adapt to the characteristics of the specific task. A combination of supervised learning and reinforcement learning is used to gradually adjust the parameters of the model to make it more suitable for the data distribution and characteristics of the specific task. During the fine-tuning process, the batch gradient descent method in S102 is used to update the parameters of the model to reduce the value of the loss function.

[0236] Step S104 is a specific step for predicting the spread of public opinion events.

[0237] In step S104, sentiment analysis library SnowNLP is used to perform sentiment polarity analysis, extract sentiment features, sentiment polarity and sentiment intensity. Specifically, SnowNLP uses a dictionary-based method to score the sentiment polarity of the input text, classifying the sentiment into three categories: positive, neutral and negative. These sentiment polarity values ​​are used as sentiment features to provide basic data support for subsequent communication simulation and prediction. In addition, based on the interactive relationship between users, the NetworkX library is used to construct a communication network diagram, and the formula for extracting the communication path is:

[0238]

[0239] Among them, P(u,v) represents the probability of user u spreading information to user v, W(u,v) represents the interaction weight between user u and user v, and D(u) is the out-degree of user u. The interaction weight W(u,v) reflects the frequency of interaction between user u and user v, and the out-degree D(u) represents the influence of user u in information dissemination. Combining these two factors can more accurately predict the dissemination path of information in social networks.

[0240] Based on graph theory, the classic SIR (Susceptible-Infected-Recovered) model is used to simulate the spread of public opinion in social networks. The specific model definition is as follows: S(t) is the proportion of infected people at time t, I(t) is the proportion of susceptible people at time t, and R(t) is the proportion of recovered people at time t. The dynamic equation of the model is:

[0241]

[0242]

[0243]

[0244] Among them, β is the propagation rate and γ is the recovery rate. Through this dynamic simulation based on differential equations, the evolution of public opinion propagation can be better captured. It is worth noting that the SIR model assumes that users are either susceptible, infected, or recovered at a certain point in time. This discrete state transition can better describe the dynamic characteristics of public opinion propagation.

[0245] In addition, we use complex network theory to build a social media public opinion propagation model. The specific method is to use the Barabasi-Albert (BA) model to generate a scale-free network to simulate the interactive relationship between social media users. The BA model assumes that newly added nodes in the network tend to connect to nodes with larger degrees, thus forming a power-law degree distribution, which is very consistent with the characteristics of real social networks. With such a network topology, we can further analyze the importance of nodes. Use the PageRank algorithm to calculate the importance of user nodes and identify key propagation nodes:

[0246]

[0247] Among them, PR(u) is the PageRank value of node u, d is the damping factor (usually taken as 0.85), N is the total number of nodes, M(u) is the set of all incoming link nodes of node u, and L(v) is the out-degree of node v. Nodes with high PageRank values ​​often represent stronger communication influence and are key nodes for the dissemination of public opinion events, which deserve special attention.

[0248] In order to further improve the prediction accuracy, the LSTM network is used for time series prediction. The input of LSTM is the emotional feature sequence, and the output is the emotional state of the next time step. The calculation formula of LSTM is as follows:

[0249] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0250] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0251]

[0252]

[0253] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0254] h t =o t tanh(C t )

[0255] Among them, f t 、it , C t , o t and h t They are the forget gate, input gate, candidate cell state, cell state, output gate and hidden state. LSTM can effectively model long-range dependencies in time series data and is very suitable for predicting the process of public opinion propagation.

[0256] Finally, the random forest algorithm is used to classify and predict the propagation path. The calculation formula of the random forest is:

[0257]

[0258] Among them, h(x) is the prediction result of random forest, N is the number of trees, and h i (x) is the prediction result of the i-th tree. Random forest can make full use of the integrated advantages of multiple propagation path decision trees. Specifically, random forest first generates multiple decision trees on the training set, and each tree is trained on a randomly selected feature subset. When predicting new samples, random forest allows all decision trees to make predictions, and then uses voting to obtain the final classification results. This integrated learning method can effectively deal with the overfitting problem and improve the generalization ability of the model. Combined with the results of sentiment analysis, the trained LSTM and random forest models are used to predict the propagation path and impact range of public opinion events, calculate the propagation probability and propagation speed of each node, and draw a propagation path diagram. Using the results of PageRank and LSTM models, key propagation nodes are identified, focusing on the emotional state and interactive behavior of these nodes, and predicting their impact on the overall propagation.

[0259] Steps S105 to S106 are specific steps for generating controlled products for multi-dimensional guidance of individual emotions and communication content.

[0260] In step S105, the features that affect the emotional effect are divided into four dimensions: media dimension, image dimension, text dimension and fusion dimension. On this basis, a framework for fusion generation of multi-dimensional features of individual emotions is constructed. A framework and method for generating emotional-oriented content based on generative adversarial networks are used to generate text, images, audio and other communication content that meets the target emotion.

[0261] The controllable generation technology of emotion-oriented graphic content uses conditional generative adversarial networks (CGANs) combined with generative models and discriminative models to generate matching emotion-oriented strategies and graphic content through given conditions (such as emotion categories or descriptions). CGANs introduces conditional variables on the basis of traditional GANs to control specific attributes of generated content. CGANs contains two neural networks: a generator (G) and a discriminator (D). The generator G receives random noise z and conditional variables c to generate samples G(z) that match the conditions; the discriminator D receives real samples or generated samples and conditional variables c to determine whether the input samples are real or generated. The goal of CGANs is to enable the generator to generate high-quality samples guided by conditional variables through mutual games, and the use of the maximum and minimum strategy is to achieve adversarial training between the generator and the discriminator. The core idea of ​​this training strategy is that through the adversarial process, the generator can generate more and more realistic samples, while the discriminator continuously improves its ability to distinguish between real samples and generated samples. The goal of the generator is to generate samples that can "fool" the discriminator, minimize the difference between the generated samples and the real samples, make the generated samples more realistic, and make it impossible for the discriminator to distinguish the difference between these samples and the real samples. The goal of the discriminator is to distinguish between real samples and generated samples, maximize the loss function, make its discriminative ability the strongest, and judge the authenticity of the input samples as accurately as possible. Its optimization goal is:

[0262]

[0263] The loss function of CGANs is for:

[0264]

[0265] Among them, x represents the real sample. data (x) represents the distribution of real data. z represents the distribution of noise p z (z). c represents the conditional variable, such as emotion category or description. G(z|c) represents the sample generated by the generator. D(x|c) represents the judgment of the discriminator on the real sample x. D(G(z|c)|c) represents the judgment of the discriminator on the generated sample.

[0266] Using the adversarial state tracking technology, combined with natural language processing (NLP) and named entity recognition (NER), a dialogue state tracking module is established to evaluate the topics and opinions in the dialogue and complete the corrections. The NER model is constructed using a bidirectional long short-term memory network (BiLSTM) combined with a conditional random field (CRF). According to the dialogue history and user feedback, the dialogue state is updated, and key correction information is recorded and fed back to the generation model. Given an input sequence x = (c1, x2, ..., x n ,), each x i Represents a word vector or character vector of a word. BiLSTM can capture context information and process the input sequence through two LSTM units, forward and backward. The output of BiLSTM is represented as:

[0267]

[0268] in and denote the hidden states of the forward and backward LSTM units at time step t, respectively, and h t is the connection between the forward and backward hidden states. The CRF layer is used to capture the dependencies between labels, especially the label transfer relationship in sequence labeling tasks. Define the transfer matrix A, where A i,j represents the score of transferring from label i to label j. Given an input sequence x and its corresponding label sequence y = (y1, y2, ..., y n ), calculate the score function:

[0269]

[0270] Where P represents the output score matrix of BiLSTM, Indicates that the label is y at time step t t The score function is the core tool for evaluating the quality of generated content. The score function directly reflects the quality of the generated text. A high score means that the generated text is more in line with expectations, while a low score may indicate that there are problems with the generated text. During the training process, by optimizing the score function, the model can learn better generation strategies. The common optimization goal is to maximize the value of the score function.

[0271] The media dimension features include the user-generated content itself and the publisher features. The image dimension features are divided into shallow pixel features and deep semantic features. The text dimension features are divided into shallow text attribute features and deep text semantic features. The fusion dimension features are used to represent the association between images and texts.

[0272] like Figure 5The framework for fusion generation of multi-dimensional features of individual emotions includes image dimension generation and text dimension generation. Generate individual emotion expression content with multi-dimensional guidance. The framework fuses the features of image and text dimensions to improve the richness and accuracy of the generated results. The fused features have content coordination and emotional coordination. Content coordination represents noun similarity, and emotional coordination represents adjective similarity. In terms of image dimension generation, the framework uses images as input and extracts the visual features of images through image processing technology. Shallow features include spatial size, providing information such as color, shape, texture, and structure, which are represented by storage space size and information entropy respectively. Deep features include emotional coordination, which is represented by adjective similarity. In order to better capture the visual features of individual emotions, a recurrent neural network (RNN) is used to extract image features. By fusing these image features with the multi-dimensional features of individual emotions, image-based individual emotion generation can be achieved. For example, for an image expressing a happy emotion, the framework can first measure its dimensional features, and then generate text descriptions or emoticons similar to its features. In terms of text dimension generation, the framework uses the text description of individual emotions as input and extracts the semantic features of the text through natural language processing technology. The shallow features are text length and liveliness, which are represented by the total number of words and the ratio of expression to text respectively. The deep features are universality and idiosyncrasy, which are represented by universal keywords and idiosyncratic keywords. In order to better capture the text features of individual emotions, a recurrent neural network (RNN) model is used to extract text features. Individual emotion generation based on text is achieved by fusing text features with the multidimensional features of individual emotions. For example, for a text describing a sad emotion, the framework first measures its dimensional features and then generates images or emoticons with similar features.

[0273] like Figure 6 ,In step S106, the memory-driven module processes event information according to ,the knowledge related to historical information, and processes individual characteristics according to individual ,emotional characteristics and opinion content, and then tracks the dialogue ,status through dialogue history and user feedback, and realizes ,optimization designs such as opposition relationship recognition, content modification, and evaluation optimization.,Based on the above two parts, the multi-dimensional guidance strategy generation ,module is assisted.

[0274] The content correction part introduces emoji embedding coding technology and model word segmentation technology to form a highly controllable and interactive multi-dimensional guidance strategy content controlled generation mechanism. With the addition of emoji embedding coding technology, the model will encode emojis as special tags together with other text tags, and then embed emojis in the generated text to enhance emotional expression. The emoji encoding scheme should have the ability to self-iterate to cope with the rapid change trend of Internet vocabulary.

[0275] Represent emojis as embedding vectors. Let E be the set of emojis, T be the set of text tokens, and V = E ∪ T be the expanded vocabulary. Each emoji e∈E and text token t∈T has a corresponding embedding vector e and t. Use a one-hot vector representation:

[0276] e onehot ∈{0,1} |V|

[0277] t onehot ∈{0,1} |V|

[0278] The embedding matrix is ​​a selection of expressions and texts, and each row corresponds to the vector representation of a specific token (such as a word or an emoji). Use the embedding matrix to get the embedding vector:

[0279] e=W e ·e onehot

[0280] t=W t ·t onehot

[0281] in is the embedding matrix corresponding to the expression, is the embedding matrix corresponding to the text, and d is the dimension of the embedding vector.

[0282] The model word segmentation technology regards word segmentation as a state transition process in a hidden Markov model (HMM). It helps to identify and segment proper nouns and terms, and improve the accuracy of text processing. In the HMM model, each word is regarded as a hidden state. The model calculates the probability of each state through the observed character sequence and determines the word segmentation position through these probabilities. The HMM model consists of five parts: state set S, observation set V, initial state probability distribution π, state transition probability matrix A, and observation probability matrix B:

[0283] S={s1,s2,…,s N}

[0284] V={v1,v2,…,v M}

[0285] π={π i}

[0286] A={a ij}

[0287] B = {b ij}

[0288] Among them, s irepresents a hidden state, such as the beginning (B), middle (M), end (E), and single word (S) states of sentiment words. i Represents an observation value (character). π in the initial state probability distribution i Represents the initial state s i The probability of a in the state transition probability matrix ij represents the probability of transitioning from state i to state j. ij Indicates that in state s i Generate observations v i probability.

[0289] Given an observation sequence O = (o1, o2, ..., o T ), the HMM model determines the word segmentation position by calculating the probability of each state sequence. When the forward algorithm calculates the time step t, it takes state s i The probability of the partial sequence at the end is α t (i)

[0290] α t (i)=P(o1,o2,…,o t ,q t =s i )

[0291] The recursive formula is:

[0292]

[0293] where α t (i) At time step t, in state s i The probability of a partial sequence ending in ij State transition probability, from state s i Transfer to state s j The probability of j (o t+1 ) in state s j Generate observations o t+1 Calculate the probability of t at time step t, from state s i The probability of generating a partial sequence is β t (i)

[0294] β t (i) = P(o t+1 ,o t+2 ,…,o T ∣q t =s i )

[0295] The recursive formula is:

[0296]

[0297] Among them, β t (i) is at time step t, with state s i The probability of the partial sequence starting. ij State transition probability, from state s i Transfer to state s j The probability of j (o t+1 ) in state s j Generate observations o t+1 The probability of . Using the Viterbi algorithm, find the most likely state sequence, that is, the position of word segmentation:

[0298]

[0299] The recursive formula is:

[0300]

[0301] Emoticon embedding coding technology and model word segmentation technology can work together to optimize and correct the generated content of the dialogue system from different angles to make it more in line with user needs and expectations. Emoticons play an important role in expressing emotions. By embedding emoticons, the system can better understand and generate emotional content. By combining emotion-oriented image and text generation technology, more natural and expected content generation can be achieved. HMM is used for word segmentation to ensure that the generated content is grammatically and semantically correct.

[0302] Steps S107 to S109 are specific steps for demonstrating and verifying the construction of a large model system for predicting and guiding emotion propagation.

[0303] In step S107, a large model system for predicting and guiding emotional communication is constructed. Figure 7 The system consists of a data processing layer, a model building layer, a demonstration and verification layer, and computing resources. The data processing layer is responsible for the classified storage of various types of data, and also has the function of annotating these data; the model building layer is responsible for building models in various research contents to complete the task of generating controlled items in a complex network environment; the demonstration and verification layer is a complex demonstration and verification of individual and group emotion propagation analysis and prediction guidance under specific public opinion events, supported by the public opinion event propagation status and trend analysis and prediction model.

[0304] In step S108, a public opinion event propagation status and trend analysis and prediction model is established, such as Figure 8As shown in the figure, we use a sentiment analysis dataset including comment data, user information and user behavior data, and sentiment analysis and cognitive models to build a multi-layer network for user interaction, generate a sentiment propagation prediction model based on the network, and finally summarize the propagation rules from the simulation results and make propagation predictions.

[0305] The generation and development of online public opinion events first start with the sender of public opinion messages, and then spread through online media, and finally involve the public opinion audience, and then generate, develop and expand according to the initiative of the public opinion audience. According to the generation and development rules of online public opinion events, the online public opinion index is established from three aspects: the source of online public opinion, the dissemination of public opinion, and the public opinion audience.

[0306] A BP neural network model for evaluating the spread of public opinion events is established to improve the accuracy and precision of the evaluation of the comprehensive effect of the spread of public opinion events. This neural network structure can process multi-dimensional data, model complex nonlinear relationships, and achieve real-time dynamic learning and adaptive adjustment, so as to better analyze the spread of public opinion events and provide reliable decision support and guidance strategies for public opinion control.

[0307] According to the number of indicators in the system described previously, there are 16 third-level indicators that affect the evaluation index of the spread of public opinion events. Therefore, the number of neurons in the input layer of the BP neural network model is 16. It is only necessary to obtain the evaluation index of the spread of public opinion events, so the number of neurons in the output layer is 1. There is a direct correlation between the number of hidden layers of the BP neural network and its learning efficiency. Theoretically, a BP neural network with at least 3 layers can approximate any continuous function, that is, a BP network with only 1 hidden layer can relatively simply achieve the required function and model almost all nonlinear systems. With the increase of the number of hidden layers, the network error gradually decreases, but the network structure becomes more complex, which is easy to cause longer network training time or overfitting. Therefore, a 3-layer BP neural network is used. The number of neurons in the hidden layer is taken as 6. If the excitation function is not selected, each layer of the neural network only performs a linear transformation on the previous layer. After multiple layers are superimposed, the final output is still a linear transformation of the original input. Therefore, a nonlinear factor is introduced, and the Sigmod function is used as the excitation function, as shown in the following formula:

[0308]

[0309] The modeling of multi-layer social networks involves processing network data at multiple relationship levels. These data contain various types of nodes (users, posts, etc.) and edges (friendships, follow-up relationships, etc.). Combined with BP neural network (back propagation neural network), complex multi-factor input modeling can be achieved. Multi-layer social network modeling: Multi-layer social networks are encoded through graph neural networks (GNNs) to extract node and edge features in each layer of the network. Assuming there are L layers of social networks, the node feature matrix of each layer of the network is H (l) , the edge feature matrix is ​​A (l) , where l = 1, 2, ..., L, the graph convolutional network (GCN) processes each layer of the social network and generates node embeddings, H (l+1) =σ(A (l) H (l) W (l) ), where σ is the activation function, W (l) is the weight matrix of the lth layer, which combines the original input feature x with the nodes H generated by each layer of the social network (l) Splice to form a comprehensive feature vector z = [x; H (1) ;H (2) ;…;H (L) ], combining the multi-layer social network with the BP network, inputting the comprehensive feature vector z into the BP neural network for training, and adjusting the weight matrix W by the back propagation algorithm to minimize the loss function, and finally obtaining the prediction model Based on the combination of BP neural network and multi-layer social network, we can jointly extract the propagation characteristics of public opinion events in spatial and temporal dimensions, and then analyze the propagation status and evolution trend of public opinion events, which provides strong technical support for improving the performance and accuracy of public opinion event prediction models and the management and guidance of public opinion events.

[0310] In step S109, the emotional communication status analysis of typical public opinion events and the prediction guidance demonstration verification are performed. Fig. 9 In the demonstration and verification of the emotion propagation status analysis and prediction guidance of typical public opinion events, from data collection and processing, emotion analysis model construction, emotion propagation path analysis to emotion trend prediction guidance, system verification and optimization, to real-time data monitoring and updating, visualization and report generation, as well as user feedback and system optimization, each step will provide key support and optimization for the establishment of the system. This complete technical route will ensure that the built system plays the greatest role in public opinion event management and provide users with comprehensive and accurate emotion propagation analysis and prediction guidance services. The following are the detailed steps for the demonstration and verification of the emotion propagation status analysis and prediction guidance of typical public opinion events.

[0311] (1) Data collection and processing: Collect data related to historical and current hot public opinion events, including original information, forwarded comment data, mainstream media reports, etc., and clean, integrate and annotate the data to ensure data quality and accuracy.

[0312] (2) Calling the sentiment analysis model: Based on natural language processing technology, a sentiment analysis model is built to identify and analyze the emotional color and emotional tendency in the text. It integrates technologies such as sentiment recognition and sentiment classification to accurately capture and analyze individual and group emotions.

[0313] (3) Emotional propagation path analysis: Use network science methods and data mining technology to analyze the paths and key nodes of emotional propagation in public opinion events. Explore the propagation patterns and influencing factors of individual and group emotions in cyberspace and reveal the mechanism of emotional propagation.

[0314] (4) Emotional trend prediction and guidance: Based on historical data and sentiment analysis models, emotional trend prediction and analysis are carried out to predict the development trend of individual and group emotions. Targeted emotional guidance strategies and suggestions are provided to help managers guide the direction of emotional dissemination in public opinion events.

[0315] (5) System verification and optimization: In typical hot event scenarios, implement demonstration verification of individual and group emotion propagation status analysis and trend prediction guidance. Verify the feasibility of the path of emotion propagation of public opinion events and the effectiveness of the guidance method, and continuously optimize and improve the emotion analysis model and system performance.

[0316] (6) Real-time data monitoring and updating: Establish a real-time data monitoring mechanism to capture new data and update the analysis model in a timely manner to ensure the system's ability to respond quickly to public opinion events. Perform data stream processing to achieve continuous data collection, cleaning and updating to maintain the real-time and accuracy of the system.

[0317] (7) Visualization and report generation: Design a visualization interface to present the analysis results of individual and group sentiment communication, including sentiment trend graphs, key node network graphs, etc., to improve users’ understanding and utilization efficiency of data. Automatically generate detailed reports, summarize analysis results and provide targeted suggestions to help managers quickly understand the situation and trend of public opinion events.

[0318] (8) User feedback and system optimization: Collect user feedback and requirements, continuously improve system functions and performance, and ensure that the system meets the needs of users' actual application scenarios. Use machine learning and deep learning technologies to continuously optimize the system and improve the accuracy and predictive ability of sentiment analysis to cope with the ever-changing public opinion environment.

[0319] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a generative AI emotion propagation prediction and guidance large model and system.

[0320] Corresponding to the above method, the present invention also provides a device, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device implements the steps of the method described above.

[0321] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0322] In summary, the present invention provides a large model and system for predicting and guiding emotion propagation based on generative AI, including: constructing a large model for emotion analysis. Aiming at social media, an open source Chinese large language model and multi-source social media data are used to integrate grammar, semantics and memory drive to construct a large model for generative emotion analysis. Sparse gated hybrid expert training technology is used to improve the efficiency and performance of model training; and controlled products for multi-dimensional guidance of individual emotions and communication content are generated. The model provides stable and effective emotion guidance strategies and content based on individual emotional characteristics and change link analysis, ensures high consistency and traceability, meets the needs of rapidly evolving Internet public opinion, improves the ability of the group to distinguish and respond to negative emotions, and supports typical individual emotional crisis intervention; constructs an emotion propagation prediction and guidance large model system for demonstration and verification. In response to diverse sudden public opinion events, an emotion propagation prediction and guidance large model system is constructed, and the propagation of network public opinion events is analyzed based on spatiotemporal characteristics, and empirical analysis is conducted to improve the accuracy and efficiency of the prediction model, providing technical support for the management and guidance of public opinion events.

[0323] Furthermore, the present invention also designs a technology for collecting and processing multi-source social media individual expression data, using crawlers and API interfaces to obtain representative and diverse social media data, performing preprocessing such as text cleaning, word segmentation, entity recognition and denoising, and providing the training data required for supervised learning through manual labeling and generating language models.

[0324] At the same time, the present invention uses technologies such as generative adversarial networks and variational autoencoders to optimize the framework for generating emotion-oriented content and solve the problem of single content generation in existing models. Based on the differences in emotional intensity, vocabulary, and expression of individuals who are the source of emotions, combined with social network analysis and communication models, the generation mechanism of emotion-oriented content elements is studied to generate a communication content guidance strategy that adapts to individual emotional characteristics.

[0325] In addition, we built an emotion propagation prediction and guidance system, designed the corresponding software architecture, and developed a prediction and guidance system. Aiming at the needs of spatiotemporal propagation evaluation and prediction of online public opinion events, we built a prediction model for the propagation scope and guidance effect of public opinion events, and established a public opinion event propagation evaluation index system based on BP neural network and multi-layer social network. Through multi-factor input modeling, we realized the extraction of multi-dimensional propagation characteristics of public opinion events, and analyzed the propagation status and evolution trend.

[0326] Finally, in response to the needs of public opinion guidance for typical hot events, we conducted demonstration and verification of individual and group emotion propagation status analysis and trend prediction and guidance. Through models and systems, we analyzed and predicted historical and current hot public opinion events, verified the effectiveness of emotion propagation paths and guidance methods, achieved iterative optimization of public opinion event management and guidance, and evaluated the performance and application effect of research results.

[0327] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0328] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0329] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0330] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A generative AI emotion propagation prediction and guidance model construction method, characterized in that: The method comprises the following steps: Step 1: Build a sentiment analysis model based on grammatical semantics and memory drive for social media. Use the language model to model grammatical structure and meaning, and combine the storage and traceability of historical information. Use the open source Chinese language model as the basis, combine social media data, and use sparse gated hybrid expert training technology to build a sentiment analysis model to understand the emotions expressed by individuals on social media and generate responses. Step 2: Carry out the propagation prediction of public opinion events based on the sentiment analysis model, collect and preprocess social media data, extract sentiment features and propagation paths, build a propagation model based on graph theory and complex network theory, simulate the process of information propagation in social networks, use LSTM and random forest to train and optimize the model to improve the prediction accuracy, and finally, combine the sentiment analysis results to predict the propagation path and impact range of public opinion events; Step 3: Generate a multi-dimensional guidance strategy for individual emotions and communication content, trace the individual emotion communication path, use the target emotion controlled generation algorithm, generate a multi-dimensional guidance strategy, achieve effective guidance of individual emotions and communication content, and enhance the group's ability to identify negative emotions and respond to public opinion; Step 4: Establish a public opinion event propagation scope and guidance effect evaluation model based on spatiotemporal feature analysis, select typical public opinion cases for empirical analysis, verify the accuracy and efficiency of the evaluation model, and achieve efficient evaluation of the propagation scope and effect of public opinion events, thereby providing a scientific basis for improving sentiment analysis prediction models and multi-dimensional guidance strategies.

2. The generative AI emotion propagation prediction and guidance large model construction method according to claim 1 is characterized in that: The construction of a large sentiment analysis model based on grammatical semantics and memory drive for social media includes the following steps: Collect and process multi-source social media individual expression data, combine 12345 historical and real-time data, use crawler technology and API interfaces to obtain representative and diverse individual expression data from major social media platforms. The goal is to obtain data covering social media and conversation content in different topics, fields and contexts, and provide data support for the generalization ability of sentiment analysis models for various emotional expressions. The data preprocessing stage will include text cleaning, word segmentation, entity recognition and denoising steps. Shallow noise removes punctuation, redundant spaces and HTML tags, and uses regular expression-based denoising: T′=regex.sub(pattern,replacement,T) Use T′=regex.sub('<.*?>',",T) to remove the crawled HTML tags. Regular expressions can also be used as dictionary mappings to replace noise. For deep-level grammatical errors, spelling errors, irrelevant words, repeated words, semantic noise, and ambiguous noise, the BERT pre-trained language model is used to monitor and remove deep-level noise: First, convert the input text into an embedded representation. For an input text, obtain its embedded representation through the BERT pre-trained language model: E=BERT(T) The classification layer is used to determine whether a word or sentence is noise. Since the specific type of noise is not important, in order to reduce the output categories, only a binary classification task of noise and non-noise is defined. For each embedded vector, its classification probability is calculated using the following formula: p i =softmax(We i +b) Among them, W is the weight matrix of the classification layer, b is the bias vector, softmax(·) is the activation function for binary classification, and the output p i Indicates the probability that the i-th word is noise: T′={w i ∣p i ≤τ} According to the probability of detected noise, a threshold τ is set. If p i ≤τ, the word or sentence is considered to be noise and removed, otherwise it is retained. T′ represents the denoised text. The preprocessed text data is manually added with emotional labels and statistically classified with basic events. Based on these labels and classifications, the GPT generative language model initially generates answers. The generated answers are manually reviewed to determine whether they are in line with the actual situation and context, and necessary adjustments are made. The opinions of psychological experts are further introduced to evaluate and adjust the answers generated by the model to ensure professionalism and accuracy. The answers to these individual expression data are obtained to provide the training data required for supervised learning for the large emotional analysis model. Construct a generative multimodal and multiscale sentiment analysis model. Multimodality refers to text, image, voice, and video modes. Multiscale refers to pixel-level scale and regional-level scale. The open source Chinese language model is used as the basis of the generative sentiment analysis model. The function of the generative multimodal and multiscale sentiment analysis model is to generate text replies with emotional colors. At the same time, it supports the access of multimodal generative models of images, voices, and videos. For different types of data, multimodal feature fusion is performed. Multimodal fusion is performed from three aspects: feature level, attention level, and decision level. At the same time, a multiscale fusion method based on the attention mechanism is used to improve the contribution of each image modality feature. The construction of the generative multimodal and multiscale sentiment analysis model is based on training data. The supervised learning and reinforcement learning methods are used to train and fine-tune the model to reduce hallucinations and increase consistency. Specifically, keep the pre-trained model parameters unchanged, add an output layer with an output size of the number of emotion categories to the target model, and randomly initialize the model parameters of this layer. The output layer of the target model is trained from scratch on the target dataset, and the model output f is calculated. θ (x i ) and calculate the loss The gradient of the original model is obtained by back propagation, and the parameters of the remaining layers are fine-tuned based on the parameters of the source model. The key to fine-tuning is to let the model learn the specific context and expression in the field of sentiment analysis so that the model can adapt to different emotional expressions more flexibly, thereby improving the overall performance of sentiment analysis and generating emotional content that is more in line with the individual expressions on social media. In order to cope with the problems of high computing power requirements and multi-card reasoning difficulties during multi-task reinforcement learning training, a sparse gated hybrid expert network is used for multi-task training. An improved DeepSpeed-MoE model training technology with multi-expert and multi-data parallelism, simplified routing, load balancing, gradient quantization and asynchronous training is proposed. The pre-trained parameter amount is maximized in a simple and computationally efficient way. At the same time, the expert model allocation strategy is dynamically adjusted according to the load and performance of each expert model to efficiently utilize computing resources. The load status of each expert model, including computing load and memory usage, is monitored in real time. The model is distributed in parallel on multiple computing nodes, and each node is responsible for a part of the model parameters, reducing the computing load and memory usage of a single node and improving the reasoning speed. Compared with the traditional MoE routing strategy, the gated network sends the input x of each token to the top-k expert models. In order to reduce communication and computing costs, a simplified strategy is adopted, and each input x is only sent to one expert model. In order to make the load of the expert model more balanced, an auxiliary loss is added: The number of experts is N, f i is the proportion of tokens allocated to expert i, P i is the sum of the probabilities of all tokens in a batch being assigned to expert i. The gradient quantization and compression formula is: Where VW is the gradient vector, and the quantization scaling factor It is used to control the quantization accuracy, reduce the amount of gradient data transmitted over the network, and reduce the communication bandwidth requirements, where n is the quantization bit width; using an asynchronous training strategy, each computing node independently calculates and updates gradients, reduces synchronization overhead, and increases training speed. Each computing node independently calculates gradients and updates parameters, reducing synchronization waiting time. Each node calculates gradients and updates parameters at its own speed to adapt to different computing resources and data distributions. A two-stage fine-tuning strategy is used in the multi-task training phase to improve the training efficiency and performance of the DeepSpeed-MoE model: In the first stage, a large-scale corpus is used to pre-train the sparse gated DeepSpeed-MoE network. Assuming there are E expert networks and an input sample x, the output of the router can be expressed as g(x) = (g1(x), g2(x), ..., g E (x)), and use the softmax function to convert these scores into probabilities, After probability sorting and mask selection, a mask vector is created, whose element value is 1 for selected experts and 0 for unselected experts. The expert with the highest probability is selected for dynamic activation to achieve dynamic allocation of the expert network for effective information integration and learning. Sparsity constraints are used to limit the number of parameters in the gating network, thereby reducing the computational complexity and memory usage of the DeepSpeed-MoE model. In the second stage, the data of the two target tasks, sentiment analysis and text generation, are used to fine-tune the entire DeepSpeed-MoE model. The layers close to the input layer are frozen, and only the later layers and the output layer are trained. This is because the earlier layers capture lower-level features, while the later layers and the output layer can recognize higher-level features and ultimately complete the task. The knowledge learned in the first stage is fine-tuned to be closer to the tasks of sentiment analysis and text generation. By fine-tuning on the target tasks, the entire DeepSpeed-MoE model is made to better fit the data distribution and characteristics of the specific task, thereby improving the performance and generalization ability of the DeepSpeed-MoE model.

3. The generative AI emotion propagation prediction and guidance large model construction method according to claim 1 is characterized in that: The public opinion event propagation prediction based on the sentiment analysis model includes the following steps: The sentiment analysis library SnowNLP is used to perform sentiment polarity analysis, extract sentiment features, sentiment polarity and sentiment intensity. In addition, based on the interactive relationship between users, the NetworkX library is used to build a propagation network diagram. The formula for extracting the propagation path is: Among them, P(u, v) represents the probability of user u spreading information to user v, W(u, v) represents the interaction weight between user u and user v, and D(u) is the out-degree of user u. Based on graph theory, the classic SIR (Susceptible-Infected-Recovered) model is used to simulate the spread of public opinion in social networks. The specific model is defined as follows: S(t) is the proportion of infected people at time t, I(t) is the proportion of susceptible people at time t, and R(t) is the proportion of recovered people at time t. The dynamic equation of the model is: Among them, β is the propagation rate and γ is the recovery rate. In addition, using complex network theory, a social media public opinion propagation model is constructed. The specific method is to use the Barabasi-Albert (BA) model to generate a scale-free network, simulate the interactive relationship between social media users, use the PageRank algorithm to calculate the importance of user nodes, and identify key propagation nodes: Among them, PR(u) is the PageRank value of node u, d is the damping factor, N is the total number of nodes, M(u) is the set of all incoming link nodes of node u, L(v) is the out-degree of node v, and LSTM network is used for time series prediction. The input of LSTM is the emotional feature sequence, and the output is the emotional state of the next time step. The calculation formula of LSTM is as follows: f t =σ(W f ·[h t-1 ,x t ]+b f ) i t =σ(W i ·[h t-1 ,x t ]+b i ) the t =σ(W o ·[h t-1 ,x t ]+b o ) h t =o t ·tanh(C t ) Among them, f t 、i t , C t , o t and h t They are forget gate, input gate, candidate cell state, cell state, output gate and hidden state. Finally, the random forest algorithm is used to classify and predict the propagation path. The calculation formula of random forest is: Among them, h(x) is the prediction result of random forest, N is the number of trees, and h i (x) is the prediction result of the ith tree. Combined with the sentiment analysis results, the trained LSTM and random forest models are used to predict the propagation path and impact range of the public opinion event, calculate the propagation probability and propagation speed of each node, and draw a propagation path diagram. Using the results of PageRank and LSTM models, key propagation nodes are identified, focusing on the emotional state and interactive behavior of these nodes, and predicting their impact on the overall propagation.

4. The generative AI emotion propagation prediction and guidance large model construction method according to claim 1 is characterized in that: The strategy for generating multi-dimensional guidance of individual emotions and communication content includes the following steps: Construct an emotion-oriented content generation framework and method. Aiming at the existing generative language model's generation form and single content problems of emotion-oriented content for an individual's own emotional state, a guidance strategy and content generation model optimization scheme based on generative adversarial networks and variational autoencoder technology is proposed to generate text, image, and audio content that conforms to the target emotion. Memory-driven guidance effect evaluation, correction, and follow-up key technologies are used. Using opposition state tracking technology, natural language processing and named entity recognition are combined to establish a dialogue state tracking module, evaluate the topic and viewpoint in the dialogue, and complete the correction. A bidirectional long short-term memory network is used in combination with conditional random fields to construct a named entity recognition model. According to the dialogue history and user feedback, the dialogue state is updated, and key correction information is recorded and fed back to the generative model. The output of BiLSTM is expressed as: in and denote the hidden states of the forward and backward LSTM units at time step t, respectively, and h t It is the connection between the forward and backward hidden states. The CRF layer is used to capture the dependencies between labels, especially the label transfer relationship in the sequence labeling task. The transfer matrix A is defined, where A i,j represents the score of transferring from label i to label j. Given an input sequence x and its corresponding label sequence y = (y1, y2, ..., y n ), calculate the score function: Where P represents the output score matrix of the bidirectional long short-term memory network, Indicates that the label is y at time step t t The scoring function is the core tool for evaluating the quality of generated content. It can solve the difficulties in guiding the communication direction of target emotions, the lack of emotional guidance evaluation and follow-up, and construct a framework for the fusion generation of multi-dimensional features of individual emotions. According to specific emotional targets, it can generate communication content and emotional guidance program strategies and specific contents that meet individual emotions. Form a multi-dimensional guidance strategy for communication content. Aiming at the emotional intensity, emotional vocabulary, expression method, and social network relationship difference processing methods of the individual source of emotion, combined with the characteristics of the target individual, the model generation mechanism of the emotional guidance content elements based on social network analysis and communication model is studied, breaking through the key nodes and communication path confirmation and evaluation verification methods in emotional communication, and the multi-dimensional guidance strategy generation algorithm model that conforms to the target emotion. Starting from the emoticon embedding coding technology and model word segmentation technology, the formation of a multi-dimensional guidance strategy for communication content that adapts to the individual emotional characteristics is realized, providing technical support for the generation of multi-dimensional guidance solutions that adapt to the individual emotional characteristics. By adding the emoticon embedding coding technology, the multi-dimensional guidance strategy generation algorithm model will use emoticons as special tags and encode them together with other text tags, and then embed emoticons in the generated text to enhance emotional expression. Emoticons are represented as embedding vectors. Let E be the emoticon set, T be the text tag set, V = E ∪ T be the expanded vocabulary, and each emoticon e∈E and text tag t∈T has corresponding embedding vectors e and t, represented by a one-hot vector: e onehot ∈{0,1} |V| t onehot ∈{0,1} |V| The embedding matrix is ​​a selection of expressions and texts. Each row corresponds to the vector representation of a tag for a specific word or emoticon. The embedding matrix is ​​used to get the embedding vector: e=W e ·have been onehot t=W t ·t onehot in is the embedding matrix corresponding to the expression, is the embedding matrix corresponding to the text, and d is the dimension of the embedding vector; The model word segmentation technology regards word segmentation as a state transition process in a hidden Markov model (HMM), which helps to identify and segment proper nouns and terms and improve the accuracy of text processing. In the HMM model, each word is regarded as a hidden state. The HMM model calculates the probability of each state through the observed character sequence and determines the word segmentation position through these probabilities. The HMM model consists of five parts: state set S, observation set V, initial state probability distribution π, state transition probability matrix A, and observation probability matrix B: S={s1,s2,...,s N } V={v1,v2,...,v M } π={π i} A={a ij } B={b ij } Among them, s i represents a hidden state, such as the beginning (B), middle (M), end (E), and single word (S) states of emotional words, and v in the observation set i Represents an observation value (character), π in the initial state probability distribution i Represents the initial state s i The probability of the state transition probability matrix a ij Represents the probability of transitioning from state i to state j. The observation probability matrix b ij Indicates that in state s i Generate observations v i Probability: Given an observation sequence O = (o1, o2, ..., o T ), the HMM model determines the word segmentation position by calculating the probability of each state sequence. When the forward algorithm calculates the time step t, it takes state s i The probability of the partial sequence at the end is α t (i) α t (i)=P(o1,o2,…,o t ,q t =s i ) The recursive formula is: where α t (i) At time step t, in state s i The probability of the partial sequence at the end, a ij State transition probability, from state s i Transfer to state s j The probability of b j (o t+1 ) in state s j Generate observations o t+1 The probability of , calculated at time step t, from state s i The probability of generating a partial sequence is β t (i) β t (i)=P(o t+1 ,o t+2 ,...,o T |q t =s i ) The recursive formula is: Among them, β t (i) is at time step t, with state s i The probability of the starting partial sequence, a ij State transition probability, from state s i Transfer to state s j The probability of b j (o t+1 ) in state s j Generate observations o t+1 The probability of using the Viterbi algorithm to find the most likely state sequence, that is, the position of word segmentation: The recursive formula is:

5. The generative AI emotion propagation prediction and guidance large model construction method according to claim 1 is characterized in that: The construction of the emotion propagation prediction model comprises the following steps: Establish a public opinion event propagation status and trend analysis and prediction model, aim at the spatiotemporal propagation evaluation and prediction needs of various public opinion events suddenly generated in cyberspace, build a public opinion event propagation scope and guidance effect prediction model, and establish a public opinion event propagation evaluation method based on BP neural network and multi-layer social network research: The indicator system framework is constructed from the three perspectives of public opinion source, public opinion propagation, and public opinion audience. The BP neural network and GCN graph convolutional network are used to perform multi-factor input modeling and multi-layer social network modeling of the three indicators of public opinion source, public opinion propagation, and public opinion audience. The best public opinion event propagation prediction model is trained. 16 third-level indicators affect the public opinion event propagation evaluation index. Therefore, the number of input layer neurons in the BP neural network model is 16. Only the public opinion event propagation evaluation index needs to be obtained. Therefore, the number of output layer neurons is 1. There is a direct correlation between the number of hidden layers of the BP neural network and its learning efficiency. Theoretically, a BP neural network with at least 3 layers can approximate any The continuous function of interest, that is, the BP network with only one hidden layer can relatively simply realize the required function and model almost all nonlinear systems. With the increase of the number of hidden layers, the network error gradually decreases, but the network structure becomes more complicated, which is easy to cause longer network training time or overfitting. Therefore, a 3-layer BP neural network is used, and the number of neurons in the hidden layer is 6. If the activation function is not selected, each layer of the neural network is just a linear transformation of the previous layer. After multiple layers are superimposed, the final output is still a linear transformation of the original input. Therefore, a nonlinear factor is introduced and the Sigmod function is used as the activation function, as shown in the following formula: The modeling of multi-layer social networks involves processing network data at multiple relationship levels. These data contain multiple types of nodes and edges. Nodes represent users and posts, and edges represent friend-like and follow-up relationships. Combined with BP neural networks, complex multi-factor input modeling can be achieved. Multi-layer social network modeling: GNN graph neural network is used to encode multi-layer social networks, and node and edge features in each layer of the network are extracted. Assuming there are L layers of social networks, the node feature matrix of each layer of the network is H (l) , the edge feature matrix is ​​A (l) , where l = 1, 2, ..., L, the GCN graph convolutional network processes each layer of the social network and generates node embeddings, H (l+1) =σ(A (l) H (l) W (l) ), where σ is the activation function, W (l) is the weight matrix of the lth layer, which combines the original input feature x with the nodes H generated by each layer of the social network (l) Splice to form a comprehensive feature vector z = [x; H (1) ;H (2) ;...;H (L) ], combining the multi-layer social network with the BP network, inputting the comprehensive feature vector z into the BP neural network for training, and adjusting the weight matrix W by the back propagation algorithm to minimize the loss function, and finally obtaining the prediction model Based on the combination of BP neural network and multi-layer social network, the propagation characteristics of public opinion events in spatial and temporal dimensions are extracted, and then the propagation status and evolution trend of public opinion events are analyzed, which provides strong technical support for improving the performance and accuracy of public opinion event prediction models and the management and guidance of public opinion events.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Citation Information

Patent Citations

  • Large language model training method and device

    CN118246505A

  • Whole-network big data public opinion monitoring system based on cloud media

    CN118349626A

  • Deep learning-based system for improving the performance of sentiment analysis using social media data and management strategies

    DE202023105444U1

Cited By

  • Long text processing method and related device

    CN119204234A

  • Intelligent management system of service consultation platform

    CN119396960A

  • Unmanned aerial vehicle cluster optimal path planning method and system

    CN119597020A

  • Man-machine collaborative public opinion monitoring method and system based on reinforcement learning

    CN119646595A

  • Optimization method for deploying hybrid expert model in edge computing environment and related equipment

    CN119721178A