Method, device and equipment for promoting retention through interactive scene prediction and storage medium
By preprocessing and extracting features from multimodal interaction data and using the hidden Markov model and Viterbi algorithm to infer emotional state paths, the problem of insufficient user emotional connection in existing technologies is solved, dynamic prediction and real-time feedback of user emotional states are achieved, and the platform retention rate is improved.
Patent Information
- Application Number
- CN202511262959.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-05
- Publication Date
- 2025-10-03
- Estimated Expiration
- 2045-09-05
AI Technical Summary
Existing emotion recognition technology has shortcomings in dynamic prediction and real-time feedback. It is difficult to achieve dynamic prediction of user emotional state through time series modeling of multimodal interaction data, resulting in insufficient user emotional connection and low platform retention rate.
By obtaining multimodal interaction data streams for preprocessing and feature extraction, a real-time multidimensional observation vector sequence is generated. The pre-trained hidden Markov model and Viterbi algorithm are used to infer the emotional state path, driving the mimetic performance of the virtual pet to provide intuitive feedback.
It achieves dynamic prediction and real-time feedback of users’ emotional states, enhances users’ emotional connections, and improves platform retention rates.
Smart Images

Figure CN120744855A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing on social platforms, and in particular to a method, apparatus, device and storage medium for improving user retention through interactive scenario prediction. Background Art
[0002] With the booming development of social applications, user emotional experience has become a core factor influencing user retention and engagement on these platforms. In one-on-one interactions, accurately perceiving and predicting the other person's emotional state and providing intuitive emotional feedback are crucial for enhancing user engagement and the interactive experience. However, existing emotion recognition technologies have significant shortcomings in dynamic prediction and real-time feedback.
[0003] Traditional sentiment analysis methods primarily rely on static rule matching or single-moment classification models. Rule-based methods rely heavily on predefined sentiment lexicons, making them difficult to adapt to the diverse ways users express themselves. They also suffer from low accuracy when processing unstructured content such as internet slang and emoticons. While static classification models can identify the emotional leaning of a single message, they ignore the temporal evolution of emotions during interactions and fail to capture the dynamic trajectory of user emotions.
[0004] Even though some research has introduced deep learning techniques, such as LSTM networks for sequence modeling, these approaches often focus on a single modality, text, and fail to fully leverage the complementary nature of multimodal information, such as speech, images, and behavior. More importantly, existing technologies lack mechanisms for translating emotion prediction results into user-perceivable interactive feedback, making it difficult for users to intuitively sense the system's understanding of their emotional state.
[0005] In addition, existing methods are not capable of handling the uncertainty of emotional state transitions. User emotions often have potential state transition patterns, but this transition path is implicit and probabilistic. Traditional methods have difficulty effectively modeling this random process, resulting in a significant decrease in prediction accuracy in complex scenarios such as continuous conversations and emotional reversals. In view of this, this application is filed. Summary of the Invention
[0006] The present invention discloses a method, apparatus, device and storage medium for improving retention through interactive scenario prediction, aiming to solve the problem that it is difficult to dynamically predict the user's emotional state through time series modeling of multimodal interactive data, resulting in insufficient user emotional connection and low platform retention rate.
[0007] A first embodiment of the present invention provides a method for improving retention through interactive scenario prediction, comprising: Obtaining current multimodal interaction data streams of Class A users and Class B users in a target community, preprocessing the current multimodal interaction data streams, and performing feature extraction on the preprocessed current multimodal interaction data streams to generate a real-time multidimensional observation vector sequence, wherein the current multimodal interaction data streams include text messages, voice messages, picture messages, and user behavior data; Inputting the real-time multidimensional observation vector sequence into a pre-trained hidden Markov model and inferring the optimal emotional state path at the current moment through the Viterbi algorithm, wherein the pre-trained hidden Markov model is trained based on the historical multidimensional observation vector sequences of class A users and class B users; The emotional state at the current moment is extracted from the optimal emotional state path, and the mimicry performance of the virtual pet is driven based on the emotional state at the current moment and displayed on the interactive interface.
[0008] Preferably, the pretreatment process includes: Noise filtering and stop word removal are performed on the text message to obtain the purified text. The processing process is as follows: filtered =Filter(T raw ), where T raw is the original text message, T filtered is the purified text after filtering; The voice message is framed and processed as follows: X=Frame-Split(Vraw)=[X1,X2,...,X n ]; where Vraw is the original speech signal, X i is the i-th speech frame, n is the total number of speech frames; Normalize the size and pixels of the image message. The expression is: I pre =Normalize(Resize(I raw ,size),μ,σ); where I raw is the original image, size is the target size, μ is the pixel mean, and σ is the pixel standard deviation; Aggregate user behavior data in time windows, and the expression is: agg =Aggregate(B raw ,Δt), where B raw is the original behavior data sequence, Δt is the time window size, B agg is the behavioral feature after aggregation, Resize is the size normalization, and Normalize is the pixel normalization operation.
[0009] Preferably, the feature extraction is performed on the pre-processed current multimodal interaction data stream to generate a real-time multi-dimensional observation vector sequence, specifically: The text sentiment feature is extracted from the purified text, and its expression is: S text =BERT(Word-Segment(T filtered )); Among them, Word-Segment is the word segmentation function, and BERT is the pre-trained language model S text is the sentiment score of the text; The speech emotion feature is extracted from the speech frame sequence, and its expression is:
[0010] in, is the fundamental frequency of the i-th frame, is the energy of the i-th frame, is the Mel frequency cepstral coefficient of the i-th frame, 、 、 is the weight coefficient; The emotional features of the image are extracted from the preprocessed image, and its expression is: S image =ResNet(I pre ); ResNet is a residual neural network, S image is the picture sentiment index; The behavioral features are extracted from the aggregated user behavior data, and its expression is: S behavior =[G score ,R delay ]; among them, G score is the normalized gift value score, R delay is the normalized response delay time; Combine to generate the observation vector o at time t t =[S text ,S voice ,S image ,G score ,R delay ] T , the observation vectors of consecutive time steps are combined into a real-time multi-dimensional observation vector sequence.
[0011] Preferably, the training process of the hidden Markov model is: Define the hidden Markov model, λ=(S,π,A,B), where S is the hidden state set, π is the initial state probability distribution, A is the state transition probability matrix, and B is the emission probability matrix. Multidimensional Gaussian distribution is used for modeling:
[0012] Where d is the dimension of the observation vector, Status The mean vector of is the covariance matrix, is observed at time t When in state probability; The Baum-Welch algorithm is used to iteratively optimize the model parameters, where the forward probability is calculated in the E step and backward probability , and then calculate the state posterior probability:
[0013] Update the model parameters in M steps: Update the initial state probability:
[0014] Update state transition probability:
[0015] Update the mean vector of emission probabilities:
[0016] in, From state s at time t i Transfer to s j The joint probability of, T is the length of the training sequence, is the updated initial state probability, is the updated slave state s i Transfer to state s j The transition probability, The updated state s j The mean vector of 、 They are in state s at time t i and s j The posterior probability of .
[0017] Preferably, the real-time multi-dimensional observation vector sequence is input into a pre-trained hidden Markov model, and the optimal emotional state path at the current moment is obtained by inference using the Viterbi algorithm, specifically: In the initialization phase: for each possible initial state, calculate the product of its initial probability and the probability of the state generating the first observation vector, obtain the path probability of each state at the first moment, and initialize the path record to empty; In the recursive stage: starting from the second moment until the current moment, for each possible target state, traverse all possible predecessor states, calculate the product of the probability of transitioning from the predecessor state to the target state and the cumulative path probability of the predecessor state, select the predecessor state that maximizes this product as the optimal predecessor, and multiply the optimal predecessor with the probability of generating the current observation vector of the target state to obtain the maximum path probability to the target state. At the same time, record the optimal predecessor state of each state for path backtracking; In the termination phase: at the current moment, compare the path probabilities of all states and select the state with the highest probability as the optimal state at the current moment; In the retrospective stage: starting from the optimal state at the current moment, according to the recorded optimal predecessor state information, gradually trace back to the first moment to obtain the complete optimal emotional state path.
[0018] Preferably, the method further includes adaptively adjusting the weight of the multimodal data, specifically: Calculate the difference in sentiment scores between adjacent moments of each modal data within the set time window, and take the average as the sentiment change rate of the modality; The modal weights are adjusted according to the emotion change rate. The weights of the modalities with high change rates are increased, and the weights of the modalities with low change rates are reduced. The weight update amplitude is controlled by the learning rate. The eigenvalues of each modality are multiplied by the updated weights to form a new observation vector, which is then normalized.
[0019] A second embodiment of the present invention provides an apparatus for improving retention through interactive scenario prediction, comprising: a vector sequence generation unit, configured to obtain current multimodal interaction data streams of Class A users and Class B users in a target community, preprocess the current multimodal interaction data streams, and perform feature extraction on the preprocessed current multimodal interaction data streams to generate a real-time multidimensional observation vector sequence, wherein the current multimodal interaction data streams include text messages, voice messages, picture messages, and user behavior data; an emotional state path generation unit, configured to input the real-time multidimensional observation vector sequence into a pre-trained hidden Markov model and obtain the optimal emotional state path at the current moment through Viterbi algorithm inference, wherein the pre-trained hidden Markov model is trained based on the historical multi-dimensional observation vector sequences of class A users and class B users; The pet driving unit is used to extract the emotional state at the current moment from the optimal emotional state path, and drive the mimicry performance of the virtual pet based on the emotional state at the current moment, and display it on the interactive interface.
[0020] A third embodiment of the present invention provides a device for improving retention through interactive scenario prediction, including a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement a method for improving retention through interactive scenario prediction as described in any one of the above items.
[0021] A fourth embodiment of the present invention provides a computer-readable storage medium, characterized in that it stores a computer program, and the computer program can be executed by a processor of the device where the computer-readable storage medium is located to implement a method for improving retention through interactive scenario prediction as described in any one of the above items.
[0022] The present invention provides a method, apparatus, device, and storage medium for improving user retention through interactive scenario prediction. This method captures the current multimodal interaction data stream between users in an interactive scenario, performs preprocessing such as noise filtering, frame segmentation, size normalization, and temporal aggregation, and extracts emotional features to generate a real-time multidimensional observation vector sequence. This sequence captures the complementarity and temporal dependencies of multimodal information, overcoming the limitations of single-modality static analysis. The observation vector sequence is input into a pretrained Hidden Markov Model (HMM), which is trained based on historical sequences and defines a set of hidden states, initial probabilities, a transition matrix, and emission probabilities. Viterbi algorithm inference is used to obtain the optimal emotional state path at the current moment, enabling probabilistic modeling and dynamic prediction of emotional transition uncertainty. The current emotional state is extracted from the path and displayed on the interactive interface as a virtual pet's mimicry, creating an intuitive feedback mechanism that strengthens emotional connection and improves user retention. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 This is a flowchart of a method for improving retention through interactive scenario prediction provided by the first embodiment of the present invention; Figure 2 This is a module diagram of a device for improving retention through interactive scenario prediction provided by the second embodiment of the present invention. DETAILED DESCRIPTION
[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0025] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0026] The present invention discloses a method, device, equipment and storage medium for improving retention through interactive scenario prediction, aiming to solve the problem that it is difficult to dynamically predict the user's emotional state through the time series modeling of multi-modal interaction data, resulting in insufficient user emotional connection and low platform retention rate.
[0027] Please refer to Figure 1 , the first embodiment of the present invention provides a method for improving retention through interactive scenario prediction, which can be executed by a device for improving retention through interactive scenario prediction (hereinafter referred to as the retention device or system), specifically, by one or more processors in the retention device or system, to at least achieve the following steps: S101, obtain the current multi-modal interaction data stream of type A users and type B users in the target community, preprocess the current multi-modal interaction data stream, and extract features from the preprocessed current multi-modal interaction data stream to generate a real-time multi-dimensional observation vector sequence, wherein the current multi-modal interaction data stream includes text messages, voice messages, picture messages and user behavior data; In this embodiment, the retention device can be a terminal with data processing capabilities such as a server, a workstation, a desktop computer, a laptop computer, etc. The retention device can be installed with corresponding operating systems and application software, and the functions required in this embodiment can be realized through the combination of the operating system and the application software; It should be noted that in the one-to-one social interaction scenario of the target community, the system needs to obtain the current multi-modal interaction data stream between type A users and type B users in real time. When users interact on the interaction interface, the system collects various data in real time through the message queue mechanism, including text messages, voice messages, picture messages sent by users, and user behavior data such as gift-giving behavior and response time. These raw data often contain a lot of noise and redundant information, and direct use will seriously affect the accuracy of emotion prediction. Therefore, systematic preprocessing is required.
[0028] For the preprocessing of text messages, the system first identifies and filters out noise content such as garbled characters, duplicate characters, and special symbols in the messages, and then removes stop words such as "de", "le", "a" that contribute less to emotion judgment. The original text T is filtered through the Filter function rawConverted to purified text Tfiltered. For example, when the user inputs "I am so happy today!!!", the system will process it as "I am so happy today", retaining the core emotional information while removing redundant expressions. Due to its continuous characteristics, voice messages need to be framed. The system uses the Frame-Split function to split the original voice signal Vraw into n voice frames [X1, X2, ..., Xn] according to a fixed duration (usually 20-40 milliseconds). Each frame is used as an independent analysis unit to facilitate the subsequent extraction of acoustic features such as pitch and energy. Picture messages often have various formats and sizes. The system uses the Resize function to convert the original picture T raw The images are uniformly resized to a standard size (e.g., 224×224 pixels), and then the pixel values are normalized using the Normalize function based on the preset pixel mean μ and standard deviation σ to ensure that images from different sources are within the same numerical range, thus improving the stability of the model processing. User behavior data is characterized by time series and sparseness. The system aggregates the original behavior sequence B within the time window Δt (e.g., 5 minutes) using the Aggregate function. raw Aggregate and count the number of gifts given and average response time during the period to form structured behavioral characteristics B agg .
[0029] After preprocessing, the system enters the feature extraction phase, converting data from different modalities into unified numerical features. For the purified text, the system first uses the Word-Segment segmentation function to segment the continuous text into word sequences. For example, the word "today is really happy" is segmented into ["today", "really", "happy"]. Then, the system inputs the BERT pre-trained language model and uses its deep semantic understanding ability to map the word sequence into a sentiment score S between -1 and 1. text , positive values indicate positive emotions, negative values indicate negative emotions, and the absolute value reflects the intensity of emotions. Speech feature extraction is more complex. The system performs i Three key features are calculated: pitch, which reflects intonation changes and is obtained through a pitch detection algorithm; energy, which reflects volume strength and is obtained by calculating the root mean square of the amplitude of the sampling points within a frame; and mel-frequency cepstral coefficients, which capture timbre characteristics and are obtained through a mel filter bank and cepstral analysis. These three features are assigned weight coefficients α, β, and γ (satisfying α+β+γ=1, usually set to α=0.4, β=0.3, and γ=0.3 in practice), and a comprehensive speech emotion score is obtained through weighted averaging. Image feature extraction utilizes the ResNet residual neural network. Pre-trained on large-scale datasets like ImageNet, this network can effectively identify visual elements such as expressions and scenes in images, outputting a sentiment index (Simage) between 0 and 4, where 0 represents extreme negativity and 4 represents extreme positivity. Behavioral feature extraction is relatively straightforward. The system normalizes the value of gift-giving behavior according to platform pricing to obtain a Gscore (range 0-1), and normalizes the response delay time according to empirical thresholds (e.g., 0 for an immediate response and 1 for a response exceeding 30 minutes) to obtain Rdelay.
[0030] Finally, the system combines all the extracted features into a five-dimensional observation vector o at time t t =[S text ,S voice ,S image ,G score ,R delay ] T , this vector describes the multimodal emotional expression of the user at that moment. As the interaction continues, the system continuously generates new observation vectors, forming a real-time multidimensional observation vector sequence o={o1,o2,...,o t This sequence fully records the dynamic changes in the user's emotions. It should be noted that through modal fusion, the system can capture complex emotional information that is difficult to reflect with a single modality. For example, a user may express calmness in writing but reveal anxiety in voice, or send a smiley face but receive a long reply delay. The comprehensive analysis of these contradictory signals can more accurately reflect the user's true emotional state.
[0031] S102: Inputting the real-time multi-dimensional observation vector sequence into a pre-trained hidden Markov model and inferring the optimal emotional state path at the current moment using a Viterbi algorithm, wherein the pre-trained hidden Markov model is trained based on the historical multi-dimensional observation vector sequences of the users in category A and category B. It's important to note that after the system generates a real-time multidimensional observation vector sequence, it needs to be input into a pre-trained hidden Markov model for emotional state inference. This hidden Markov model is trained based on historical interaction data from a large number of Category A and Category B users within the target community. The model parameters incorporate the emotional transition patterns of the user group. The model defines four hidden emotional states: High Positive, Neutral Positive, Neutral Negative, and High Negative. While these states cannot be directly observed, they can be inferred from users' multimodal behavioral expressions.
[0032] When a real-time observation vector sequence is input into the model, the system uses the Viterbi algorithm to infer the optimal path. This algorithm, essentially a dynamic programming algorithm, finds the path with the highest probability among all possible state sequences. The inference process begins with the initialization phase, where the system first processes the first observation vector o1. For each of the four possible initial states, the system calculates the product of the initial probability π of each state and the emission probability b(o1) of that state generating the observation vector o1. For example, if a user opens the conversation with a "Nice to meet you" message accompanied by a smiley emoji, the initial probability of the HighPositive state might be 0.3. The probability of this state generating this positive observation is 0.8, resulting in an initial probability of 0.24 for the path. The system calculates this initial path probability δ1 for each state and creates an empty path record ψ1 for subsequent backtracking.
[0033] After entering the recursive phase, the system starts to process subsequent observation vectors step by step from the second moment. For each possible target state s at time t j , the system needs to determine the optimal path to reach this state. Specifically, the system traverses all four possible predecessor states s i , calculated from s i Transfer to s j The state transition probability a ij With s i The cumulative path probability δ at time t-1 t -1(s i ). For example, at time t-1, if the path probability of the NeutralPositive state is 0.4 and the probability of transitioning from NeutralPositive to HighPositive is 0.3, then the score of this transition path is 0.12. The system calculates such scores for all predecessor states, selects the one with the highest score as the optimal predecessor, and compares this maximum score with the target state s. j Generate the current observation vector o t The emission probability b j (o t ) and multiply it to get the arrival state s j The maximum path probability δ t (s j ). At the same time, the system records the path ψ t (s j ) stores the index of the optimal predecessor state in preparation for subsequent backtracking. This recursive process continues. As the interaction deepens, the system continuously updates the optimal path probability for each state, dynamically tracking the evolution of user emotions.
[0034] After processing the observation vector at the current time T, the system enters the termination phase. At this point, the four emotional states have their own path probabilities δ T, the system compares these four probability values and selects the largest one as the optimal termination state q T * For example, if the path probability for the High Negative state is 0.45 at the current moment, while the probabilities for the other three states are 0.25, 0.20, and 0.10, respectively, the system determines that the current user is most likely in the High Negative state. This judgment is based not only on the current moment's performance but, more importantly, takes into account the emotional evolution trajectory throughout the interaction history.
[0035] After determining the optimal termination state, the system enters the backtracking phase and gradually reconstructs the complete emotional state path from the current moment forward through the previously recorded optimal precursor information ψ. T * Initially, the system searches for ψ T The optimal precursor state q recorded in T-1 * , then continue to search for q T-1 * The optimal front drive q T-2 * , and repeat until we trace back to the first moment q1 * The final complete path Q*={q1 * , q2 * , ..., q T * This reveals the complete evolution of user emotions. For example, a typical emotional path might be: NeutralPositive, HighPositive, HighPositive, NeutralPositive, NeutralNegative, HighNegative. This indicates that the user starts out in a calm and positive state, experiences a brief upswing, then gradually turns negative, ultimately falling into a highly negative state.
[0036] It's important to note that the advantage of the Viterbi algorithm-based inference method is that it doesn't judge emotions at each moment in isolation, but rather comprehensively considers the entire observation sequence to arrive at a globally optimal solution. Even if an observation at a particular moment is ambiguous (for example, a user sends a neutral "oh"), the system can make a reasonable inference based on the emotional context of the preceding and following sentences. Furthermore, because the model parameters are trained based on extensive historical data and incorporate common emotional transition patterns across the user community, the inference results are highly reliable. The resulting optimal emotional state path not only drives the current virtual pet's performance but is also stored for updating the user profile.
[0037] S103, extracting the emotional state at the current moment from the optimal emotional state path, and driving the mimicry performance of the virtual pet based on the emotional state at the current moment, and displaying it on the interactive interface.
[0038] In the 1v1 scenario of social interaction applications, the specific process of extracting the current emotional state from the optimal emotional state path is to derive the hidden state sequence Q through the Viterbi algorithm. * , where q1 corresponding to the current time step T is * * is the current emotional state, such as High Positive, Neutral Positive, Neutral Negative, or High Negative. This state is calculated based on the maximum probability path of historical and real-time multimodal observation vector sequences. When driving the mimicry of the virtual pet based on the extracted emotional state, a mapping relationship between the emotional state and the pet's actions is first established: when a highly positive emotional state is recognized, the virtual pet performs cheerful actions, including jumping, glowing, and spinning around the user to enhance the vitality of the interaction; when a neutral positive emotional state is recognized, the virtual pet performs gentle actions, including standing and gently wagging its tail, to maintain a peaceful companionship atmosphere; when a neutral negative emotional state is recognized, the virtual pet performs depressed actions, including curling up, napping, and blinking, to convey understanding and empathy; when a highly negative emotional state is recognized, the virtual pet performs sad actions, including lying down, facing away from the user, and emitting a mournful animation, to avoid further irritating the user. Through this mapping, the system converts abstract emotion predictions into visual pet behaviors and displays them in real time on the interactive interface, such as rendering corresponding animation effects and sound effects in the virtual pet area of the interactive window, thereby enhancing the user's emotional connection.
[0039] To further optimize the naturalness and adaptability of mimicry, the system dynamically adjusts action execution based on the confidence level of the emotional state. Specifically, the system compares the path probability of the current emotional state with the maximum path probability of all states to generate a confidence ratio. A higher confidence ratio indicates greater certainty in the system's judgment of the current emotion, and accordingly increases the probability and intensity of the corresponding action. For example, when the confidence ratio exceeds 0.8, the system increases the frequency of the action and the brightness of the visual effects to avoid misleading performance under low confidence levels. Furthermore, the system adjusts action intensity based on the duration of the emotion. The system monitors the continuous duration of the same emotional state. When the duration is short (e.g., less than 30 seconds), the action intensity is maintained at a base level. As the duration increases, the amplitude and frequency of the action are gradually increased, such as increasing the height of a jump or the speed of a tail wag. However, a cap is set to prevent excessive performance. The reference duration is set to 60 seconds. After the duration exceeds twice the reference duration, the action intensity is no longer increased to prevent user fatigue or disruption to the normal rhythm of interaction. This system combines the probabilistic output of the HMM model with time-series monitoring to ensure that the virtual pet's performance is dynamically linked to the user's emotions.
[0040] In one possible implementation of the present invention, the training process of the hidden Markov model is: First, a large amount of historical interaction data of Class A users and Class B users is collected from the target community. After preprocessing and feature extraction, these data are formed into a sequence of observation vectors for training. Each sequence records the dynamic changes of user emotions in a complete interaction. Based on these historical data, the system constructs a hidden Markov model λ=(S,π,A,B), where the hidden state set S is defined as four emotional states: HighPositive (enthusiastic and happy), NeutralPositive (calm and friendly), NeutralNegative (indifferent and tired), and HighNegative (irritable and angry). Although these states cannot be directly observed, they can fully cover the user's emotional spectrum during the interaction process. The initial state probability distribution π=[π1,π2,π3,π4] represents the prior probability that the user is in various emotional states at the beginning of the interaction. It is obtained by statistically analyzing the opening emotional distribution of users in historical data. For example, if most users are in the NeutralPositive state when they start the interaction, then π2 will be relatively high. The state transition probability matrix A is a 4×4 matrix, where the element a ij Indicates that from state s i Transfer to state s j The probability of transferring from HighPositive to NeutralPositive is usually higher than that of transferring directly to HighNegative, which reflects the gradual change of emotions.
[0041] The emission probability matrix B describes the probability distribution of generating an observation vector under a specific emotional state. Since the observation vector is a five-dimensional continuous vector containing text emotion score, voice emotion score, picture emotion index, gift value and reply delay, the system uses multi-dimensional Gaussian distribution for modeling. j , whose emission probability is defined as a five-dimensional Gaussian distribution composed of the mean vector μj and the covariance matrix Σ j Completely determined. The mean vector μ j =[μ j1 ,μ j2 ,μ j3 ,μ j4 ,μ j5 ] describes the typical values of each feature in this emotional state. For example, the mean vector of the HighPositive state may be [0.8, 0.7, 3.5, 0.6, 0.2], which means that the text and voice emotion scores are high, the image emotion index tends to be positive, the probability of gift giving is high, and the reply delay is short. j It is a 5×5 symmetric positive definite matrix that describes the correlation and variation between features. The diagonal elements reflect the variance of each feature, and the off-diagonal elements reflect the covariance between features. The calculation formula of the emission probability includes the normalization constant and exponential terms , the former ensures that the integral of the probability density function is 1, and the latter measures the degree of deviation between the observation vector and the state center through the Mahalanobis distance.
[0042] The model training uses the Baum-Welch algorithm, which is a specific application of the expectation maximization (EM) algorithm in the hidden Markov model. It maximizes the likelihood probability of the observation sequence through iterative optimization. In the E step (expectation step), the system first calculates the forward probability , which means that from the beginning of the sequence to time t in state s i And observe that some sequences {o1,o2,...,o t The joint probability of} is calculated from front to back through the recursive relationship: Initialize, then Recursion. Calculate the backward probability at the same time , indicating that from time t in state s i From the beginning to the end of the sequence, the remaining sequence {o t+1 ,o t+2 ,...,o T}, by recursively working from back to front: =1 initialize, then Recursion. Based on the forward and backward probabilities, the system calculates the posterior probability of the state , which means that given the entire observation sequence, time t is in state si This probability combines past and future information and is more accurate than simple forward or backward reasoning.
[0043] In the M step (maximization step), the system updates the model parameters based on the posterior probability calculated in the E step. The initial state probability is updated to , that is, the posterior probability of each state at the first moment is used as the new initial probability estimate. The update of the state transition probability requires first calculating the joint posterior probability of the state transition , indicating that time t is in state s i And at time t+1 it is in state s j The probability of Update, the numerator is all the moments from s i Transfer to s j The expected number of times, the denominator is the expected total time in state si, and the ratio is the maximum likelihood estimate of the transition probability. In the update of the emission probability parameter, the mean vector is updated as , which is the observation vector at all times in state s j The posterior probability of is weighted averaged, reflecting the central position of the observation vector in this state.
[0044] The entire training process is carried out through repeated iterations of the E-step and the M-step. Each iteration will improve the log-likelihood probability log P(O|λ) of the observation sequence. The system monitors the change in likelihood probability between two adjacent iterations. When the change is less than a preset threshold (e.g., 10 -6 ) or reaches the maximum number of iterations (such as 100). After the training is completed, the model parameter λ * This model captures the emotional transition patterns and expression characteristics of user groups, accurately describing how behavioral sequences correspond to emotional states and how transitions between emotional states occur. This trained model is deployed to online systems to infer the emotional states of new users in real time.
[0045] In a possible implementation of the present invention, the weight of the multimodal data is also adaptively adjusted. Specifically: The system discovered significant differences in the level of emotional expression during interaction across different modal data. In some periods, users may primarily express their emotions through text, while in other periods they rely more on voice or emojis. Therefore, the weights of each modality need to be dynamically adjusted to improve the accuracy of emotion prediction. During operation, the system continuously monitors the emotional changes in each modal data and assesses the level of emotional expression in each modality by setting a sliding time window. This time window is typically set to 3 to 5 time steps, which allows it to capture short-term emotional trends without sacrificing real-time performance due to an overly large window.
[0046] For each modality, the system calculates the difference in sentiment scores between adjacent moments within a time window to quantify the magnitude of sentiment change. For the text modality, for example, if the sequence of text sentiment scores over five consecutive time steps is [0.2, 0.3, 0.7, 0.8, 0.6], then the difference between adjacent moments is [0.1, 0.4, 0.1, 0.2]. The absolute values of these differences reflect the magnitude of sentiment fluctuation. The system averages the absolute values of all differences within the window to obtain the sentiment change rate for that modality: ΔEtext = (|0.1| + |0.4| + |0.1| + |0.2|) / 4 = 0.2. The same method is applied to other modalities, such as voice, images, gifting, and reply time, to calculate their respective sentiment change rates. A higher sentiment change rate indicates that the modality is more actively conveying sentiment information during the current time period. For example, if a user frequently switches between emojis or their voice fluctuates significantly, the change rate of the corresponding modality will increase significantly.
[0047] Based on the calculated sentiment change rate, the system dynamically adjusts the weight of each modality. When the sentiment change rate of a modality is high, it indicates that users are actively expressing emotions through that channel, and the system accordingly increases the weight of that modality. Conversely, when the change rate is low, it indicates that the modality currently conveys limited emotional information, and its weight is appropriately reduced. Weight updates use a gradual strategy to avoid drastic adjustments that could lead to system instability. The specific update formula is w_mode^(t+1)=w_mode^t+α·sign(ΔE_mode-threshold)·|ΔE_mode|^β, where the learning rate α controls the update amplitude and is typically set between 0.01 and 0.1 in practice. A smaller learning rate ensures smooth weight changes. For example, if the text modality's change rate is 0.3, which is higher than the average threshold of 0.15, and the current weight is 0.2, and the learning rate is 0.05, the updated weight is approximately 0.2+0.05×0.3=0.215. The system also introduces a nonlinear adjustment factor β (usually between 0.5 and 1) so that modes with particularly high rates of change can obtain more significant weight increases, while modes with rates of change slightly above the threshold only receive a moderate weight increase.
[0048] After the weight update is completed, the system recalculates the observation vector using the new weights. The original observation vector contains five components [S_text, S_voice, S_image, G_score, R_delay], which correspond to the text sentiment score, voice sentiment score, image sentiment index, gift value score, and reply delay time, respectively. The system multiplies each component by its corresponding updated weight. For example, the weighted text features are w_text×S_text, the voice features are w_voice×S_voice, and so on. This weighting mechanism allows the most active emotional expression channels to occupy a larger proportion in the comprehensive observation vector. For example, during a heated argument, the weights of text and voice will automatically increase, while during a warm interaction, the weights of emoticons and gift-giving behaviors may be higher.
[0049] To ensure numerical stability and comparability of observation vectors at different times, the system normalizes the weighted observation vectors. Normalization uses the L2 norm, which is to calculate the Euclidean length of the vector and then divide each component of the vector by this length so that the normalized vector length is 1. For example, if the weighted observation vector is [0.4, 0.3, 0.6, 0.2, 0.1], its L2 norm is , the normalized vector is approximately [0.493, 0.369, 0.739, 0.246, 0.123]. Normalization eliminates the vector scale differences caused by weight changes at different times, allowing the hidden Markov model to infer emotional states within a unified numerical range.
[0050] Through this dynamic weight adjustment mechanism, the system can adaptively capture the user's primary emotional expressions at different stages of interaction. In actual applications, the system has observed that this mechanism significantly improves the accuracy of emotion prediction, especially in scenarios where the user's emotional expression method changes, such as switching from pure text interaction to voice call, or from normal conversation to emoji battles. The system can promptly adjust the importance weight of each modality, ensuring that the virtual pet's performance always syncs with the user's true emotions, thereby enhancing the user's emotional resonance and interactive experience.
[0051] See also Figure 2 A second embodiment of the present invention provides an apparatus for improving retention through interactive scenario prediction, comprising: A vector sequence generation unit 201 is configured to obtain a current multimodal interaction data stream of Class A users and Class B users in a target community, preprocess the current multimodal interaction data stream, and perform feature extraction on the preprocessed current multimodal interaction data stream to generate a real-time multidimensional observation vector sequence, wherein the current multimodal interaction data stream includes text messages, voice messages, picture messages, and user behavior data; An emotional state path generation unit 202 is configured to input the real-time multidimensional observation vector sequence into a pre-trained hidden Markov model and use a Viterbi algorithm to infer the optimal emotional state path at the current moment. The pre-trained hidden Markov model is trained based on the historical multi-dimensional observation vector sequences of class A users and class B users. The pet driving unit 203 is used to extract the emotional state at the current moment from the optimal emotional state path, and drive the mimicry performance of the virtual pet based on the emotional state at the current moment, and display it on the interactive interface.
[0052] A third embodiment of the present invention provides a device for improving retention through interactive scenario prediction, including a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement a method for improving retention through interactive scenario prediction as described in any one of the above items.
[0053] A fourth embodiment of the present invention provides a computer-readable storage medium, characterized in that it stores a computer program, and the computer program can be executed by a processor of the device where the computer-readable storage medium is located to implement a method for improving retention through interactive scenario prediction as described in any one of the above items.
[0054] The present invention provides a method, apparatus, device, and storage medium for improving user retention through interactive scenario prediction. This method captures the current multimodal interaction data stream between users in an interactive scenario, performs preprocessing such as noise filtering, frame segmentation, size normalization, and temporal aggregation, and extracts emotional features to generate a real-time multidimensional observation vector sequence. This sequence captures the complementarity and temporal dependencies of multimodal information, overcoming the limitations of single-modality static analysis. The observation vector sequence is input into a pretrained Hidden Markov Model (HMM), which is trained based on historical sequences and defines a set of hidden states, initial probabilities, a transition matrix, and emission probabilities. Viterbi algorithm inference is used to obtain the optimal emotional state path at the current moment, enabling probabilistic modeling and dynamic prediction of emotional transition uncertainty. The current emotional state is extracted from the path and displayed on the interactive interface as a virtual pet's mimicry, creating an intuitive feedback mechanism that strengthens emotional connection and improves user retention.
[0055] For example, the computer programs described in the third and fourth embodiments of the present invention may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the device for implementing a battery self-modeling system based on cyclic charge and discharge. For example, the device described in the second embodiment of the present invention.
[0056] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the downhill assist method for a pure electric vehicle, and utilizes various interfaces and lines to connect the various parts of the method for implementing the downhill assist method for a pure electric vehicle.
[0057] The memory can be used to store the computer programs and / or modules. The processor implements various functions of a downhill assist method for a pure electric vehicle by running or executing the computer programs and / or modules stored in the memory and accessing data stored in the memory. The memory may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or a text conversion function); the data storage area may store data generated based on the use of the mobile phone (such as audio data and text message data). Furthermore, the memory may include high-speed random access memory (RAM) and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.
[0058] If the implementation module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.
[0059] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.
[0060] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.
Claims
1. A method for improving retention through interactive scenario prediction, characterized in that: include: Obtaining current multimodal interaction data streams of Class A users and Class B users in a target community, preprocessing the current multimodal interaction data streams, and performing feature extraction on the preprocessed current multimodal interaction data streams to generate a real-time multidimensional observation vector sequence, wherein the current multimodal interaction data streams include text messages, voice messages, picture messages, and user behavior data; Inputting the real-time multidimensional observation vector sequence into a pre-trained hidden Markov model and inferring the optimal emotional state path at the current moment through the Viterbi algorithm, wherein the pre-trained hidden Markov model is trained based on the historical multidimensional observation vector sequences of class A users and class B users; The emotional state at the current moment is extracted from the optimal emotional state path, and the mimicry performance of the virtual pet is driven based on the emotional state at the current moment and displayed on the interactive interface.
2. The method for improving retention through interactive scenario prediction according to claim 1, characterized in that: The pretreatment process includes: Noise filtering and stop word removal are performed on the text message to obtain the purified text. The processing process is as follows: filtered =Filter(T raw ), where T raw is the original text message, T filtered is the purified text after filtering; The voice message is framed and processed as follows: X=Frame-Split(Vraw)=[X1,X2,...,X n ]; where Vraw is the original speech signal, X i is the i-th speech frame, n is the total number of speech frames; Normalize the size and pixels of the image message. The expression is: I pre =Normalize(Resize(I raw ,size),μ,σ); where I raw is the original image, size is the target size, μ is the pixel mean, and σ is the pixel standard deviation; Aggregate user behavior data in time windows, and the expression is: agg =Aggregate(B raw ,Δt), where B raw is the original behavior data sequence, Δt is the time window size, B agg is the behavioral feature after aggregation, Resize is the size normalization, and Normalize is the pixel normalization operation.
3. The method for improving retention through interactive scenario prediction according to claim 2, characterized in that: The feature extraction is performed on the pre-processed current multimodal interaction data stream to generate a real-time multi-dimensional observation vector sequence, specifically: The text sentiment feature is extracted from the purified text, and its expression is: S text =BERT(Word-Segment(T filtered )); Among them, Word-Segment is the word segmentation function, and BERT is the pre-trained language model S text is the sentiment score of the text; The speech emotion feature is extracted from the speech frame sequence, and its expression is: in, is the fundamental frequency of the i-th frame, is the energy of the i-th frame, is the Mel frequency cepstral coefficient of the i-th frame, 、 、 is the weight coefficient; The emotional features of the image are extracted from the preprocessed image, and its expression is: S image =ResNet(I pre ); ResNet is a residual neural network, S image is the picture sentiment index; The behavioral features are extracted from the aggregated user behavior data, and its expression is: S behavior =[G score ,R delay ]; among them, G score is the normalized gift value score, R delay is the normalized response delay time; Combine to generate the observation vector o at time t t =[S text ,S voice ,S image ,G score ,R delay ] T , the observation vectors of consecutive time steps are combined into a real-time multi-dimensional observation vector sequence.
4. The method for improving retention through interactive scenario prediction according to claim 2, characterized in that: The training process of the hidden Markov model is: Define the hidden Markov model, λ=(S,π,A,B), where S is the hidden state set, π is the initial state probability distribution, A is the state transition probability matrix, and B is the emission probability matrix. Multidimensional Gaussian distribution is used for modeling: Where d is the dimension of the observation vector, Status The mean vector of is the covariance matrix, is observed at time t When in state probability; The Baum-Welch algorithm is used to iteratively optimize the model parameters, where the forward probability is calculated in the E step and backward probability , and then calculate the state posterior probability: Update the model parameters in M steps: Update the initial state probability: Update state transition probability: Update the mean vector of emission probabilities: in, From state s at time t i Transfer to s j The joint probability of, T is the length of the training sequence, is the updated initial state probability, is the updated slave state s i Transfer to state s j The transition probability, The updated state s j The mean vector of 、 They are in state s at time t i and s j The posterior probability of .
5. The method for improving retention through interactive scenario prediction according to claim 1, characterized in that: The real-time multi-dimensional observation vector sequence is input into the pre-trained hidden Markov model, and the optimal emotional state path at the current moment is obtained through Viterbi algorithm reasoning, specifically: In the initialization phase: for each possible initial state, calculate the product of its initial probability and the probability of the state generating the first observation vector, obtain the path probability of each state at the first moment, and initialize the path record to empty; In the recursive stage: starting from the second moment until the current moment, for each possible target state, traverse all possible predecessor states, calculate the product of the probability of transitioning from the predecessor state to the target state and the cumulative path probability of the predecessor state, select the predecessor state that maximizes this product as the optimal predecessor, and multiply the optimal predecessor with the probability of generating the current observation vector of the target state to obtain the maximum path probability to the target state. At the same time, record the optimal predecessor state of each state for path backtracking; In the termination phase: at the current moment, compare the path probabilities of all states and select the state with the highest probability as the optimal state at the current moment; In the retrospective stage: starting from the optimal state at the current moment, according to the recorded optimal predecessor state information, gradually trace back to the first moment to obtain the complete optimal emotional state path.
6. The method for improving retention through interactive scenario prediction according to claim 1, characterized in that: It also includes adaptive adjustment of the weights of multimodal data, specifically: Calculate the difference in sentiment scores between adjacent moments of each modal data within the set time window, and take the average as the sentiment change rate of the modality; The modal weights are adjusted according to the emotion change rate. The weights of the modalities with high change rates are increased, and the weights of the modalities with low change rates are reduced. The weight update amplitude is controlled by the learning rate. The eigenvalues of each modality are multiplied by the updated weights to form a new observation vector, which is then normalized.
7. A device for improving retention through interactive scenario prediction, characterized in that: include: a vector sequence generation unit, configured to obtain current multimodal interaction data streams of Class A users and Class B users in a target community, preprocess the current multimodal interaction data streams, and perform feature extraction on the preprocessed current multimodal interaction data streams to generate a real-time multidimensional observation vector sequence, wherein the current multimodal interaction data streams include text messages, voice messages, picture messages, and user behavior data; an emotional state path generation unit, configured to input the real-time multidimensional observation vector sequence into a pre-trained hidden Markov model and obtain the optimal emotional state path at the current moment through Viterbi algorithm inference, wherein the pre-trained hidden Markov model is trained based on the historical multi-dimensional observation vector sequences of class A users and class B users; The pet driving unit is used to extract the emotional state at the current moment from the optimal emotional state path, and drive the mimicry performance of the virtual pet based on the emotional state at the current moment, and display it on the interactive interface.
8. A device for improving retention through interactive scenario prediction, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement a method for improving retention through interactive scenario prediction as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that A computer program is stored, and the computer program can be executed by a processor of the device where the computer-readable storage medium is located to implement a method for improving retention through interactive scenario prediction as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Generative AI emotion propagation prediction and guidance large model construction method and system
CN119047512A
Emotion analysis method and system based on multi-modal feature fusion and SVM classification
CN119312813A
Digital human control method and device based on multiple modes and electronic equipment
CN119441403A
Customized virtual pet interactive simulation method and system
CN120258034A
Intelligent health steward interaction system and method based on multi-modal sentiment analysis
CN120523323A
Cited By
Carton production line monitoring method and system
CN121278608A
