A scene-adaptive digital human behavior prediction method, device and storage medium

Abstract

The present invention proposes a scene-adapted digital human behavior prediction device and storage medium, and the method includes: the present invention involves collecting multimodal user interaction data such as text, voice, and image, and preprocessing. A multimodal feature vector is extracted through a pre-trained multimodal feature extraction model. Then, the pre-trained scene recognition model and behavior prediction model are used to predict the behavior of the digital human in different scenarios. In order to improve the accuracy of the prediction, the present invention adopts transfer learning technology to optimize the model parameters or structure based on the differences between historical and current data. At the same time, user feedback is collected in real time, and the prediction model is iteratively updated to meet the needs of different user groups. This adaptive method not only improves the accuracy of digital human behavior prediction, but also enhances its understanding and adaptability to different scenarios, thereby providing more intelligent and humane services.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a scene-adaptive digital human behavior prediction method, device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI), digital humans have become a hot topic of research. These virtual humans are constructed using digital technology, simulating human characteristics and existing in a non-physical world. They leverage natural language processing, facial recognition, and voice recognition technologies to enable intelligent interaction with users. They are widely used in fields such as virtual livestream customer service and online education.

[0003] Currently, existing digital human behavior prediction technologies primarily include rule-based models and traditional machine learning models. Rule-based models often require a large number of rules to be manually set. For example, in a virtual anchor scenario, response rules must be set for different types of audience questions. However, this approach suffers from low prediction accuracy when faced with complex and changing user behavior, as the rules are difficult to enumerate exhaustively. Traditional machine learning models, such as decision tree models, may exhibit deviations in predicting digital human behavior in customer service scenarios due to a lack of understanding of the specific needs of different customer groups. Furthermore, most of these existing technologies are developed based on single scenarios or simple behavioral patterns, lacking a deep understanding of and adaptability to different scenarios. This results in low accuracy in predicting digital human behavior in practical applications, making it difficult to provide intelligent and personalized services. Summary of the Invention

[0004] The present invention aims to at least solve the technical problem of low prediction accuracy in the prior art, and particularly innovatively proposes a scene-adaptive digital human behavior prediction method, device and storage medium.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a scene-adaptive digital human behavior prediction method, the method comprising:

[0006] S1. Collecting multimodal user interaction data and preprocessing the multimodal user interaction data; the multimodal user interaction data includes text data, voice data, and image data;

[0007] S2. Obtaining a multimodal feature vector in the multimodal user interaction data based on a pre-trained multimodal feature extraction model;

[0008] S3. Obtaining a scene recognition result using a pre-trained scene recognition model based on the multimodal feature vector, and predicting the digital human's behavior using a behavior prediction model based on the scene recognition result to obtain a digital human behavior prediction result;

[0009] S4. Based on the digital human behavior prediction results, a transfer learning mechanism is introduced to optimize the behavior prediction model:

[0010] S5. Collect user feedback information in real time, and iteratively update the prediction model based on the user feedback information.

[0011] As an optional embodiment of the present invention, optionally, obtaining the multimodal feature vector in the multimodal user interaction data in step S2 includes:

[0012] S101, extracting semantic features from the text data based on a pre-trained BERT model;

[0013] S102, converting the speech data into text based on a speech recognition model, and extracting emotional features;

[0014] S103, extracting visual features from the image data based on a face recognition model;

[0015] S104: fusing the semantic features, emotional features, and visual features based on an attention mechanism to obtain a multimodal feature vector.

[0016] As an optional embodiment of the present invention, optionally, the expression for obtaining the multimodal feature vector in step S104 is:

[0017]

[0018]

[0019] in, Represents the semantic feature vector after mapping, W T Represents the weight matrix of semantic features, T represents the semantic features, b T The bias vector representing the semantic feature, Represents the mapped emotional feature vector, W A Represents the weight matrix of emotional features, A represents emotional features, b A The bias vector representing the sentiment feature, Represents the mapped visual feature vector, W V Represents the weight matrix of visual features, V represents visual features, b V The bias vector representing the visual feature, α T Represents the attention weight of semantic features, exp() represents the exponential function, and Both represent weight vectors, b′ T , b′ A and b′ V express and The corresponding bias term, F represents the fused multimodal feature vector.

[0020] As an optional embodiment of the present invention, optionally, obtaining the digital human behavior prediction result in step S3 includes:

[0021] S301, collecting historical behavior data of digital humans, and sorting the historical behavior data according to time series to obtain a time series data set;

[0022] S302, extracting digital human behavior-related features from the time series data set;

[0023] S303, obtaining the digital human behavior trend based on the behavior-related characteristics;

[0024] S304: Based on the behavior trend and scene recognition results, the trained behavior prediction model is used to predict the behavior of the digital human, and a prediction probability distribution of the digital human behavior is obtained;

[0025] S305: Based on the digital human behavior prediction probability distribution, select the behavior with the highest probability as the digital human behavior prediction result.

[0026] As an optional embodiment of the present invention, optionally, the expression for obtaining the digital human behavior trend based on the behavior-related features is:

[0027] D t =η·M t +β·S t +γ·E t

[0028] M t =λ·(Y t -Y t-1 )+(1-λ)·M t-1

[0029]

[0030] Among them, D t represents the behavior trend, η, β and γ are weight coefficients, M t represents the momentum of the behavior change at time t, S t Indicates seasonal changes in behavior, E t represents the impact of external factors on behavior, λ represents the momentum smoothing factor, and Y t Indicates the digital human behavior label at time t, Y t-1 The digital human behavior label at time t-1, M t-1 The momentum of the behavior change at time t-1, K represents the length of the seasonal cycle, φ k represents the seasonal weight coefficient, Y t-k·Prepresents the behavior label at time tk·P, P represents the basic unit of seasonal cycle, N represents the number of external influencing factors, θ i Indicates the weight coefficient of the i-th external influencing factor, E i,t Represents the value of the i-th external influencing factor at time t.

[0031] As an optional embodiment of the present invention, optionally, in step S304, the expression for the predicted probability distribution of digital human behavior is obtained as follows:

[0032] P t =f(D t ,H t )

[0033] P t =softmax(W o ·h t +b o )

[0034] h t =σ(W h ·[D t ;H t ]+b h )

[0035] Among them, P t represents the probability distribution of various behaviors that the digital human may take at time t, f() represents the behavior prediction model, and D t Indicates behavioral trend, H t represents the scene recognition result at time t, softmax() represents the activation function, W o represents the weight matrix of the output layer, h t represents the hidden layer output at time t, b o Represents the bias vector of the output layer, σ() represents the ReLU function, W h represents the weight matrix of the hidden layer, [;] represents the concatenation operation, b h Bias vector for the hidden layer.

[0036] As an optional embodiment of the present invention, optionally, introducing a transfer learning mechanism to optimize the behavior prediction model in step S4 includes:

[0037] S401. Obtaining a behavior prediction model performance deviation based on a difference between historical prediction data and current prediction data;

[0038] S402: Select a transfer learning strategy based on the performance deviation;

[0039] S403: Apply the transfer learning strategy to adjust the parameters or structure of the behavior prediction model;

[0040] S404, evaluating the performance of the behavior prediction model after transfer learning, and determining whether the performance of the behavior prediction model has improved based on the evaluation result;

[0041] If so, save the model parameters after transfer learning;

[0042] If not, return to step S402 to reselect the transfer learning strategy until the optimal transfer learning strategy is found.

[0043] As an optional embodiment of the present invention, optionally, in step S5, iteratively updating the prediction model based on the user feedback information includes:

[0044] S501, collecting user behavior feedback data during actual use, wherein the behavior feedback data includes user satisfaction, number of behavior corrections, and behavior prediction accuracy;

[0045] S502: Calculating a performance indicator of the behavior prediction model based on the behavior feedback data;

[0046] S503, determining whether the performance indicator meets a preset threshold;

[0047] If satisfied, the current behavior prediction model parameters remain unchanged;

[0048] If not, adjust the model parameters or introduce new feature variables based on the behavioral feedback data;

[0049] S504: Repeat steps S501 to S503 until the performance indicator meets the preset conditions.

[0050] In another aspect, the present invention further provides a computer device comprising:

[0051] processor;

[0052] a memory for storing processor-executable instructions;

[0053] The processor is configured to implement a scene-adaptive digital human behavior prediction method when executing the executable instructions.

[0054] In another aspect, the present invention further provides a computer-readable storage medium comprising:

[0055] a memory having a computer program stored thereon;

[0056] A processor is used to execute the program in the memory to implement a scene-adaptive digital human behavior prediction method.

[0057] Beneficial effects of the present invention: The present invention collects multimodal user interaction data such as text data, voice data and image data, and uses a pre-trained multimodal feature extraction model to obtain the feature vectors of these data. Based on these feature vectors, the present invention further uses pre-trained scene recognition models and behavior prediction models to predict the behavior of digital humans in different scenarios. In order to improve the prediction accuracy, the present invention introduces a transfer learning mechanism, which optimizes the parameters or structure of the behavior prediction model based on the differences between historical prediction data and current prediction data, thereby improving the prediction accuracy of digital humans in practical applications. In addition, the present invention also collects user feedback information in real time and iteratively updates the prediction model to adapt to the special needs of different user groups. This scene-adapted digital human behavior prediction method not only improves the prediction accuracy of digital human behavior, but also enables digital humans to better understand and adapt to different scenarios, providing more intelligent and humane services.

[0058] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0060] Figure 1 This is a flow chart of a scene-adapted digital human behavior prediction method of the present invention. DETAILED DESCRIPTION

[0061] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0062] Example 1

[0063] like Figure 1 As shown, a scene-adaptive digital human behavior prediction method includes:

[0064] S1. Collecting multimodal user interaction data and preprocessing the multimodal user interaction data; the multimodal user interaction data includes text data, voice data, and image data;

[0065] It should be noted that to more accurately capture digital human behavior, the present invention performs cleaning, denoising, and normalization on multimodal user interaction data during the preprocessing phase. For text data, operations such as word segmentation, stop word removal, and stemming are performed; for voice data, voice enhancement, noise suppression, and voice endpoint detection are performed; and for image data, image enhancement and face detection are performed. These preprocessing steps significantly improve the efficiency and accuracy of subsequent feature extraction and model training.

[0066] S2. Obtaining a multimodal feature vector in the multimodal user interaction data based on a pre-trained multimodal feature extraction model;

[0067] It should be noted that the multimodal feature extraction model of this embodiment includes multiple sub-models, each of which focuses on extracting features from data of a specific modality. For example, there is a text feature extraction sub-model specifically for text data, which can capture key features such as keywords, phrases, and contextual information from text data. Similarly, there is a speech feature extraction sub-model for speech data, which can analyze speech characteristics such as pitch, speaking rate, and volume to understand the speaker's emotions and intentions. For image data, the image feature extraction sub-model focuses on identifying visual elements such as objects, faces, and actions in the image. These sub-models are trained and optimized using advanced technologies such as deep learning and neural networks to ensure efficient and accurate extraction of feature vectors for their respective modalities. The features extracted from different modalities by different sub-models are then fused to generate a multimodal feature vector. This vector incorporates information from multiple modalities, including text, speech, and images, and can comprehensively reflect the user's behavior and intentions when interacting with the digital human. By inputting the multimodal feature vector into the subsequent scene recognition model and behavior prediction model, the present invention can accurately predict the digital human's behavior in different scenarios.

[0068] S3. Obtaining a scene recognition result using a pre-trained scene recognition model based on the multimodal feature vector, and predicting the digital human's behavior using a behavior prediction model based on the scene recognition result to obtain a digital human behavior prediction result;

[0069] It should be noted that both the scene recognition model and the behavior prediction model are built based on a deep learning framework and are trained using a large amount of historical data. The scene recognition model can identify the specific scenarios in which users interact with digital humans, such as office, home, and outdoors, thereby providing contextual information for the behavior prediction model. The behavior prediction model then predicts the possible behaviors of the digital human in that scenario based on the scene recognition results; specific behaviors include but are not limited to greeting, asking questions, answering questions, expressing emotions, providing advice, or performing specific tasks. To improve the accuracy of behavior prediction, this embodiment also considers the impact of time factors and the external environment on the behavior of the digital human. For example, the behavior of the digital human will be different during specific time periods (such as working hours or rest time) or under specific environmental conditions (such as light brightness, noise level, etc.). Therefore, the present invention incorporates considerations of time factors and external environmental factors into the behavior prediction model, making the prediction results more consistent with actual conditions.

[0070] S4. Based on the digital human behavior prediction results, a transfer learning mechanism is introduced to optimize the behavior prediction model:

[0071] It should be noted that the transfer learning mechanism can automatically adjust the parameters or structure of the behavior prediction model according to the difference between historical prediction data and current prediction data, thereby improving the adaptability and prediction accuracy of the model. Specifically, transfer learning can be achieved in the following ways: first, model-based migration, that is, using the existing pre-trained model as a starting point, and adapting to the new prediction task by fine-tuning the model parameters; second, feature-based migration, that is, extracting effective features from the existing model and using it in the new prediction task; third, instance-based migration, that is, screening and weighting the existing data set according to the requirements of the new task to construct a data set that better meets the requirements of the new task. Through these transfer learning strategies, the present invention can achieve continuous optimization of the behavior prediction model and improve its prediction performance in different scenarios.

[0072] S5. Collect user feedback information in real time, and iteratively update the prediction model based on the user feedback information.

[0073] It should be noted that user feedback is an important basis for evaluating and improving the performance of prediction models. In practical applications, feedback information such as user satisfaction, number of behavior corrections, and behavior prediction accuracy can intuitively reflect the performance of the prediction model in real-world scenarios. Therefore, the present invention collects this feedback information in real time and iteratively updates the prediction model to continuously improve its prediction accuracy and adaptability. Specifically, the present invention first collects user behavioral feedback data from actual use, including but not limited to user satisfaction scores, number of behavior corrections, and behavior prediction accuracy. Then, based on this feedback data, the prediction model's performance metrics, such as precision, recall, and F1 score, are calculated to comprehensively evaluate the model's performance. Next, the present invention determines whether these performance metrics meet preset thresholds. If they do, the current prediction model's performance is sufficiently good, and the current model parameters can be maintained. If they do not, the model has room for improvement, and it is necessary to adjust the model parameters or introduce new feature variables based on the feedback data. During this process, the present invention leverages valuable information from user feedback, such as behavior patterns with low user satisfaction and types of behavior that are frequently corrected, to perform targeted model optimization. At the same time, new model structures and algorithms are constantly being tested to find a more optimal prediction model. Finally, the present invention repeats the aforementioned steps: collecting user feedback in real time, calculating performance metrics, determining whether performance meets preset conditions, adjusting model parameters, or introducing new feature variables, until the prediction model's performance metrics meet preset conditions or reach an optimal state. In this way, the present invention enables continuous iterative updates of the prediction model to meet the specific needs of different user groups, improving the accuracy and intelligence of digital human behavior predictions.

[0074] In summary, this embodiment proposes an innovative scenario-adaptive method for predicting digital human behavior. This method not only combines the collection and processing of multimodal user interaction data but also incorporates a pre-trained multimodal feature extraction model, a scenario recognition model, and a behavior prediction model to accurately predict digital human behavior in different scenarios. Furthermore, by introducing a transfer learning mechanism and collecting user feedback in real time, the present invention continuously optimizes the prediction model, improving its predictive performance and adaptability in different scenarios.

[0075] As an optional embodiment of the present invention, optionally, obtaining the multimodal feature vector in the multimodal user interaction data in step S2 includes:

[0076] S101, extracting semantic features from the text data based on a pre-trained BERT model;

[0077] It should be noted that the BERT model (Bidirectional Encoder Representations from Transformers) is a pre-trained deep bidirectional model that can efficiently capture the semantic features in the text by understanding the contextual information of the text data. In step S101, the BERT model is used to process the text data to extract key semantic information such as keywords, phrases, and logical relationships between sentences, providing strong support for subsequent scene recognition and behavior prediction. At the same time, the powerful generalization ability of the BERT model enables it to adapt to text data of different fields and styles, improving the accuracy and applicability of feature extraction.

[0078] S102, converting the speech data into text based on a speech recognition model, and extracting emotional features;

[0079] It should be noted that in step S102, the voice data is first input into a trained voice recognition model, which can accurately convert the voice signal into the corresponding text content. Subsequently, the natural language processing technology is used to perform sentiment analysis on the converted text to extract emotional features such as joy, sadness, anger, etc. These emotional features are of great significance for understanding the user's emotional state and predicting their behavioral responses. In the process of extracting multimodal feature vectors, the present invention also fully considers the correlation and complementarity between different modal data, and through an effective feature fusion strategy, the feature vectors from different modalities are organically combined to form a more comprehensive and accurate multimodal feature representation. This representation method not only improves the efficiency and accuracy of feature extraction, but also provides a more reliable foundation for subsequent scene recognition and behavior prediction.

[0080] S103, extracting visual features from the image data based on a face recognition model;

[0081] It should be noted that in step S103, the image data is processed using advanced facial recognition technology, accurately capturing key visual features such as facial contours, expressions, and movements. These visual features are crucial for identifying the user's identity, emotional state, and interaction intent. For example, by analyzing the user's facial expression, it is possible to determine whether they are happy, surprised, or confused, thereby predicting their behavioral responses. Similarly, by analyzing the user's body movements, such as gestures and posture, it is possible to further understand their intentions and needs. In the process of extracting multimodal feature vectors, the present invention utilizes a variety of advanced deep learning models and algorithms to ensure that useful feature information can be efficiently and accurately extracted from multimodal data. This feature information not only captures the user's behavior and intentions when interacting with the digital human, but also reflects the correlation and complementarity between different modal data. By organically integrating and representing this feature information, the present invention enables accurate prediction of the digital human's behavior in different scenarios and provides intelligent services.

[0082] S104: fusing the semantic features, emotional features, and visual features based on an attention mechanism to obtain a multimodal feature vector.

[0083] It should be noted that the attention mechanism is an effective deep learning technology that can help the model automatically focus on the information that is most important for the current task when processing complex input data. In step S104, the attention mechanism is used to fuse semantic features, emotional features, and visual features, and the weights of different features in the final decision can be dynamically adjusted to ensure that the prediction model can more accurately capture the key information when the user interacts with the digital human. Specifically, the attention mechanism assigns a corresponding attention weight to each feature based on its importance. This weight reflects the contribution of the feature to predicting the behavior of the digital human. During the fusion process, the model will give priority to features with higher attention weights and ignore those with lower scores. In this way, the attention mechanism can enable the prediction model to focus more on the information that is most critical to the current task, thereby improving the accuracy and robustness of the prediction.

[0084] As an optional embodiment of the present invention, optionally, the expression for obtaining the multimodal feature vector in step S104 is:

[0085]

[0086]

[0087] in, Represents the semantic feature vector after mapping, W T Represents the weight matrix of semantic features, T represents the semantic features, b TThe bias vector representing the semantic feature, Represents the mapped emotional feature vector, W A Represents the weight matrix of emotional features, A represents emotional features, b A The bias vector representing the sentiment feature, Represents the mapped visual feature vector, W V Represents the weight matrix of visual features, V represents visual features, b V The bias vector representing the visual feature, α T Represents the attention weight of semantic features, exp() represents the exponential function, and Both represent weight vectors, b′ T , b′ A and b′ V express and The corresponding bias term, F represents the fused multimodal feature vector, α A represents the attention weight of the emotional feature, α V represents the attention weight of the visual feature, .

[0088] It should be noted that α T +α A +α V = 1; Using the above expression, we can calculate the fused multimodal feature vector. This vector not only incorporates semantic, emotional, and visual information, but also effectively integrates and weights these three aspects through the attention mechanism. This allows the model to more accurately understand and utilize multimodal data from user interactions with the digital human during subsequent scene recognition and behavior prediction, thereby improving prediction accuracy and intelligence. Furthermore, this expression offers a degree of flexibility and scalability, allowing the weights and bias vectors of different features to be adjusted as needed to accommodate diverse scenarios and tasks.

[0089] As an optional embodiment of the present invention, optionally, obtaining the digital human behavior prediction result in step S3 includes:

[0090] S301, collecting historical behavior data of digital humans, and sorting the historical behavior data according to time series to obtain a time series data set;

[0091] It should be noted that time series datasets can reflect the behavioral patterns and regularities of digital humans at different points in time and are an important foundation for behavioral prediction. In step S301, this embodiment constructs a complete and ordered time series dataset by collecting the digital human's historical behavioral data and sorting it according to time series. This dataset contains all behavioral records of the digital human over a period of time, such as movements, expressions, and language, providing rich data support for subsequent behavioral prediction. In the process of collecting historical behavioral data, this embodiment fully considers the diversity and completeness of the data, ensuring that the dataset can fully reflect the behavioral characteristics and patterns of the digital human. At the same time, by sorting the data in time series, this embodiment further improves the orderliness and predictability of the data, providing a more reliable data foundation for subsequent behavior prediction algorithms.

[0092] S302, extracting digital human behavior-related features from the time series data set;

[0093] It should be noted that this embodiment uses advanced deep learning models such as long short-term memory (LSTM) networks to extract behavioral features related to digital humans from time series datasets. These features include, but are not limited to, movement frequency, expression changes, and language patterns, which can reflect the behavioral patterns and characteristics of digital humans in different scenarios. In the process of feature extraction, this embodiment fully utilizes the powerful learning capabilities of deep learning models to effectively process and analyze complex time series data, thereby obtaining more accurate and comprehensive behavioral feature representations. These feature representations not only provide strong support for subsequent behavioral predictions, but also help to better understand the behavioral patterns and patterns of digital humans, providing a foundation for their intelligent services.

[0094] S303, obtaining the digital human behavior trend based on the behavior-related characteristics;

[0095] It should be noted that in step S303, by conducting an in-depth analysis of the extracted behavioral features, this embodiment is able to predict the digital human's behavioral trends over the next period of time. This prediction process not only considers the digital human's current behavioral state but also incorporates its historical behavioral data, thereby achieving a comprehensive understanding and grasp of the digital human's behavioral patterns. Specifically, this embodiment utilizes advanced prediction algorithms, such as time series analysis and machine learning, to process the behavioral features to predict the digital human's possible behavioral paths and patterns. This prediction result is of great significance for identifying potential risks and opportunities in advance and optimizing digital human behavioral strategies.

[0096] S304: Based on the behavior trend and scene recognition results, the trained behavior prediction model is used to predict the behavior of the digital human, and a prediction probability distribution of the digital human behavior is obtained;

[0097] It should be noted that in step S304, this embodiment combines behavioral trends with scenario recognition results as input to the behavior prediction model. This strategy fully accounts for the differences and diversity of digital human behavior patterns in different scenarios, enabling the prediction model to more accurately capture these differences and provide behavior prediction results that are consistent with the actual situation. Specifically, by combining behavioral trends with scenario information, the model can comprehensively consider the digital human's current behavioral state, historical behavioral patterns, and the specific scenario it is in, thereby accurately predicting the digital human's future behavior. This prediction result is presented in the form of a probability distribution, reflecting the different possible behaviors the digital human may adopt and their corresponding probabilities. In this way, during subsequent service and interaction processes, the system can make corresponding preparations and adjustments in advance based on the prediction results, providing a more intelligent and personalized service experience. At the same time, this prediction result can also provide an important reference for optimizing and adjusting the digital human's behavior strategy, helping the system better understand and adapt to user needs and preferences.

[0098] S305: Based on the digital human behavior prediction probability distribution, select the behavior with the highest probability as the digital human behavior prediction result.

[0099] It should be noted that in step S305, this embodiment employs a simple yet effective strategy: selecting the behavior with the highest probability as the predicted result for the digital human behavior. This strategy is based on the fundamental principles of probability theory, which posits that, under given conditions, the event with the highest probability of occurrence is most likely to materialize. Therefore, after obtaining the predicted probability distribution for the digital human behavior, this embodiment compares the predicted probabilities of different behaviors and selects the behavior with the highest probability as the final prediction result. This result not only reflects the digital human's most likely behavior in the current scenario but also provides clear guidance for subsequent service and interaction. For example, in an intelligent customer service scenario, if the prediction results indicate that the digital human is most likely to answer a user's question, the system can prepare the corresponding response in advance to improve interaction efficiency and user experience. Similarly, in an intelligent navigation scenario, if the prediction results indicate that the digital human is most likely to choose a certain path, the system can plan the navigation route in advance to provide more intelligent service.

[0100] As an optional embodiment of the present invention, optionally, the expression for obtaining the digital human behavior trend based on the behavior-related features is:

[0101] D t =η·M t +β·S t +γ·E t

[0102] M t =λ·(Yt -Y t-1 )+(1-λ)·M t-1

[0103]

[0104] Among them, D t represents the behavior trend, η, β and γ are weight coefficients, M t represents the momentum of the behavior change at time t, S t Indicates seasonal changes in behavior, E t represents the impact of external factors on behavior, λ represents the momentum smoothing factor, and Y t Indicates the digital human behavior label at time t, Y t-1 The digital human behavior label at time t-1, M t-1 The momentum of the behavior change at time t-1, K represents the length of the seasonal cycle, φ k represents the seasonal weight coefficient, Y t-k·P represents the behavior label at time tk·P, P represents the basic unit of seasonal cycle, N represents the number of external influencing factors, θ i Indicates the weight coefficient of the i-th external influencing factor, E i,t Represents the value of the i-th external influencing factor at time t.

[0105] It's important to note that this expression comprehensively considers multiple aspects of digital human behavior, including the momentum of behavioral change, seasonal variations, and the influence of external factors. By introducing the weighting coefficients η, β, and γ, this expression balances the contributions of different factors to behavioral trends, thereby more accurately reflecting the behavioral patterns of digital humans. The momentum term captures the changing trends of behavior over time, helping to identify the persistence and direction of behavior change. The seasonal variation term accounts for cyclical patterns in behavior that may exist at different time points, such as changes in the morning and evening hours, or the difference between weekdays and weekends. The external factor term represents external events or conditions that influence digital human behavior, such as weather, holidays, and special events.

[0106] As an optional embodiment of the present invention, optionally, in step S304, the expression for the predicted probability distribution of digital human behavior is obtained as follows:

[0107] P t =f(D t ,H t )

[0108] P t =softmax(W o ·h t +b o )

[0109] h t =σ(W h ·[D t ;H t ]+b h )

[0110] Among them, P t represents the probability distribution of various behaviors that the digital human may take at time t, f() represents the behavior prediction model, and D t Indicates behavioral trend, H t represents the scene recognition result at time t, softmax() represents the activation function, W o represents the weight matrix of the output layer, h t represents the hidden layer output at time t, b o Represents the bias vector of the output layer, σ() represents the ReLU function, W h represents the weight matrix of the hidden layer, [;] represents the concatenation operation, b h Bias vector for the hidden layer.

[0111] It's important to note that this expression is a core component of the behavior prediction model. It defines how to calculate the probability distribution of various possible behaviors of the digital human from the input behavioral trends and scene recognition results. In this expression, the behavior prediction model receives behavioral trends and scene recognition results as input and, through a series of calculations and transformations, ultimately outputs a probability distribution. This probability distribution reflects the different behaviors the digital human can take in a given scenario and their corresponding probabilities. Parameters such as the activation function, the output layer's weight matrix, and the bias vector jointly determine the model's output and predictive performance. By adjusting these parameters, the model's predictive accuracy and generalization ability can be optimized, making it more adaptable to the needs of different scenarios and tasks. Furthermore, through multiple layers of nonlinear transformations, the model extracts useful features and information from the raw input, and uses this information to make accurate predictions and decisions.

[0112] As an optional embodiment of the present invention, optionally, introducing a transfer learning mechanism to optimize the behavior prediction model in step S4 includes:

[0113] S401. Obtaining a behavior prediction model performance deviation based on a difference between historical prediction data and current prediction data;

[0114] It should be noted that in step S401, a transfer learning mechanism is introduced to optimize the performance of the behavior prediction model. Specifically, this embodiment first analyzes the differences between historical prediction data and current prediction data. These differences may arise from a variety of factors, such as changes in digital human behavior patterns, increased scene complexity, or changes in the external environment. By conducting in-depth analysis of these differences, this embodiment is able to identify deviations in the performance of the behavior prediction model. This step is the foundation of the application of the transfer learning mechanism, and it helps determine the direction and focus of model optimization.

[0115] S402: Select a transfer learning strategy based on the performance deviation;

[0116] It should be noted that in step S402, based on the identified performance deviation, this embodiment selects a suitable transfer learning strategy to optimize the behavior prediction model. The choice of transfer learning strategy depends on the specific type and degree of the deviation, as well as the similarity and correlation between the current task and the previous task. For example, if the deviation mainly comes from changes in the behavior pattern of the digital human, a strategy based on feature transfer can be selected to transfer useful features learned in the previous task to the current task to improve the adaptability and accuracy of the model. If the deviation is caused by the increase in scene complexity, a strategy based on model transfer can be selected to use the model trained in the previous task as a starting point and adapt it to the new complex scene through fine-tuning or expansion. In addition, multiple transfer learning strategies can be combined to achieve more comprehensive model optimization.

[0117] S403: Apply the transfer learning strategy to adjust the parameters or structure of the behavior prediction model;

[0118] It should be noted that in step S403, the transfer learning strategy is specifically applied to the optimization process of the behavior prediction model. This step aims to correct performance deviations and improve the prediction accuracy of the model by adjusting the parameters or structure of the model. Specifically, according to the selected transfer learning strategy, this embodiment makes corresponding adjustments to the behavior prediction model. These adjustments include modifying the model's weight matrix, bias vector and other parameters, or changing the model's structure, such as increasing or decreasing the number of layers, changing the type of layer or the connection method, etc. Through these adjustments, the model can better adapt to the needs of the current task and reduce the performance degradation caused by the difference between historical data and current data. In addition, the application of the transfer learning strategy also helps to improve the generalization ability of the model, enabling it to better handle unseen scenarios and data, thereby further improving the performance and reliability of digital human behavior prediction.

[0119] S404, evaluating the performance of the behavior prediction model after transfer learning, and determining whether the performance of the behavior prediction model has improved based on the evaluation result;

[0120] If so, save the model parameters after transfer learning;

[0121] If not, return to step S402 to reselect the transfer learning strategy until the optimal transfer learning strategy is found.

[0122] It should be noted that in step S404, a comprehensive evaluation of the performance of the behavior prediction model after transfer learning is performed. This evaluation process is intended to verify the effectiveness of the transfer learning strategy and determine whether model performance has been improved. Evaluation metrics include prediction accuracy, generalization ability, computational efficiency, and other aspects to ensure that the model meets actual requirements across multiple dimensions. If the evaluation results show that model performance has improved, the model parameters after transfer learning will be saved for subsequent use. These parameters reflect the optimization results of the model during the transfer learning process and ensure that the model performs better on new tasks and data. However, if the evaluation results show that model performance has not improved, or the improvement does not meet expectations, this embodiment will return to step S402 and reselect the transfer learning strategy. This iterative process will continue until the optimal transfer learning strategy is found. In this way, this embodiment can continuously optimize the performance of the behavior prediction model, ensuring that it can adapt to changing task and data requirements. In addition, throughout the entire transfer learning process, this embodiment also focuses on maintaining the stability and reliability of the model. When adjusting model parameters or structure, the robustness and generalization ability of the model are fully considered to avoid performance degradation due to overfitting or underfitting.

[0123] As an optional embodiment of the present invention, optionally, in step S5, iteratively updating the prediction model based on the user feedback information includes:

[0124] S501, collecting user behavior feedback data during actual use, wherein the behavior feedback data includes user satisfaction, number of behavior corrections, and behavior prediction accuracy;

[0125] It should be noted that in step S501, user behavioral feedback data from actual usage is collected and used to iteratively update the prediction model. This behavioral feedback data covers multiple aspects, including user satisfaction, number of behavioral corrections, and behavior prediction accuracy. User satisfaction reflects the user's acceptance and satisfaction with the model's prediction results and is an important indicator for evaluating model performance. The number of behavioral corrections records the number of times users corrected a model prediction error, which helps identify blind spots and deficiencies in the model's predictions. Behavior prediction accuracy directly measures the consistency between the model's predictions and actual behavior and is a key indicator for evaluating model prediction performance. By collecting this behavioral feedback data, this embodiment can comprehensively analyze and evaluate the prediction model, identifying problems and deficiencies in the model's predictions. This data provides strong support for subsequent iterative model updates, helping to continuously optimize model performance and improve prediction accuracy. After collecting sufficient behavioral feedback data, this embodiment will iteratively update the prediction model based on this data. This process includes adjusting model parameters, optimizing the model structure, or introducing new features. Through continuous iterative updates, the model can gradually adapt to user behavior patterns and changing needs, improving prediction accuracy and reliability. At the same time, this can also enhance the generalization ability of the model, enabling it to better handle unseen scenarios and data, thereby further improving the performance of digital human behavior prediction and user experience.

[0126] S502: Calculating a performance indicator of the behavior prediction model based on the behavior feedback data;

[0127] It should be noted that in step S502, based on the collected behavioral feedback data, this embodiment calculates multiple performance indicators of the behavior prediction model. These performance indicators are intended to comprehensively evaluate the model's performance and identify the model's strengths and weaknesses in prediction. Specifically, these performance indicators include, but are not limited to, prediction accuracy, user satisfaction improvement rate, and reduction in behavior correction rate. Prediction accuracy is a key indicator for measuring the consistency between the model's predictions and actual behavior, and it directly reflects the model's predictive capabilities. The user satisfaction improvement rate assesses the impact of model optimization on user acceptance by comparing user satisfaction data before and after the model update. The reduction in behavior correction rate records whether the number of user corrections due to prediction errors has decreased after the model's iterative update, which helps verify the effectiveness of model optimization in reducing prediction errors. By calculating these performance indicators, this embodiment can quantitatively evaluate the performance of the prediction model, providing strong data support for subsequent model optimization and iterative updates. Furthermore, these performance indicators can serve as a visual representation of model performance improvement, demonstrating to users the improvements and practical application value of the model during the optimization process.

[0128] S503, determining whether the performance indicator meets a preset threshold;

[0129] If satisfied, the current behavior prediction model parameters remain unchanged;

[0130] If not, adjust the model parameters or introduce new feature variables based on the behavioral feedback data;

[0131] It should be noted that in step S503, the calculated behavior prediction model performance indicators are rigorously evaluated. This evaluation process aims to determine whether the model performance meets preset standards and requirements. Preset thresholds are determined based on factors such as actual application scenarios and requirements, as well as historical model performance data. These thresholds reflect the expected levels of the model in terms of prediction accuracy, user satisfaction, and behavior correction rate. If the performance indicators meet the preset thresholds, the performance of the current behavior prediction model has met the expected requirements. In this case, to maintain model stability and reliability, this embodiment will maintain the parameters of the current behavior prediction model unchanged. This means that the model will continue to predict in its current state without the need for additional adjustment or optimization. However, if the performance indicators do not meet the preset thresholds, this indicates that the current behavior prediction model has deficiencies in prediction accuracy, user satisfaction, or behavior correction rate. In this case, this embodiment will adjust the model parameters or introduce new feature variables based on the behavior feedback data. This step aims to improve the model's prediction performance and user satisfaction by optimizing the model structure and parameters. Adjusting model parameters involves modifying the weight matrix and bias vector, while introducing new feature variables involves adding new input features or improving the feature extraction method. Through these adjustments and optimizations, the model can better adapt to user behavior patterns and demand changes, and improve the accuracy and reliability of predictions.

[0132] S504: Repeat steps S501 to S503 until the performance indicator meets the preset conditions.

[0133] It should be noted that in step S504, to ensure that the performance of the behavior prediction model can continuously improve and meet the needs of practical applications, this embodiment employs an iterative update process. This process forms a closed-loop feedback mechanism by repeating steps S501 to S503. In each iteration, user behavioral feedback data from actual use is collected and, based on this data, performance indicators of the behavior prediction model are calculated. These performance indicators are then rigorously evaluated to determine whether the model performance meets preset standards and requirements. If the performance indicators do not meet the preset thresholds, model parameters are adjusted or new feature variables are introduced based on the behavioral feedback data to optimize the model structure and improve prediction performance. This process continues until the performance indicators meet the preset conditions. In this way, this embodiment can continuously iteratively update and optimize the behavior prediction model, enabling it to better adapt to user behavior patterns and changing needs. This also ensures that the model demonstrates good performance across multiple dimensions, including prediction accuracy, user satisfaction, and generalization ability. Ultimately, this will help improve the performance and user experience of digital human behavior prediction, providing more intelligent and reliable support for practical application scenarios.

[0134] Example 2

[0135] A computer device comprising:

[0136] processor;

[0137] a memory for storing processor-executable instructions;

[0138] The processor is configured to implement a scene-adaptive digital human behavior prediction method when executing the executable instructions.

[0139] It should be noted that the computer device includes: a processor, a memory, and may further include one or more of a multimedia component, an input / output (I / O) interface, and a communication component.

[0140] The processor is used to control the overall operation of the computer device to complete all or part of the steps in the above-mentioned scene-adapted digital human behavior prediction method.

[0141] The memory is used to store various types of data to support the operation of the computer device. Such data may include, for example, instructions for any application or method operating on the computer device, as well as application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0142] The multimedia component may include a screen and an audio component, wherein the screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals; for example, the audio component may include a microphone for receiving external audio signals, and the received audio signals may be further stored in a memory or sent through a communication component; the audio component also includes at least one speaker for outputting audio signals.

[0143] The I / O interface provides an interface between the processor and other interface modules, such as a keyboard, a mouse, buttons, etc. These buttons can be virtual buttons or physical buttons.

[0144] The communication component is used for wired or wireless communication between the computer device and other devices; wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G or 5G, or one or a combination of them, so the corresponding communication component may include: Wi-Fi module, Bluetooth module, NFC module, mobile phone communication module.

[0145] As a preferred solution of this embodiment, the computer device can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned scenario-adapted digital human behavior prediction method.

[0146] Example 3

[0147] A computer-readable storage medium comprising:

[0148] a memory having a computer program stored thereon;

[0149] A processor is used to execute the program in the memory to implement a scene-adaptive digital human behavior prediction method.

[0150] It should be noted that the electronic device according to the embodiment of the present disclosure includes a processor and a memory for storing processor executable instructions, wherein the processor is configured to implement any of the above-mentioned scene-adapted digital human behavior prediction methods when executing the executable instructions.

[0151] It should be noted that the number of processors can be one or more. Furthermore, the electronic device in the embodiments of the present disclosure may also include an input device and an output device. The processor, memory, input device, and output device may be connected via a bus or other means, which are not specifically limited here.

[0152] Memory, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the scenario-adaptive digital human behavior prediction method in the embodiments of the present disclosure. The processor executes the software programs or modules stored in the memory to perform various functional applications and data processing of the electronic device.

[0153] The input device can be used to receive input numbers or signals. The signals can be key signals related to user settings and function control of the device / terminal / server. The output device can include a display device such as a display screen.

[0154] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and alterations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A scene-adaptive digital human behavior prediction method, characterized in that: The method comprises: S1. Collecting multimodal user interaction data and preprocessing the multimodal user interaction data; the multimodal user interaction data includes text data, voice data, and image data; S2. Obtaining a multimodal feature vector in the multimodal user interaction data based on a pre-trained multimodal feature extraction model; S3. Obtaining a scene recognition result using a pre-trained scene recognition model based on the multimodal feature vector, and predicting the digital human's behavior using a behavior prediction model based on the scene recognition result to obtain a digital human behavior prediction result; Obtaining the digital human behavior prediction results in step S3 includes: S301, collecting historical behavior data of digital humans, and sorting the historical behavior data according to time series to obtain a time series data set; S302, extracting digital human behavior-related features from the time series data set; S303, obtaining the digital human behavior trend based on the behavior-related characteristics; S304: Based on the behavior trend and scene recognition results, the trained behavior prediction model is used to predict the behavior of the digital human, and a prediction probability distribution of the digital human behavior is obtained; S305: Based on the predicted probability distribution of the digital human behavior, select the behavior with the highest probability as the digital human behavior prediction result; S4. Based on the digital human behavior prediction results, a transfer learning mechanism is introduced to optimize the behavior prediction model: Introducing a transfer learning mechanism to optimize the behavior prediction model in step S4 includes: S401. Obtaining a behavior prediction model performance deviation based on a difference between historical prediction data and current prediction data; S402: Select a transfer learning strategy based on the performance deviation; S403: Apply the transfer learning strategy to adjust the parameters or structure of the behavior prediction model; S404, evaluating the performance of the behavior prediction model after transfer learning, and determining whether the performance of the behavior prediction model has improved based on the evaluation result; If so, save the model parameters after transfer learning; If not, return to step S402 to reselect the transfer learning strategy until the optimal transfer learning strategy is found; S5. Collect user feedback information in real time, and iteratively update the prediction model based on the user feedback information.

2. The scene-adaptive digital human behavior prediction method according to claim 1, characterized in that: Acquiring the multimodal feature vector in the multimodal user interaction data in step S2 includes: S101, extracting semantic features from the text data based on a pre-trained BERT model; S102, converting the speech data into text based on a speech recognition model, and extracting emotional features; S103, extracting visual features from the image data based on a face recognition model; S104: fusing the semantic features, emotional features, and visual features based on an attention mechanism to obtain a multimodal feature vector.

3. The scene-adaptive digital human behavior prediction method according to claim 2, characterized in that: The expression for obtaining the multimodal feature vector in step S104 is: in, Represents the semantic feature vector after mapping, W T Represents the weight matrix of semantic features, T represents the semantic features, b T The bias vector representing the semantic feature, Represents the mapped emotional feature vector, W A Represents the weight matrix of emotional features, A represents emotional features, b A The bias vector representing the sentiment feature, Represents the mapped visual feature vector, W V Represents the weight matrix of visual features, V represents visual features, b V The bias vector representing the visual feature, α T Represents the attention weight of semantic features, exp() represents the exponential function, and Both represent weight vectors, α A represents the attention weight of the emotional feature, α V represents the attention weight of the visual feature, b′ T , b′ A and b′ V express and The corresponding bias term, F represents the fused multimodal feature vector.

4. The scene-adaptive digital human behavior prediction method according to claim 1, characterized in that: The expression for obtaining the digital human behavior trend based on the behavior-related features is: D t =η·M t +β·S t +γ·E t M t =λ·(Y t -Y t-1 )+(1-λ)·M t-1 Among them, D t represents the behavior trend, η, β and γ are weight coefficients, M t represents the momentum of the behavior change at time t, S t Indicates seasonal changes in behavior, E t represents the impact of external factors on behavior, λ represents the momentum smoothing factor, and Y t Indicates the digital human behavior label at time t, Y t-1 The digital human behavior label at time t-1, M t-1 The momentum of the behavior change at time t-1, K represents the length of the seasonal cycle, φ k represents the seasonal weight coefficient, Y t-k·P represents the behavior label at time tk·P, P represents the basic unit of seasonal cycle, N represents the number of external influencing factors, θ i Indicates the weight coefficient of the i-th external influencing factor, E i,t Represents the value of the i-th external influencing factor at time t.

5. The scene-adaptive digital human behavior prediction method according to claim 1, characterized in that: The expression for the predicted probability distribution of digital human behavior obtained in step S304 is: P t =f(D t ,H t ) P t =softmax(W o ·h t +b o ) h t =σ(W h ·[D t ;H t ]+b h ) Among them, P t represents the probability distribution of various behaviors that the digital human may take at time t, f() represents the behavior prediction model, and D t Indicates behavioral trend, H t represents the scene recognition result at time t, softmax() represents the activation function, W o represents the weight matrix of the output layer, h t represents the hidden layer output at time t, b o Represents the bias vector of the output layer, σ() represents the ReLU function, W h represents the weight matrix of the hidden layer, [;] represents the concatenation operation, b h Bias vector for the hidden layer.

6. The scene-adaptive digital human behavior prediction method according to claim 1, characterized in that: In step S5, iteratively updating the prediction model based on the user feedback information includes: S501, collecting user behavior feedback data during actual use, wherein the behavior feedback data includes user satisfaction, number of behavior corrections, and behavior prediction accuracy; S502: Calculating a performance indicator of the behavior prediction model based on the behavior feedback data; S503, determining whether the performance indicator meets a preset threshold; If satisfied, the current behavior prediction model parameters remain unchanged; If not, adjust the model parameters or introduce new feature variables based on the behavioral feedback data; S504: Repeat steps S501 to S503 until the performance indicator meets the preset conditions.

7. A computer device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement a scene-adapted digital human behavior prediction method according to any one of claims 1 to 6 when executing the executable instructions.

8. A computer-readable storage medium, characterized in that include: a memory having a computer program stored thereon; A processor is used to execute the program in the memory to implement the scene-adaptive digital human behavior prediction method according to any one of claims 1 to 6.

Images