Artificial intelligence-based personalized virtual companion interaction method, system, device and medium

Abstract

This application relates to a personalized virtual companion interaction method, system, device, and medium based on artificial intelligence. The method includes: extracting multimodal features from multimodal interaction data to obtain a current user feature vector; dynamically updating the user profile based on the current user feature vector to obtain a global user profile vector; mapping character personality parameters based on the global user profile vector to obtain a virtual companion personality parameter vector; updating the memory based on the current user feature vector to obtain an updated memory bank; retrieving the memory based on the current user feature vector and the updated memory bank to obtain a memory context; making behavioral decisions based on the virtual companion personality parameter vector, the memory context, and the current environmental state to obtain an interaction intent guidance signal; and generating a multimodal interaction data packet based on the interaction intent guidance signal, the memory context, and the virtual companion personality parameter vector. This method can dynamically adapt character personality and user characteristics, and improve interaction coherence through memory management.

Description

Technical Field

[0001] This invention belongs to the field of virtual interaction technology, and in particular relates to a personalized virtual companion interaction method, system, device and medium based on artificial intelligence. Background Technology

[0002] With the rapid development of artificial intelligence technology and human-computer interaction, interactive systems such as virtual companions and intelligent assistants are becoming increasingly popular and are widely used in scenarios such as emotional support, intelligent customer service, and educational tutoring. Users' requirements for interactive experiences have shifted from simple information acquisition to the pursuit of immersive experiences with anthropomorphic features, personalized styles, and a sense of long-term companionship. This requires virtual interactive systems to accurately perceive the user's state and make coherent responses that conform to the role settings.

[0003] In traditional technologies, virtual companion interaction methods typically employ rule engines, template matching, or deep learning dialogue models. Regarding user modeling, existing technologies largely rely on static attribute labels or simple behavioral statistical features actively entered by users to construct user profiles. In terms of role setting, fixed parameters are often preset during the development phase, resulting in virtual character personalities that are often indistinguishable across different users. Furthermore, in terms of context management, existing technologies mostly use sliding window mechanisms to store only a limited amount of recent dialogue history, lacking a long-term storage and management mechanism for historical interaction information.

[0004] However, existing virtual companion interaction methods suffer from technical problems such as the inability to dynamically adapt user profiles to character personalities and poor interaction consistency due to long-term memory loss. Because current technologies lack a mechanism to dynamically adjust character personality parameters based on real-time user characteristics, virtual companions cannot present differentiated interaction styles according to changes in user status. Simultaneously, existing context processing methods cannot achieve continuous memory and accurate retrieval of key historical events, causing the system to easily forget important information during long-term interactions, making it difficult to maintain a stable personalized interaction experience. Summary of the Invention

[0005] Therefore, it is necessary to address the aforementioned technical issues by providing an AI-based personalized virtual companion interaction method, system, device, and medium that can dynamically adapt to user characteristics and role personalities and achieve long-term memory management to improve interaction coherence.

[0006] Firstly, this application provides a personalized virtual companion interaction method based on artificial intelligence, including:

[0007] S1. Extract multimodal features based on multimodal interaction data to obtain the current user feature vector;

[0008] S2. Dynamically update the user profile based on the current user feature vector to obtain the global user profile vector;

[0009] S3. Based on the global user profile vector, perform character personality parameter mapping to obtain the virtual partner personality parameter vector;

[0010] S4. Update the memory based on the current user feature vector to obtain the updated memory.

[0011] S5. Perform memory retrieval based on the current user feature vector and the updated memory bank to obtain the memory context;

[0012] S6. Make behavioral decisions based on the virtual partner's personality parameter vector, memory context, and current environmental state to obtain interaction intention guidance signals;

[0013] S7. Based on the interaction intent guidance signal, memory context and virtual partner personality parameter vector, multimodal response is generated to obtain multimodal interaction data packets.

[0014] In one embodiment, S1 includes:

[0015] S11. Semantically encode the text data in the multimodal interaction data to obtain the text semantic vector;

[0016] S12. Extract acoustic features from the speech data in the multimodal interaction data to obtain the acoustic emotion vector;

[0017] S13. Extract visual features from the video data in the multimodal interaction data to obtain the visual state vector;

[0018] S14. Based on preset weight coefficients, the text semantic vector, acoustic emotion vector, and visual state vector are weighted and fused to obtain the current user feature vector.

[0019] In one embodiment, S2 includes:

[0020] S21. Obtain historical interaction logs and extract the global user profile vector from the historical interaction logs for the previous moment.

[0021] S22. Determine the update learning rate at the current moment according to the preset profile update strategy;

[0022] S23. Calculate the global user profile vector by performing an exponential moving average based on the current user feature vector, the global user profile vector of the previous time step, and the updated learning rate.

[0023] In one embodiment, S4 includes:

[0024] S41. Perform feature value analysis based on the current user feature vector to obtain the memory importance score;

[0025] S42. When the memory importance score exceeds the preset importance threshold, the current interaction event is vectorized and encoded to obtain the memory embedding vector.

[0026] S43. Write the memory embedding vector into the memory bank to obtain the updated memory bank.

[0027] In one embodiment, S5 includes:

[0028] S51. Calculate the semantic similarity between the current user feature vector and the memory embedding vector in the updated memory bank, and obtain the semantic similarity value.

[0029] S52. Calculate the time decay factor by performing exponential decay calculation based on the time difference between the current time and the memory generation time.

[0030] S53. The semantic similarity scores and time decay factors are weighted and summed to obtain the memory retrieval score;

[0031] S54. Based on the memory retrieval score, sort the memory embedding vectors in the updated memory bank in descending order to obtain a descending sorted list, and extract a preset number of memory fragments from the descending sorted list to obtain the memory context.

[0032] In one embodiment, S6 includes:

[0033] S61. Monitor user interaction status based on real-time interaction logs to obtain the user's current interaction mode; the user's current interaction mode includes silent state and active state.

[0034] S62. When the user's current interaction mode is silent, obtain the initiative weight parameter and the user's silence duration from the virtual partner's personality parameter vector.

[0035] S63. Calculate the probability of proactive interaction based on the initiative weight parameter and the user's silence duration to obtain the probability of proactive interaction triggering; the formula for calculating the probability of proactive interaction triggering is:

[0036]

[0037] in, This represents the probability of triggering an active interaction. For adjustment coefficients, For proactive weighting parameters, User silence duration;

[0038] S64. Compare the probability of active interaction with the preset trigger threshold to obtain the interaction intention guidance signal.

[0039] In one embodiment, S7 includes:

[0040] S71. Based on the virtual partner's personality parameter vector, interaction intention guidance signals, and memory context, cue words are assembled to obtain structured cue words;

[0041] S72. Generate text from the structured prompt words to obtain the text response content;

[0042] S73. Based on the emotion weight parameter in the virtual partner's personality parameter vector, perform multimodal rendering on the text response content to obtain the speech stream and virtual character animation.

[0043] S74. Encapsulate the text response content, voice stream, and virtual character animation to obtain a multimodal interactive data packet.

[0044] Secondly, this application also provides a personalized virtual companion interaction system based on artificial intelligence, including:

[0045] The multimodal feature extraction module is used to extract multimodal features based on multimodal interaction data to obtain the current user's feature vector;

[0046] The dynamic user profile update module is used to dynamically update the user profile based on the current user feature vector to obtain the global user profile vector.

[0047] The personality parameter mapping module is used to map character personality parameters based on the global user profile vector to obtain the virtual partner personality parameter vector.

[0048] The memory update module is used to update the memory based on the current user feature vector to obtain the updated memory.

[0049] The memory retrieval module is used to perform memory retrieval based on the current user feature vector and the updated memory bank to obtain the memory context.

[0050] The behavior decision-making module is used to make behavior decisions based on the virtual partner's personality parameter vector, memory context, and current environmental state, and to obtain interaction intention guidance signals.

[0051] The multimodal response generation module is used to generate multimodal responses based on interaction intent guidance signals, memory context, and virtual partner personality parameter vectors, resulting in multimodal interaction data packets.

[0052] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the AI-based personalized virtual companion interaction method as described in the first aspect.

[0053] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the AI-based personalized virtual companion interaction method as described in the first aspect.

[0054] The aforementioned AI-based personalized virtual companion interaction method, system, device, and medium extract features from multimodal interaction data to obtain the current user feature vector, and dynamically update it to generate a global user profile vector. This global user profile is then mapped to a virtual companion personality parameter vector, enabling adaptive matching of the virtual companion's personality as user characteristics change. Simultaneously, a memory update and retrieval mechanism based on the current user feature vector constructs and calls an updated memory bank to obtain memory context, ensuring long-term storage and accurate retrieval of historical information. Finally, by integrating the virtual companion's personality parameters, memory context, and current environmental state, behavioral decisions and multimodal response generation are performed. This allows the virtual companion to adjust its interaction style based on the dynamic profile and maintain coherent dialogue logic based on historical memory, thus solving the problems of fixed personality and long-term memory loss in existing technologies, significantly improving the personalization of virtual interaction and the coherence of the user experience. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a personalized virtual companion interaction method based on artificial intelligence provided by the present invention;

[0057] Figure 2 A flowchart illustrating a method for generating interactive intent guidance signals in an optional embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of the structure of a personalized virtual companion interaction system based on artificial intelligence, provided by the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] In one embodiment, such as Figure 1As shown, a personalized virtual companion interaction method based on artificial intelligence is provided. This embodiment illustrates the method applied to a terminal, but it is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0061] S1. Extract multimodal features based on multimodal interaction data to obtain the current user feature vector.

[0062] Optionally, the collected multimodal interaction data is preprocessed first. For text data, word segmentation, stop word removal, and part-of-speech tagging are performed. For speech data, voiceprint feature extraction, speech frame segmentation, and acoustic feature parsing are performed. For visual interaction data, keyframe extraction, feature point detection, and semantic content parsing are performed. For example, text data is extracted using BERT (Bidirectional Encoder Representations from Transformers), speech data is extracted using the Mel-frequency cepstral coefficient algorithm, and video data is extracted using ResNet (Residual Neural Network). Then, the features extracted from each modality are fused. The fusion process is achieved by feature concatenation, where feature vectors of different dimensions are concatenated in dimensional order to form a feature set of a unified dimension. Finally, the fused feature set is normalized to eliminate the dimensional differences between features of different modalities, thereby obtaining a current user feature vector that can represent the user's current interaction state. The dimension of this current user feature vector is adapted to the rules of feature extraction and fusion, and can comprehensively reflect the user's multidimensional behavior and expression characteristics in this interaction.

[0063] S2. Dynamically update the user profile based on the current user feature vector to obtain the global user profile vector.

[0064] Optionally, the stored historical global user profile vector is retrieved first. This vector is a comprehensive representation of all past user interaction features. Then, the current user feature vector is fused with the historical global user profile vector. The fusion process adopts a weighted fusion method, where the weight of the current user feature vector is allocated according to the real-time nature and importance of the interaction. Preferably, the current user feature vector generated by recent interactions is assigned a higher weight. The fused feature vector is then updated and optimized. Redundant features are removed through feature screening, and core features that can reflect changes in user characteristics are retained. At the same time, the values ​​in the feature vector are iteratively updated so that the updated vector can accurately represent the latest feature state of the user. Furthermore, the optimized feature vector is structured and stored to form a new global user profile vector. This vector can dynamically evolve with each user interaction, comprehensively and in real time reflecting the user's comprehensive attributes such as interests, preferences, interaction behavior characteristics, and language style characteristics.

[0065] S3. Based on the global user profile vector, perform character personality parameter mapping to obtain the virtual partner personality parameter vector.

[0066] Optionally, a predefined personality parameter space is first retrieved. This space contains various personality construction parameters, language expression style parameters, interaction initiative parameters, emotional response weight parameters, and behavioral strategy weight parameters. Each parameter exists in vector form and covers different personality characteristic dimensions. Then, the global user profile vector is input into a preset mapping model. This model is constructed using a combination of clustering and rule matching. Specifically, the global user profile vector is first matched to similar clusters in the personality parameter space using the clustering model. Then, rule matching is used to select the personality parameter combination with the highest matching degree with the global user profile vector features in this cluster. Subsequently, the selected personality parameters are vectorized and normalized to integrate different types of personality parameters into a vector form with a unified dimension, ultimately obtaining the virtual partner personality parameter vector. This vector can achieve differentiated matching based on the features of the global user profile vector, making the virtual partner's personality characteristics compatible with the user's characteristics.

[0067] S4. Update the memory based on the current user feature vector to obtain the updated memory.

[0068] Optionally, feature parsing is performed on the current user feature vector to extract core features that reflect the user's latest interests, preferences, interactive behaviors, and key interactive information. Then, a stored memory bank is retrieved, which includes a short-term memory area and a long-term memory area. The short-term memory area stores the current session context, while the long-term memory area stores the user's past preferences, historical events, and key interaction results. Subsequently, the memory bank is updated according to a memory update strategy. Specifically, the core features extracted from the current user feature vector are stored in the short-term memory area or the long-term memory area of ​​the memory bank according to a comprehensive rule of time decay, importance score, and usage frequency. Core features with high importance scores and low usage frequency are stored in the long-term memory area, while session-related features with strong real-time characteristics are stored in the short-term memory area. At the same time, the original content in the memory bank is iterated, and redundant content that has reached the time decay threshold and has no importance is removed. The updated content is replaced with numerical values ​​and features, and finally, an updated memory bank is obtained. This memory bank can realize the continuous storage and dynamic evolution of user features.

[0069] S5. Based on the current user feature vector and the updated memory bank, perform memory retrieval to obtain the memory context.

[0070] Optionally, the current user's feature vector is first encoded into a feature vector that matches the data format in the memory. Then, based on a semantic similarity algorithm, the similarity between the retrieved feature vector and all feature data in the updated memory is calculated, such as cosine similarity, which can accurately match the semantic associations between feature vectors. At the same time, the retrieval is optimized by combining the index structure, creating a dedicated index for different content in the memory, matching the corresponding index according to the feature dimension of the retrieved feature vector, and quickly locating relevant content in the memory. Subsequently, the similarity calculation results are filtered by a threshold to select memory content that meets the preset similarity standard. The selected content is then spliced ​​and structured, and integrated into a coherent content set according to the time sequence and logical association of the interaction, finally obtaining the memory context. This memory context can accurately reflect the historical interaction information and user preferences related to the current user's feature vector.

[0071] S6. Make behavioral decisions based on the virtual partner's personality parameter vector, memory context, and current environmental state to obtain interaction intention guidance signals.

[0072] Optionally, the current environmental state is first retrieved, which includes environmental feature parameters related to time, scene, and interaction scenario. Simultaneously, the virtual partner personality parameter vector, memory context, and current environmental state are all converted into feature vectors of a unified dimension, ensuring that the data formats of each vector are compatible. The converted feature vectors are then input into the behavior decision model, which is constructed based on preset behavior triggering conditions. These conditions include interaction rules corresponding to the virtual partner's personality characteristics, user interaction habit rules in the memory context, and scene interaction rules corresponding to the current environmental state. Specifically, the behavior decision model first determines the basic style and initiative threshold of the interaction based on the virtual partner's personality parameter vector, then matches the user's corresponding interaction response habits with the memory context, and simultaneously filters behavior decision directions that meet the scene requirements based on the current environmental state. Subsequently, the three types of rules are comprehensively weighted and judged. When the judgment result reaches the threshold of the behavior triggering condition, a corresponding interaction intent guidance signal is generated. This signal contains core information such as the type of interaction, initiative, and content orientation.

[0073] S7. Based on the interaction intent guidance signal, memory context and virtual partner personality parameter vector, multimodal response is generated to obtain multimodal interaction data packets.

[0074] Optionally, the interaction intent guidance signal is subjected to feature analysis to extract core guidance information such as interaction type, initiative, and content orientation. Then, using the virtual partner's personality parameter vector as a constraint, the language expression style, emotional response level, and interaction presentation form of the multimodal response are determined to ensure that the response content is consistent with the personality characteristics of the virtual partner. At the same time, the memory context is used as the core basis for the response content. User preferences and historical interaction information related to the current interaction are extracted from the memory context to ensure that the response content matches the user's characteristics and past interaction situations. Subsequently, response content for each modality is generated separately. For the text modality, natural language text is generated according to the determined expression style and content orientation. For the speech modality, speech synthesis parameters are set according to the text content and emotional characteristics to generate speech data. For the visual modality, corresponding visual presentation data is generated according to the interaction scenario and emotional characteristics. Furthermore, the response content generated by each modality is synchronously managed and formatted uniformly. The content of different modalities is integrated into a structured data package, and finally a multimodal interaction data package is obtained. This data package can realize the collaborative output of text, speech, vision, and other multimodal modes to meet the user's multimodal interaction needs.

[0075] The aforementioned AI-based personalized virtual companion interaction method starts with feature extraction from multimodal interaction data, dynamically updates user profiles, and accurately maps virtual companion personality parameters. It then effectively retrieves historical interaction information through memory bank updates and retrieval, combines environmental conditions to make behavioral decisions, and ultimately generates multimodal interaction responses. This achieves dynamic adaptation between user characteristics and virtual companion personality, long-term memory and accurate retrieval of user interaction information, and proactive behavioral decision-making and multimodal collaborative responses. This allows virtual companion interactions to align with users' real-time characteristics and long-term preferences, resulting in a personalized, continuous, and natural virtual companion interaction experience. This effectively solves the problems of homogenized interactions, memory loss, and passive responses in existing technologies.

[0076] In one embodiment, S1 includes:

[0077] S11. Semantically encode the text data in the multimodal interaction data to obtain the text semantic vector.

[0078] Optionally, preprocessing operations are performed on the text data in the multimodal interaction data, sequentially completing word segmentation, part-of-speech tagging, and stop word removal. This removes invalid and redundant information from the text data, retaining core words with actual semantic expression. Specifically, word segmentation is achieved using a dictionary-based matching algorithm, and stop word removal is performed based on a general natural language processing stop word list. The preprocessed text data is then input into a pre-trained word embedding model. For example, the BERT (Bidirectional Encoder Representations from Transformers) model is used to complete semantic encoding. This BERT model captures the contextual semantic relationships between words in the text data through a bidirectional attention mechanism, transforming discrete text words into continuous low-dimensional numerical vectors. The word vectors output by the model are then pooled to obtain a vector result that can represent the complete semantic information of the entire text data. Finally, a text semantic vector is obtained. The dimension of this vector matches the output dimension of the pre-trained word embedding model, accurately reflecting the user's expressive intent and semantic features in the text data.

[0079] S12. Perform acoustic feature extraction on the speech data in the multimodal interaction data to obtain the acoustic emotion vector.

[0080] Optionally, the speech data in the multimodal interaction data is preprocessed by sequentially performing speech frame segmentation, pre-emphasis, and windowing operations. The continuous speech signal is divided into fixed-length speech frames. Pre-emphasis enhances the signal strength of the high-frequency components in the speech frames, and windowing reduces signal distortion at the edges of the speech frames. A Hamming window is used for windowing to effectively suppress spectral leakage. Subsequently, Mel-frequency cepstral coefficients are extracted from the preprocessed speech frames. These coefficients simulate the auditory characteristics of the human ear, accurately capturing the acoustic features in the speech data. Simultaneously, a speech emotion recognition algorithm is used to analyze the extracted acoustic features in terms of emotional dimension, identifying the emotional tendencies and intensity-related features contained in the speech data. The acoustic and emotional features are then fused to form a unified-dimensional feature vector, ultimately yielding an acoustic emotion vector. This vector simultaneously represents the acoustic properties of the speech data and the user's emotional expression characteristics.

[0081] S13. Visual features are extracted from the video data in the multimodal interaction data to obtain the visual state vector.

[0082] Optionally, the video data in the multimodal interaction data is preprocessed by sequentially performing video frame extraction, image denoising, and size normalization. Key video frames are extracted from the continuous video stream at fixed time intervals, and noise information in the video frames is removed using a Gaussian filtering algorithm. All video frames are adjusted to a uniform pixel size to ensure consistency in feature extraction. Subsequently, the preprocessed key video frames are input into a convolutional neural network to complete visual feature extraction. This convolutional neural network captures visual features such as texture, contour, human posture, and facial expressions in the video frames through layer-by-layer operations of convolutional and pooling layers. For example, ResNet (Residual Neural Network) is used to achieve deep extraction of visual features. Then, the visual features output by the network are filtered to remove redundant features that are irrelevant to the user's state, retaining the core features that can reflect the user's facial expressions, body movements, and other state information. After fusion, a visual state vector is formed, which can accurately represent the user's visual behavior and state features in the video data.

[0083] S14. Based on preset weight coefficients, the text semantic vector, acoustic emotion vector, and visual state vector are weighted and fused to obtain the current user feature vector.

[0084] Optionally, preset weighting coefficients are retrieved. These coefficients are numerically predetermined based on the importance of multimodal interaction data in user feature representation. The text semantic vector, acoustic emotion vector, and visual state vector each correspond to independent weighting coefficients, and the sum of all weighting coefficients is 1, ensuring the normalization of the vector values ​​after feature fusion. Subsequently, the text semantic vector, acoustic emotion vector, and visual state vector are multiplied by their respective preset weighting coefficients to obtain weighted text semantic vectors, weighted acoustic emotion vectors, and weighted visual state vectors. Specifically, the vector multiplication involves multiplying the elements of the corresponding dimension by the weighting coefficient, keeping the vector dimensions unchanged. The three weighted feature vectors are then added element-wise according to their dimensions to complete the weighted fusion operation, resulting in a fused feature vector. Finally, the fused feature vector is normalized to eliminate the influence of differences in the dimensions of different feature vectors, ultimately yielding the current user feature vector. This vector comprehensively represents the overall user characteristics in the text, voice, and video multimodal interaction data.

[0085] In the above embodiments, targeted feature extraction and encoding are performed on text, voice, and video data in multimodal interaction data to obtain feature vectors corresponding to each modality. Then, the multimodal feature vectors are weighted and fused by preset weight coefficients, realizing refined feature analysis and efficient fusion of multimodal data. It can comprehensively capture the user's interaction features from multiple dimensions such as semantics, emotion, and visual state, so that the obtained current user feature vector has multi-dimensional and all-round user feature representation capabilities.

[0086] In one embodiment, S2 includes:

[0087] S21. Obtain historical interaction logs and extract the global user profile vector from the previous moment in the historical interaction logs.

[0088] Optionally, the stored historical interaction logs are retrieved. These logs are structured data containing all past multimodal interaction data, feature extraction results, and user profile update records. They cover multi-dimensional historical interaction-related information such as user interest preference tags, interaction behavior characteristics, and language style characteristics, and all data is stored sequentially according to the interaction time order. The historical interaction logs are then parsed, and core data related to the previous global user profile vector is selected from the logs according to preset feature extraction rules. This core data is the final feature vector data stored after the previous profile update. The selected data is then format-validated to ensure that the data dimensions and vector format are compatible with the next calculation requirement. After successful validation, the corresponding vector data is directly extracted, ultimately obtaining the previous global user profile vector. This vector can fully represent the user's comprehensive feature state at the previous moment.

[0089] S22. Determine the update learning rate at the current moment based on the preset profile update strategy.

[0090] Optionally, a preset profile update strategy is retrieved. This strategy is a pre-defined update rule based on the characteristic change patterns of user interaction behavior, mainly including three judgment dimensions: interaction time interval, user feature change magnitude, and importance of interaction content. Each dimension corresponds to different judgment criteria and coefficient allocation rules. Subsequently, combining the current interaction scenario and the user's real-time interaction characteristics, the three judgment dimensions are quantitatively judged in turn. Specifically, the time interval between the current interaction and the previous interaction is determined first, then the change magnitude of the current user feature vector relative to historical features is calculated, and the importance level of the current interaction content is identified. Then, according to the preset dimension weights in the profile update strategy, the judgment results of each dimension are comprehensively weighted and calculated. The calculation result is mapped to the corresponding numerical coefficient, which is the update learning rate at the current moment. The update learning rate can reflect the degree of influence of the current interaction features on the update of the user's global profile.

[0091] S23. Calculate the global user profile vector by performing an exponential moving average based on the current user feature vector, the global user profile vector of the previous time step, and the updated learning rate.

[0092] Optionally, the current user feature vector and the previous global user profile vector are first checked for dimensionality consistency to ensure that the two vectors have the same dimensions. If there is a dimensional inconsistency, dimension completion is performed according to a preset vector completion rule to ensure the feasibility of the exponential moving average calculation. The exponential moving average calculation achieves dynamic and smooth updates of data by assigning different weights to data at different times, with recent data being assigned higher weights to highlight its impact on the results. Then, the current update learning rate is used as the weight coefficient of the current user feature vector, and 1 minus the current update learning rate is used as the weight coefficient of the previous global user profile vector. The two feature vectors are then multiplied element-wise by their respective weight coefficients to obtain the weighted current user feature vector and the weighted previous global user profile vector. Finally, the two weighted feature vectors are summed element-wise to complete the exponential moving average calculation, ultimately obtaining the updated global user profile vector at the current time. This vector enables the dynamic evolution and accurate representation of user features.

[0093] In the above embodiments, the global user profile vector of the previous moment is first accurately extracted from the historical interaction log. Then, the update learning rate adapted to the current interaction is determined according to the preset profile update strategy. Finally, the dynamic update of the user profile is completed by calculating the exponential moving average. This realizes the continuous and dynamic update of the user profile, allowing the global user profile vector to keep up with the latest interaction features of the user in real time, while taking into account the user's historical feature attributes. This effectively avoids the abruptness of profile updates and improves the accuracy and stability of user profile representation.

[0094] In one embodiment, S4 includes:

[0095] S41. Perform feature value analysis based on the current user feature vector to obtain the memory importance score.

[0096] Optionally, a full-dimensional feature value analysis is performed on the current user feature vector, extracting feature values ​​representing user interests, interactive behavior characteristics, and language style characteristics from each dimension of the vector. Simultaneously, considering the current interaction scenario attributes, feature weights matching the scenario are assigned to different dimensions of feature values. These feature weights are pre-set based on the interaction scenario's focus on user characteristics. Subsequently, a comprehensive quantitative calculation is performed on the weighted feature values ​​of each dimension. The calculation process uses the prominence of the feature value, the difference from the user's historical characteristics, and the representational quality of the interaction content as core judgment indicators. The calculation results of these three types of indicators are fused, and the fused result is presented in numerical form as the basis for judging the importance of the corresponding interaction content. Finally, a memory importance score is obtained, which accurately reflects the actual importance of the interaction event corresponding to the current user feature vector to the user's memory representation.

[0097] S42. When the memory importance score exceeds the preset importance threshold, the current interaction event is vectorized and encoded to obtain the memory embedding vector.

[0098] Optionally, a preset importance threshold is retrieved. This threshold is a fixed value pre-set according to the storage rules of the memory bank and the importance judgment criteria of the interaction event. It serves as the core basis for selecting interaction events to be stored. Then, the memory importance score is compared with the preset importance threshold. When the memory importance score is determined to exceed the preset importance threshold, the current interaction event is structurally decomposed to extract core elements such as user expression content, interaction behavior, emotional tendency, and scene information. Subsequently, the decomposed core elements are input into a pre-trained embedding model for vectorization encoding. This model is constructed using conventional semantic encoding algorithms in the field of natural language processing, such as the Word2Vec algorithm, which can transform discrete interaction event elements into continuous low-dimensional numerical vectors, ultimately obtaining a memory embedding vector. This vector can completely and accurately represent all the core information of the current interaction event.

[0099] S43. Write the memory embedding vector into the memory bank to obtain the updated memory bank.

[0100] Optionally, the memory embedding vectors are first validated to ensure that their dimensions and data format are compatible with the storage specifications of the memory. If a format mismatch exists, the vector format is converted according to the memory's preset rules. Simultaneously, a timestamp and importance score label are added to the memory embedding vectors. The timestamp accurately records the occurrence time of the current interaction event, and the importance score label directly relates to the memory importance score corresponding to the vector. Then, based on the memory importance score and the memory's partitioning storage rules, the memory area where the memory embedding vector should be written is determined. Specifically, high-importance memory embedding vectors are written to the long-term memory area, and others that meet the storage conditions are written to the short-term memory area. Next, a dedicated semantic index is created for the memory embedding vectors according to the memory's indexing rules. Finally, the tagged and indexed memory embedding vectors are written to the designated storage location in the corresponding memory area. At the same time, redundant data in the memory is cleaned up according to time decay rules, resulting in an updated memory.

[0101] In the above embodiments, the importance score of the memory is obtained by analyzing the feature values ​​of the current user feature vector, and the interaction events exceeding the threshold are vectorized to obtain the memory embedding vector. Finally, the vector is written into the memory bank according to the rules to complete the update. This realizes the important filtering and accurate storage of interaction events, allowing the memory bank to retain only the interaction information that is important to the user. At the same time, it establishes standardized tags and indexes for the memory content, which not only improves the storage efficiency of the memory bank, but also provides an accurate and standardized data foundation for further memory retrieval, effectively realizing the dynamic and efficient updating of the memory bank.

[0102] In one embodiment, S5 includes:

[0103] S51. Calculate the semantic similarity between the current user feature vector and the memory embedding vector in the updated memory bank, and obtain the semantic similarity value.

[0104] Optionally, a dimensionality consistency check is performed on the current user feature vector and all memory embedding vectors in the updated memory bank to ensure that the vector dimensions match perfectly. If there are dimensional differences, the vector dimension completion process is completed according to the preset vector dimension completion rules to ensure the feasibility of similarity calculation. Then, the semantic similarity between the two is calculated using the cosine similarity algorithm. The cosine similarity algorithm represents the degree of semantic association by calculating the cosine value of the angle between the two vectors. Specifically, the magnitude of the current user feature vector and the single memory embedding vector are calculated separately, and then the dot product of the two vectors is calculated. The dot product result is divided by the product of the magnitudes of the two vectors, and the obtained value is the semantic similarity value between the single memory embedding vector and the current user feature vector. The above calculation operation is performed sequentially on all memory embedding vectors in the updated memory bank to obtain the semantic similarity value corresponding to each memory embedding vector.

[0105] S52. Calculate the time decay factor by performing exponential decay calculation based on the time difference between the current time and the memory generation time.

[0106] Optionally, first extract the memory generation time corresponding to each memory embedding vector in the updated memory bank from the current time. The memory generation time is the timestamp information marked when the memory embedding vector is written to the memory bank. Then, calculate the time difference between the two times. This time difference is the time interval between the current time and the memory generation time, and its time unit is consistent with the preset exponential decay calculation rule. Subsequently, perform exponential decay calculation, which makes the value decrease exponentially with the increase of the time interval. Specifically, substitute the calculated time difference into the preset exponential decay formula: ,in, This is the time decay factor, with a value range of (0,1]. The larger the value, the lower the degree of decay of the memory content over time and the stronger its timeliness. It is a natural constant. This is the decay coefficient, a pre-set positive constant that can be flexibly adjusted according to actual memory decay needs (such as the decay rate of short-term / long-term memory). The time difference is the time interval between the current time and the memory generation time. Using the time difference as the independent variable, the result is the time decay factor of the corresponding memory embedding vector. This time decay factor can reflect the degree of decay of the importance of memory content over time.

[0107] S53. The semantic similarity scores and time decay factors are weighted and summed to obtain the memory retrieval score.

[0108] Optionally, a preset weighting coefficient is retrieved. This coefficient is a fixed value pre-set based on the importance of semantic relevance and time decay in memory retrieval, wherein the semantic similarity value corresponds to the first weighting coefficient. The time decay factor corresponds to the second weighting coefficient. ,and Both weight coefficients are constants greater than 0 and less than 1, and can be set conventionally according to the actual needs of memory retrieval. Then, a weighted summation is performed on each memory embedding vector in the updated memory bank, using the following formula: ,in For memory retrieval score, This represents the semantic similarity score. The time decay factor is used to sum the values, which is the memory retrieval score of the corresponding memory embedding vector. The above calculation operation is performed on all memory embedding vectors in the updated memory bank in sequence to finally obtain the unique memory retrieval score corresponding to each memory embedding vector. This score can comprehensively represent the matching degree and timeliness of the memory content with the current user characteristics.

[0109] S54. Based on the memory retrieval score, sort the memory embedding vectors in the updated memory bank in descending order to obtain a descending sorted list, and extract a preset number of memory fragments from the descending sorted list to obtain the memory context.

[0110] Optionally, based on memory retrieval scores, all memory embedding vectors in the updated memory bank are sorted in descending order, with higher-scoring vectors appearing earlier. This results in a descending list containing all memory embedding vectors, which fully preserves the sorting information and corresponding memory content association information of the embedding vectors. A preset number of memory fragments are retrieved, pre-determined based on the representation requirements of the memory context and the content design of subsequent interactive responses. Then, memory fragments corresponding to the top preset number of memory embedding vectors are extracted according to the order of the descending list. These extracted memory fragments are then structured and assembled according to the chronological order of memory generation and semantic logical association, ultimately yielding a memory context that accurately matches the current user's characteristics.

[0111] In the above embodiments, the semantic similarity between the feature vector and the memory embedding vector is first calculated, then the time decay factor is obtained by combining the time difference and the weighted sum is used to obtain the memory retrieval score. Finally, the memory fragments are extracted according to the scores to obtain the memory context, which realizes accurate retrieval of the memory bank. It takes into account both the semantic association between the memory content and the current user's features and the time decay characteristics of the memory content, so that the extracted memory context can accurately and closely reflect the historical information related to the current user's interaction, effectively improving the accuracy and rationality of memory retrieval.

[0112] In an optional embodiment, such as Figure 2 As shown, S6 includes:

[0113] S61. Monitor user interaction status based on real-time interaction logs to obtain the user's current interaction mode. The user's current interaction mode includes a silent state and an active state.

[0114] Optionally, continuous data stream monitoring is performed on the real-time interaction logs. The real-time interaction logs are a time-series data collection recording all interaction behaviors between the user and the virtual partner, covering all interaction-related information such as interaction initiation time, interaction content format, interaction response duration, and input / output data characteristics. Specifically, the real-time interaction logs are parsed according to a fixed time granularity to extract the user's interaction behavior trigger nodes and behavior continuity characteristics. Then, based on preset interaction state judgment rules, the activity level of the user's interaction behavior is quantitatively analyzed. The silent state is judged as no user-initiated interaction behavior within a preset time granularity, and the active state is judged as user-initiated interaction behavior within a preset time granularity. The parsed log data is matched with the state through this judgment rule to finally obtain the user's current interaction mode that can accurately represent the user's current interaction behavior state.

[0115] S62. When the user's current interaction mode is silent, obtain the initiative weight parameter and the user's silence duration from the virtual partner's personality parameter vector.

[0116] Optionally, the user's current interaction pattern is verified. If the pattern indicates a silent state, the generated virtual partner personality parameter vector is retrieved. This vector contains various personality-related parameters, including personality construction parameters, language expression style parameters, and interaction initiative parameters. The initiative weight parameter is the core numerical parameter representing the degree to which the virtual partner initiates interaction behavior. The value of this initiative weight parameter is directly extracted from the virtual partner personality parameter vector. Simultaneously, the real-time interaction log is analyzed in terms of time dimension. The time interval between the timestamp of the user's last proactive interaction behavior and the current monitoring time is extracted. This time interval is the user's silent duration, and its time unit is consistent with the parameter unit used to calculate the probability of proactive interaction.

[0117] S63. Based on the initiative weight parameter and the user's silence duration, calculate the probability of proactive interaction to obtain the probability of proactive interaction triggering. The formula for calculating the probability of proactive interaction triggering is:

[0118]

[0119] in, This represents the probability of triggering an active interaction. For adjustment coefficients, For proactive weighting parameters, User silence duration.

[0120] Optionally, the extracted initiative weight parameters and user silence duration are numerically validated to ensure that both types of parameters are valid values ​​and their dimensions match, and then the adjustment coefficient is retrieved. This coefficient is a pre-set positive constant based on the needs of virtual companion interaction scenarios, and can be flexibly adjusted according to the actual frequency of proactive interaction. Then, the initiative weight parameter... User silence duration With adjustment coefficient The calculation is performed by substituting the values ​​into a given preset formula for calculating the probability of active interaction. Specifically, the adjustment coefficient is calculated first. Proactive weight parameters Duration of silence with users The product of these factors is then negativeened and substituted into the natural exponential function for calculation. Finally, 1 is subtracted from the result of the natural exponential function calculation to obtain the numerical value that represents the probability of triggering active interaction. The probability value ranges from 0 to 1, and can accurately represent the degree of likelihood that a virtual companion will initiate an interaction in the current scenario.

[0121] S64. Compare the probability of active interaction with the preset trigger threshold to obtain the interaction intention guidance signal.

[0122] Optionally, a preset trigger threshold is retrieved. This preset trigger threshold is a fixed value pre-set based on the scenario adaptability and user experience requirements of the virtual companion's proactive interaction. Its value range is consistent with the value range of the proactive interaction trigger probability, serving as the core quantitative basis for determining whether to initiate proactive interaction. The calculated proactive interaction trigger probability is compared with the preset trigger threshold. If the proactive interaction trigger probability is greater than or equal to the preset trigger threshold, an interaction intent guidance signal representing "initiating proactive interaction" is generated. This signal contains basic guidance information such as the type of proactive interaction and content orientation. If the proactive interaction trigger probability is less than the preset trigger threshold, an interaction intent guidance signal representing "not initiating proactive interaction and maintaining a passive response" is generated. Both types of signals are structured feature identifiers that can provide clear behavioral decision guidance for multimodal response generation.

[0123] In the above embodiments, the user's current interaction mode is obtained by monitoring real-time interaction logs, relevant parameters are extracted for the silent state and the probability of triggering active interaction is calculated, and finally, an interaction intention guidance signal is generated by comparing it with a preset threshold. This realizes the decision-making of the virtual companion's active interaction behavior, making the triggering of active interaction compatible with the virtual companion's personality traits and the user's actual interaction state, avoiding meaningless active interaction interference, and filling the interaction gap when the user is silent. This effectively realizes the mode switch from passive response to active interaction, and improves the naturalness and companionship of the virtual companion's interaction.

[0124] In one embodiment, S7 includes:

[0125] S71. Based on the virtual partner's personality parameter vector, interaction intention guidance signals, and memory context, cue words are assembled to obtain structured cue words.

[0126] Optionally, the virtual partner's personality parameter vector, interaction intent guidance signal, and memory context are analyzed separately to extract core representational information. The virtual partner's personality parameter vector extracts core parameters such as personality construction, language expression style, and emotional response weight. The interaction intent guidance signal extracts core guidance information such as interaction behavior type and initiation direction. The memory context extracts core memory information such as user history preferences and key interaction events related to the current interaction. Then, a preset prompt word assembly rule is invoked. This rule is a pre-defined structured combination rule based on the semantic logic of natural language generation and the expression norms of virtual partner interaction, clarifying the arrangement order, expression form, and association logic of various types of information in the prompt words. According to this assembly rule, the three types of core information are transformed into text fragments conforming to natural language expression habits. These text fragments are then spliced ​​and integrated according to semantic association and logical order, while adding expression tone markers that conform to the virtual partner's personality characteristics. Finally, a structured prompt word with clear generation guidance, personality constraints, and memory association is obtained.

[0127] S72. Generate text from the structured prompt words to obtain the text response content.

[0128] Optionally, the structured prompts are formatted to ensure that their text format and semantic logic match the input requirements of the text generation model. If format discrepancies exist, the text format is adjusted according to the model's input specifications. The validated structured prompts are then input into a pre-trained large language model for text generation. Examples of suitable pre-trained generative large language models in natural language processing include GPT (Generative Pre-trained Transformer), LLaMA (Large Language Model Meta AI), or Wenxin Yiyan. These models, based on the input prompt information and combined with the model's semantic understanding and generation capabilities, can generate natural language text that conforms to semantic logic and expression norms. During text generation, the language expression style corresponding to the virtual partner's personality parameter vector is used as a constraint to ensure that the generated text content maintains a high degree of consistency with the virtual partner's personality traits in terms of sentence structure, tone, and expression habits. Simultaneously, information from the remembered context is incorporated to ensure that the generated text content aligns with the user's historical interaction characteristics, ultimately resulting in text response content that meets personality constraints, semantic logic, and interaction requirements.

[0129] S73. Based on the emotion weight parameter in the virtual partner's personality parameter vector, perform multimodal rendering on the text response content to obtain the speech stream and virtual character animation.

[0130] Optionally, an emotion weight parameter is precisely extracted from the virtual partner's personality parameter vector. This parameter is a core numerical parameter characterizing the different emotional expression tendencies and intensities of the virtual partner, directly determining the emotional tone during multimodal rendering. Based on this emotion weight parameter, and combined with the semantics and emotional tendency of the text response content, the rendering style and characteristics of the speech stream and virtual character animation are determined. Specifically, for speech stream rendering, the emotion weight parameter and the text response content are input into the speech synthesis model, for example, using VITS (Variational Inference with Adversarial Learning for End-to-End)... Text-to-Speech, based on end-to-end speech synthesis using variational inference and adversarial learning, or models like Tacotron2 or FastSpeech2, completes the speech stream synthesis and generation. It can adjust the speech rate, tone, timbre, and emotional coloring according to the input text content and emotion parameters to generate a speech stream that matches the virtual companion's personality and the text's emotional tone. For virtual character animation rendering, based on emotion weight parameters and the semantic features of the text response content, it matches preset virtual character action, expression, and facial animation clips. Then, through an animation fusion algorithm, it synchronizes the matched animation clips with the rhythm of the text response content to generate a virtual character animation that highly matches the text content and emotional tone. Finally, it obtains a multimodal rendered speech stream and virtual character animation.

[0131] S74. Encapsulate the text response content, voice stream, and virtual character animation to obtain a multimodal interactive data packet.

[0132] Optionally, the text response content, audio stream, and virtual avatar animation are standardized in data format. The text response content is converted into a preset structured text format, the audio stream's sampling rate and encoding format are uniformly adjusted, and the virtual avatar animation's frame rate, resolution, and file format are standardized to ensure that the formats of the three types of data meet the transmission and display requirements of multimodal interaction. Then, a preset data encapsulation protocol is retrieved. This protocol is a pre-defined encapsulation rule based on the collaborative display requirements of multimodal data, clarifying the storage structure, association identifiers, and synchronization trigger information for each type of data. According to this encapsulation protocol, a unified interaction identifier is added to the three types of standardized data. This identifier enables precise synchronous triggering of the three types of data during the display process. The text response content, audio stream, and virtual avatar animation, after the identifier is added, are then integrated into a single data unit according to the preset storage structure. Simultaneously, this data unit is compressed and verified to ensure the integrity and validity of the data during transmission and use, ultimately resulting in a multimodal interactive data package capable of multimodal collaborative output.

[0133] In the above embodiments, structured prompt words are obtained by assembling multi-dimensional information, and then text response content that conforms to personality constraints is generated based on the prompt words. Subsequently, multimodal rendering of voice and animation is completed by combining emotion weight parameters. Finally, various response data are standardized and encapsulated to obtain multimodal interaction data packages. This realizes the accurate generation and collaborative encapsulation of multimodal responses of virtual companions, so that the response content not only fits the personality characteristics of virtual companions, but also relates to the user's historical memory. At the same time, it realizes the collaborative output of text, voice, and virtual image animation, effectively improving the naturalness, realism and personalization of virtual companion interaction, and bringing users an immersive multimodal interactive experience.

[0134] In the aforementioned AI-based personalized virtual companion interaction method, system, device, and medium, feature vectors of the current user are obtained through feature extraction from multimodal interaction data. These vectors are then dynamically updated to form a global user profile vector, which is mapped to a unique virtual companion personality parameter vector. Simultaneously, based on user features, the memory bank is dynamically updated and accurately retrieved to obtain memory context. Combining virtual companion personality, memory context, and environmental state, behavioral decisions are made to generate interaction intent guidance signals. Finally, based on these signals, memory context, and personality parameters, multimodal responses are generated and encapsulated into interaction data packets. This technical solution achieves user feature-driven personality matching, full lifecycle management of memory, proactive interaction decision-making, and multimodal collaborative output. It solves the problem of homogenized interaction experience through dynamic adaptation of user features and virtual companion personality, achieves contextual coherence of long-term interaction through memory update and retrieval mechanisms, empowers the virtual companion with the ability to proactively initiate interactions through proactive interaction probability calculation, and achieves collaborative control of multi-channel interaction based on multimodal rendering and data encapsulation. This addresses the shortcomings of existing virtual interaction technologies in personalization, long-term memory, proactive interaction, and multimodal collaboration.

[0135] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0136] Based on the same inventive concept, this application also provides a system for implementing the aforementioned AI-based personalized virtual companion interaction method. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations in one or more AI-based personalized virtual companion interaction system embodiments provided below can be found in the limitations of the AI-based personalized virtual companion interaction method described above, and will not be repeated here.

[0137] In one exemplary embodiment, such as Figure 3 As shown, a personalized virtual companion interaction system 10 based on artificial intelligence is provided, including:

[0138] The multimodal feature extraction module 11 is used to extract multimodal features based on multimodal interaction data to obtain the current user feature vector;

[0139] The dynamic profile update module 12 is used to dynamically update the profile based on the current user feature vector to obtain the global user profile vector.

[0140] The personality parameter mapping module 13 is used to map the personality parameters of the character based on the global user profile vector to obtain the personality parameter vector of the virtual partner.

[0141] Memory update module 14 is used to update the memory based on the current user feature vector to obtain the updated memory bank;

[0142] Memory retrieval module 15 is used to perform memory retrieval based on the current user feature vector and the updated memory bank to obtain the memory context;

[0143] Behavior decision module 16 is used to make behavior decisions based on virtual partner personality parameter vector, memory context and current environmental state, and obtain interaction intention guidance signals;

[0144] The multimodal response generation module 17 is used to generate multimodal responses based on interaction intent guidance signals, memory context and virtual partner personality parameter vectors to obtain multimodal interaction data packets.

[0145] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the aforementioned AI-based personalized virtual companion interaction method.

[0146] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned AI-based personalized virtual companion interaction method.

[0147] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0148] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A personalized virtual companion interaction method based on artificial intelligence, characterized in that, The method includes: S1. Extract multimodal features based on multimodal interaction data to obtain the current user feature vector; S2. Dynamically update the user profile based on the current user feature vector to obtain the global user profile vector; S3. Based on the global user profile vector, perform character personality parameter mapping to obtain the virtual partner personality parameter vector; S4. Update the memory based on the current user feature vector to obtain the updated memory bank; S5. Based on the current user feature vector and the updated memory bank, perform memory retrieval to obtain the memory context; S6. Make behavioral decisions based on the virtual partner's personality parameter vector, the memory context, and the current environmental state to obtain an interaction intention guidance signal; S7. Based on the interaction intention guidance signal, the memory context and the virtual partner personality parameter vector, generate a multimodal response to obtain a multimodal interaction data packet.

2. The method according to claim 1, characterized in that, S1 includes: S11. Semantically encode the text data in the multimodal interaction data to obtain a text semantic vector; S12. Perform acoustic feature extraction on the speech data in the multimodal interaction data to obtain an acoustic emotion vector; S13. Visual features are extracted from the video data in the multimodal interaction data to obtain a visual state vector; S14. The text semantic vector, the acoustic emotion vector, and the visual state vector are weighted and fused based on preset weight coefficients to obtain the current user feature vector.

3. The method according to claim 1, characterized in that, S2 includes: S21. Obtain historical interaction logs and extract the global user profile vector of the previous moment from the historical interaction logs. S22. Determine the update learning rate at the current moment according to the preset profile update strategy; S23. Based on the current user feature vector, the global user profile vector of the previous time step, and the updated learning rate, perform an exponential moving average calculation to obtain the global user profile vector.

4. The method according to claim 1, characterized in that, S4 includes: S41. Perform feature value analysis based on the current user feature vector to obtain a memory importance score; S42. When the memory importance score exceeds a preset importance threshold, the current interaction event is vectorized and encoded to obtain a memory embedding vector. S43. Write the memory embedding vector into the memory bank to obtain the updated memory bank.

5. The method according to claim 4, characterized in that, S5 includes: S51. Calculate the semantic similarity between the current user feature vector and the memory embedding vector in the updated memory bank, and obtain the semantic similarity value; S52. Calculate the time decay factor by performing exponential decay calculation based on the time difference between the current time and the memory generation time. S53. The semantic similarity score and the time decay factor are weighted and summed to obtain the memory retrieval score; S54. Based on the memory retrieval score, sort the memory embedding vectors in the updated memory bank in descending order to obtain a descending sorted list, and extract a preset number of memory fragments from the descending sorted list to obtain the memory context.

6. The method according to claim 1, characterized in that, S6 includes: S61. Monitor user interaction status based on real-time interaction logs to obtain the user's current interaction mode; the user's current interaction mode includes a silent state and an active state. S62. When the user's current interaction mode is silent, obtain the initiative weight parameter and the user's silence duration from the virtual partner's personality parameter vector. S63. Based on the initiative weight parameter and the user silence duration, calculate the initiative interaction probability to obtain the initiative interaction trigger probability; the calculation formula for the initiative interaction trigger probability is: in, This represents the probability of being triggered by active interaction. For adjustment coefficients, For proactive weighting parameters, User silence duration; S64. The active interaction trigger probability is compared with the preset trigger threshold to obtain the interaction intention guidance signal.

7. The method according to claim 1, characterized in that, S7 includes: S71. Based on the virtual partner personality parameter vector, the interaction intention guidance signal, and the memory context, the prompt words are assembled to obtain structured prompt words; S72. Generate text from the structured prompt words to obtain text response content; S73. Based on the emotion weight parameter in the virtual partner personality parameter vector, perform multimodal rendering on the text response content to obtain the voice stream and virtual character animation; S74. Encapsulate the text response content, the voice stream, and the virtual character animation to obtain the multimodal interaction data packet.

8. A personalized virtual companion interaction system based on artificial intelligence, characterized in that, The system includes: The multimodal feature extraction module is used to extract multimodal features based on multimodal interaction data to obtain the current user's feature vector; The dynamic profile update module is used to dynamically update the profile based on the current user feature vector to obtain a global user profile vector. The personality parameter mapping module is used to map the character personality parameters based on the global user profile vector to obtain the virtual partner personality parameter vector. The memory update module is used to update the memory based on the current user feature vector to obtain the updated memory bank. The memory retrieval module is used to perform memory retrieval based on the current user feature vector and the updated memory bank to obtain the memory context; The behavior decision module is used to make behavior decisions based on the virtual partner's personality parameter vector, the memory context, and the current environmental state, and to obtain an interaction intention guidance signal. The multimodal response generation module is used to generate multimodal responses based on the interaction intent guidance signal, the memory context, and the virtual partner personality parameter vector to obtain multimodal interaction data packets.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images