Method and system for optimizing interaction feedback of animation characters based on emotional feature clustering

Abstract

The application relates to the technical field of intelligent interaction, and proposes an animation character interaction feedback optimization method and system based on emotional feature clustering, which comprises the following steps: performing multi-modal joint embedding on multi-modal historical interaction data to obtain an emotional joint vector; performing time sequence evolution coding and sliding window interception on the emotional joint vector to obtain a trajectory segment; performing feature clustering and effect evaluation on the trajectory segment to obtain a causal effect strength; taking an interaction scene category as a node and the causal effect strength as an edge to construct an interaction scene graph; embedding a real-time emotional joint vector into the interaction scene graph as a node, and performing correlation reasoning on adjacent nodes to obtain a target emotional evolution trajectory; performing causal modulation decoding on the target emotional evolution trajectory to obtain an interaction feedback parameter, and taking the interaction feedback parameter as incremental data to fine-tune and optimize the interaction scene graph; and the application can improve the efficiency of animation character interaction feedback optimization.

Description

Technical Field

[0001] This invention relates to the field of intelligent interaction technology, and in particular to a method and system for optimizing interactive feedback of animated characters based on emotional feature clustering. Background Technology

[0002] In the application of interactive feedback technologies for anime characters, existing technologies have failed to effectively integrate and deeply analyze users' multimodal interaction data. Relying solely on single-dimensional interaction information for feedback decisions fails to accurately capture the evolutionary patterns of users' emotions and the characteristics of interaction scenarios. Furthermore, the lack of temporal analysis and clustering of users' emotional trajectories makes it difficult to construct an interactive association system that aligns with user behavior. This results in insufficient accuracy in predicting users' emotional trends, and a low degree of consistency between the generated interactive feedback and the user's real-time emotional state.

[0003] Current technologies lack a dynamic interaction scenario reasoning and optimization mechanism. Once feedback parameters are generated, they cannot be incrementally adjusted or systematically optimized based on the user's immediate reactions. This makes it difficult for the adaptability of the interaction feedback system to continuously improve with user interaction behavior. Furthermore, the generation process of feedback parameters lacks efficient graph-based reasoning support, resulting in low overall generation efficiency and control accuracy of interaction feedback. Therefore, improving the accuracy, adaptability, and generation efficiency of interactive feedback for anime characters has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a method and system for optimizing interactive feedback of anime characters based on emotional feature clustering, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides an optimization method for interactive feedback of anime characters based on emotion feature clustering, comprising:

[0006] Step a: Perform multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's sentiment joint vector;

[0007] Step b: Perform temporal evolution encoding on the joint emotion vector to obtain the user's emotion evolution trajectory, and perform sliding window extraction on the emotion evolution trajectory to obtain the user's trajectory segment;

[0008] Step c: Perform feature clustering on the trajectory segment to obtain the user's interaction scenario category, and evaluate the effect of the interaction scenario category to obtain the causal effect strength of the user;

[0009] Step d: Construct the user's interaction scenario graph using the interaction scenario category as nodes and the causal effect strength as edges;

[0010] Step e: Embed the user's real-time sentiment joint vector as a node into the interaction scene graph, and perform association reasoning on the user's neighboring nodes to obtain the user's target sentiment evolution trajectory;

[0011] Step f: Perform causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and use the interaction feedback parameters as incremental data to fine-tune and optimize the interaction scenario map.

[0012] In a preferred embodiment, the step of performing multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's joint sentiment vector includes:

[0013] Obtain users' multimodal historical interaction data;

[0014] The text interaction data in the multimodal historical interaction data is segmented into words to obtain the user's text sentiment vector;

[0015] Acoustic feature analysis is performed on the voice interaction data in the multimodal historical interaction data to obtain the user's voice emotion vector;

[0016] Facial action unit detection is performed on the facial image data in the multimodal historical interaction data to obtain the user's facial expression feature vector;

[0017] The text sentiment vector, the voice sentiment vector, and the facial expression feature vector are time-aligned to obtain the user's joint sentiment vector.

[0018] In a preferred embodiment, the step of performing temporal evolution encoding on the joint emotion vector to obtain the user's emotion evolution trajectory, and then performing sliding window truncation on the emotion evolution trajectory to obtain the user's trajectory segment, includes:

[0019] The user's original emotional evolution trajectory is obtained by sorting the joint emotional vectors in ascending order.

[0020] The user's displacement vector is obtained by performing a difference operation on adjacent vectors in the original emotional evolution trajectory. The magnitude of the displacement vector is marked as the emotional fluctuation amplitude, and the direction of the displacement vector is marked as the emotional migration direction.

[0021] Based on the amplitude of the emotional fluctuation and the direction of the emotional migration, the original emotional evolution trajectory is divided into trend segments to obtain the candidate trajectory segments of the user;

[0022] The integrity of the candidate trajectory segments is verified to obtain the user's trajectory segment.

[0023] In a preferred embodiment, the step of clustering the trajectory segments to obtain the user's interaction scenario categories, and evaluating the effect of the interaction scenario categories to obtain the causal effect strength of the user, includes:

[0024] Principal component analysis was performed on the trajectory segment to obtain the user's reduced-dimensional trajectory features;

[0025] The reduced trajectory features are subjected to mean clustering to obtain the user's feature clusters, and the identifiers of the feature clusters are used as the user's interaction scenario categories;

[0026] Based on the interaction scenario category, the difference between the subsequent emotional vector of the trajectory segment and the emotional base of the trajectory segment is quantized to obtain the effect difference of the user;

[0027] Based on the interaction scenario category, the mean of the effect difference is calculated to obtain the average change of the interaction scenario category, and the average change is used as the causal effect strength of the user.

[0028] In a preferred embodiment, the formula for calculating the intensity of the causal effect is as follows:

[0029] ;

[0030] in, For the first The strength of the causal effect of the interaction scenario category described above. For the first The first interactive scenario category described above The difference in the effect, For the first The maximum effect difference value under the category of the interaction scenario described above. For the first The first under the category of interactive scenarios described above Principal component scores of the reduced-dimensional trajectory features of the said trajectory segments For the first The number of trajectory segments under the category of the interaction scenario described above. The total number of the trajectory segments. The number of emotion vectors for the trajectory segment.

[0031] In a preferred embodiment, constructing the user's interaction scenario graph using the interaction scenario category as nodes and the causal effect strength as edges includes:

[0032] The interaction scene category is mapped to an independent node in the interaction scene graph, and the independent node is assigned a category identifier;

[0033] Based on the location of the emotional node of the causal effect strength, determine the two interaction scene categories associated with the causal effect strength, and use the causal effect strength as the directed edge weight of the interaction scene category;

[0034] A topological association is performed between the category identifier and the directed edge weight to obtain the user's interaction scenario graph.

[0035] In a preferred embodiment, the step of embedding the user's real-time sentiment joint vector as a node into the interaction scene graph and performing association reasoning on the user's neighboring nodes to obtain the user's target sentiment evolution trajectory includes:

[0036] The real-time sentiment joint vector is encapsulated into nodes to obtain the user's real-time graph nodes;

[0037] The real-time graph node is inserted into the interactive scene graph, and the spatial proximity between the real-time graph node and the nodes in the interactive scene graph is evaluated to obtain the neighbor candidate nodes of the real-time graph node.

[0038] Based on the strength of the causal effect, the candidate nodes in the neighborhood are sorted and filtered to obtain the effective neighbors of the real-time graph node;

[0039] The prototype trajectories of the effective neighboring points are time-aligned to obtain the candidate trajectory segments of the user;

[0040] Based on the spatial proximity and the causal effect strength, the candidate trajectory segments are weighted and fused to obtain the user's target emotion evolution trajectory.

[0041] In a preferred embodiment, the step of temporally aligning the prototype trajectories of the effective neighboring points to obtain the candidate trajectory segments of the user includes:

[0042] The prototype trajectory is read from the node attribute data of the effective neighbors, and the prototype trajectory is a sequence of sentiment joint vectors arranged in chronological order;

[0043] Based on the emotional feature matching results between the real-time graph node and the prototype trajectory, the time axis mapping position of the real-time graph node on the prototype trajectory is determined;

[0044] Based on the time axis mapping position, the joint emotional vector sequence of the prototype trajectory is extracted to obtain the initial trajectory continuation segment corresponding to the effective neighbor point;

[0045] The initial trajectory continuation segment is normalized to obtain the same length trajectory continuation segment for the user.

[0046] The same length trajectory segments are smoothed and filtered to obtain the user's candidate trajectory segments.

[0047] In a preferred embodiment, the step of performing causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and using the interaction feedback parameters as incremental data to fine-tune and optimize the interaction scenario graph, includes:

[0048] Based on the strength of the causal effect, the target emotion evolution trajectory is weighted and modulated to obtain the modulated emotion feature sequence of the user;

[0049] The modulated emotional feature sequence is mapped with feedback parameters to obtain the user's interaction feedback parameter sequence;

[0050] Based on the interactive feedback parameter sequence, the user's interactive feedback content is collected in real time to obtain the user's instantaneous reaction data.

[0051] By comparing the instantaneous response data with the joint emotional vector in the target emotional evolution trajectory, the user's feedback response deviation is obtained.

[0052] Based on the feedback response deviation, the causal effect intensity in the interaction scene graph is incrementally updated, and the updated causal effect intensity is reassigned to the directed edges in the interaction scene graph, thereby achieving fine-tuning and optimization of the interaction scene graph.

[0053] To address the aforementioned problems, this invention also provides an animation character interaction feedback optimization system based on emotion feature clustering, the system comprising:

[0054] The multimodal joint embedding module is used to perform multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's sentiment joint vector;

[0055] The temporal trajectory processing module is used to perform temporal evolution encoding on the joint emotion vector to obtain the user's emotion evolution trajectory, and to perform sliding window extraction on the emotion evolution trajectory to obtain the user's trajectory segment;

[0056] The effect evaluation module is used to perform feature clustering on the trajectory segment to obtain the user's interaction scenario category, and to evaluate the effect of the interaction scenario category to obtain the causal effect strength of the user.

[0057] The scene graph construction module is used to construct the user's interaction scene graph with the interaction scene category as nodes and the causal effect intensity as edges;

[0058] The real-time sentiment graph reasoning module is used to embed the user's real-time sentiment joint vector as a node into the interaction scene graph, and to perform association reasoning on the user's neighboring nodes to obtain the user's target sentiment evolution trajectory.

[0059] The graph optimization module is used to perform causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and use the interaction feedback parameters as incremental data to fine-tune and optimize the interaction scene graph.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] 1. This invention performs joint embedding and temporal evolution coding on users' multimodal historical interaction data, and combines feature clustering to achieve accurate segmentation of interaction scenarios. At the same time, it constructs an interaction scenario graph based on the strength of causal effects, and completes neighbor-to-neighbor association inference after embedding the user's real-time emotional vector into the graph. This can accurately deduce the user's target emotional evolution trajectory, making the generated interaction feedback parameters of animation characters more consistent with the user's emotional change patterns, greatly improving the accuracy and relevance of interaction feedback, and making the feedback content highly matched with the user's real-time emotional state.

[0062] 2. After obtaining the interactive feedback parameters, this invention also uses the user's real-time reaction data as incremental data. By dynamically fine-tuning and optimizing the interactive scene graph through the feedback response deviation, it achieves self-iteration and self-optimization of the interactive feedback system, continuously strengthens the ability to predict the evolution of user emotions, and continuously improves the adaptability and fluency of the interactive feedback of animation characters. This allows the emotional fit of the interaction process to be continuously improved. At the same time, by using the fusion analysis of multimodal data and the graph-based reasoning method, it effectively improves the efficiency and stability of the generation of interactive feedback parameters. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating an embodiment of the present invention for optimizing the interactive feedback of anime characters based on emotion feature clustering.

[0064] Figure 2 A functional block diagram of an animation character interaction feedback optimization system based on emotion feature clustering provided in an embodiment of the present invention;

[0065] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0066] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0067] This application provides a method for optimizing the interactive feedback of anime characters based on emotion feature clustering. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0068] Reference Figure 1 The diagram shown is a flowchart illustrating an animation character interaction feedback optimization method based on emotion feature clustering, according to an embodiment of the present invention. In this embodiment, the animation character interaction feedback optimization method based on emotion feature clustering includes:

[0069] Step a: Perform multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's sentiment joint vector;

[0070] In this embodiment of the invention, the step of performing multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's sentiment joint vector includes:

[0071] Obtain users' multimodal historical interaction data;

[0072] The text interaction data in the multimodal historical interaction data is segmented into words to obtain the user's text sentiment vector;

[0073] Acoustic feature analysis is performed on the voice interaction data in the multimodal historical interaction data to obtain the user's voice emotion vector;

[0074] Facial action unit detection is performed on the facial image data in the multimodal historical interaction data to obtain the user's facial expression feature vector;

[0075] The text sentiment vector, the voice sentiment vector, and the facial expression feature vector are time-aligned to obtain the user's joint sentiment vector.

[0076] The system retrieves all multimodal historical interaction data generated by the user within a preset time interval threshold from a preset local storage server or cloud database. This retrieval process is achieved by establishing a communication connection with the storage terminal. The data format follows the preset unified specification for multimodal interaction data, ensuring that the retrieved data packets contain complete text interaction data, voice interaction data, and facial image data. Each type of modal data is accompanied by a corresponding interaction timestamp and device identification information, providing complete basic data support for subsequent modal processing.

[0077] The retrieved text interaction data is segmented. First, invalid characters and redundant symbols in the text data are removed through text preprocessing. Then, a semantic rule-based segmentation algorithm is used to decompose the continuous text sequence into basic lexical units with independent semantics. Subsequently, each basic lexical unit is labeled with sentiment polarity features and mapped with semantic vectors. The mapped semantic features are weighted and fused to finally generate a user text sentiment vector that can accurately represent the emotional tendency of users in the text interaction process. The dimension and feature weight of this vector correspond to the core semantic information of the text sentiment expression.

[0078] Acoustic feature analysis is performed on the voice interaction data. First, the voice signal is pre-emphasized to enhance high-frequency components. Then, the continuous voice signal is divided into continuous short time frames through frame-by-frame windowing. Subsequently, core acoustic features such as fundamental frequency parameters, energy values, Mel frequency cepstral coefficients, and formant frequencies are extracted from each frame of the voice signal. The extracted acoustic features are normalized to eliminate environmental noise interference. Then, a feature mapping model is used to convert the normalized acoustic features into user voice emotion vectors that can represent the intensity and type of voice emotion. The vector features are highly consistent with emotion-related information such as emotional fluctuations and speech rate changes in the voice.

[0079] Facial action unit detection is performed on facial image data. First, a facial key point detection algorithm is used to locate the facial features and contour key points in the image to determine the coordinates of the basic physiological structure of the face. Then, based on the preset facial action unit encoding rules, the facial muscle movement state corresponding to each action unit is identified one by one, detecting the degree of muscle contraction, displacement changes and morphological features. The detected action unit features are sequence integrated and feature enhancement processing, which is transformed into a high-dimensional feature sequence that can fully reflect the details of facial expression changes. Finally, a user expression feature vector that can accurately represent the user's facial expression state is generated. The vector features cover all emotional expression information of facial micro-expressions and macro-expressions.

[0080] The generated text sentiment vector, speech sentiment vector, and facial expression feature vector are subjected to temporal alignment processing. Using the interaction timestamps corresponding to the three types of sentiment vectors as the core alignment benchmark, the sentiment vectors of different modalities are accurately matched in the time dimension through a time axis matching algorithm, eliminating the time offset error in the data collection process of different modalities. Then, the three types of sentiment vectors are horizontally spliced ​​and fused according to the order of interaction to form a multi-dimensional feature sequence containing text semantic features, speech acoustic features, and facial expression features. Finally, a user sentiment joint vector that can comprehensively reflect the emotional state of users' multimodal interactions is obtained. This vector completely retains the core features of the emotional information of each modality and has a unified time dimension attribute.

[0081] The beneficial effects are that by refining the modality and aligning the time sequence of multimodal historical interaction data, the deep integration and unified representation of three types of emotional information—text, voice, and facial expressions—are achieved. This enables the comprehensive and accurate capture of users' real emotional states in different dimensions during the interaction process, providing a high-dimensional and high-quality feature foundation for subsequent interaction scenario segmentation, causal effect intensity calculation, and emotional evolution reasoning. It effectively improves the accuracy, adaptability, and generation efficiency of interactive feedback, ensuring a high degree of fit between the interactive feedback system and users' emotional needs.

[0082] Step b: Perform temporal evolution encoding on the joint emotion vector to obtain the user's emotion evolution trajectory, and perform sliding window extraction on the emotion evolution trajectory to obtain the user's trajectory segment;

[0083] In this embodiment of the invention, the step of performing temporal evolution encoding on the joint emotion vector to obtain the user's emotion evolution trajectory, and then performing sliding window truncation on the emotion evolution trajectory to obtain the user's trajectory segment, includes:

[0084] The user's original emotional evolution trajectory is obtained by sorting the joint emotional vectors in ascending order.

[0085] The user's displacement vector is obtained by performing a difference operation on adjacent vectors in the original emotional evolution trajectory. The magnitude of the displacement vector is marked as the emotional fluctuation amplitude, and the direction of the displacement vector is marked as the emotional migration direction.

[0086] Based on the amplitude of the emotional fluctuation and the direction of the emotional migration, the original emotional evolution trajectory is divided into trend segments to obtain the candidate trajectory segments of the user;

[0087] The integrity of the candidate trajectory segments is verified to obtain the user's trajectory segment.

[0088] Based on the interaction timestamps corresponding to the sentiment joint vectors, all sentiment joint vectors are arranged in ascending order according to their chronological sequence. During the arrangement process, the complete feature information and time identifier of each sentiment joint vector are preserved, so that the arranged sentiment joint vector sequence can fully reflect the continuous change process of user emotions over time, and finally obtain the original sentiment evolution trajectory that can reflect the time evolution law of user emotions.

[0089] The system sequentially extracts two sets of adjacent joint emotional vectors from the original emotional evolution trajectory. It then calculates the difference in the feature dimensions of the two sets of vectors dimension by dimension, integrates the calculated differences in each dimension into a multi-dimensional vector form, and obtains a displacement vector that can reflect the changes in adjacent joint emotional vectors. The magnitude of this displacement vector is calculated, and the calculation result is marked as the emotional fluctuation amplitude. At the same time, the direction of this displacement vector in the feature space is determined and marked as the emotional migration direction. This completes the extraction and marking of the displacement vector, emotional fluctuation amplitude, and emotional migration direction of each set of adjacent joint emotional vectors in the original emotional evolution trajectory.

[0090] Based on the labeled emotional fluctuation amplitude and emotional migration direction, the characteristic change trend of the original emotional evolution trajectory is analyzed. The continuous emotional joint vector sequence with consistent emotional fluctuation amplitude and emotional migration direction is divided into the same characteristic trend segment. During the division process, the consistency and continuity of emotional evolution characteristics within each trend segment are ensured. After the division is completed, each independent characteristic trend segment is used as the basic unit to obtain the user's candidate trajectory segment. Each candidate trajectory segment retains the corresponding time range and emotional evolution characteristic information.

[0091] All candidate trajectory segments undergo integrity verification. During the verification process, each candidate trajectory segment is checked to see if it contains complete emotional evolution trend information, has continuous time series identification, and retains all emotional joint vector feature dimensions. Candidate trajectory segments with missing information, broken time series, or incomplete feature dimensions are eliminated during the verification process. Candidate trajectory segments that pass all verification criteria are determined as the final user trajectory segments, providing a complete and effective data foundation for subsequent feature clustering and interaction scenario classification.

[0092] The beneficial effects are that by temporal encoding of the joint emotional vector and refined extraction of trajectory segments, the temporal and characteristic representation of the user's emotional evolution process is realized, accurately capturing the fluctuation amplitude and migration pattern of user emotions. At the same time, the effectiveness and integrity of the trajectory segments are ensured by trend segment division and integrity verification, providing high-quality data support that fits the user's real emotional changes for subsequent interaction scenario clustering analysis, and improving the accuracy and rationality of interaction scenario category division.

[0093] Step c: Perform feature clustering on the trajectory segment to obtain the user's interaction scenario category, and evaluate the effect of the interaction scenario category to obtain the causal effect strength of the user;

[0094] In this embodiment of the invention, the step of performing feature clustering on the trajectory segment to obtain the user's interaction scenario category, and evaluating the effect of the interaction scenario category to obtain the causal effect strength of the user, includes:

[0095] Principal component analysis was performed on the trajectory segment to obtain the user's reduced-dimensional trajectory features;

[0096] The reduced trajectory features are subjected to mean clustering to obtain the user's feature clusters, and the identifiers of the feature clusters are used as the user's interaction scenario categories;

[0097] Based on the interaction scenario category, the difference between the subsequent emotional vector of the trajectory segment and the emotional base of the trajectory segment is quantized to obtain the effect difference of the user;

[0098] Based on the interaction scenario category, the mean of the effect difference is calculated to obtain the average change of the interaction scenario category, and the average change is used as the causal effect strength of the user.

[0099] The formula for calculating the intensity of the causal effect is as follows:

[0100] ;

[0101] in, For the first The strength of the causal effect of the interaction scenario category described above. For the first The first interactive scenario category described above The difference in the effect, For the first The maximum effect difference value under the category of the interaction scenario described above. For the first The first under the category of interactive scenarios described above Principal component scores of the reduced-dimensional trajectory features of the said trajectory segments For the first The number of trajectory segments under the category of the interaction scenario described above. The total number of the trajectory segments. The number of emotion vectors for the trajectory segment.

[0102] Principal component analysis is performed on all trajectory segments. First, the joint emotional vector features contained in the trajectory segments are centered to eliminate the differences in dimensions and the influence of offset between feature dimensions. Then, the core feature dimensions that can cover the largest variance proportion are extracted through feature space transformation. The high-dimensional trajectory segment features are mapped to the low-dimensional space, and finally, the user dimensionality-reduced trajectory features that can retain the core emotional evolution information of the trajectory segments are obtained.

[0103] The obtained dimensionality reduction trajectory features are subjected to mean clustering. First, a threshold for the number of clusters is preset, and then a corresponding number of dimensionality reduction trajectory features are randomly selected as initial cluster centers. Next, the feature distance between each dimensionality reduction trajectory feature and each initial cluster center is calculated, and it is assigned to the cluster corresponding to the nearest cluster center. Then, the mean of each cluster is updated as the new cluster center. The distance calculation and cluster assignment operations are repeated until the cluster centers no longer change. Each feature cluster is finally assigned a unique identifier, which is the user's interaction scenario category.

[0104] Based on the established interaction scenario categories, the sentiment base of each trajectory segment under the corresponding category and the subsequent sentiment vector within a preset time range after the segment ends are extracted. The corresponding feature dimensions of the subsequent sentiment vector and the sentiment base are calculated dimension by dimension. The calculated difference results are integrated into a complete difference sequence, and finally, a user effect difference that can reflect the degree of sentiment change under the interaction scenario category is obtained.

[0105] Based on the identified interaction scenario categories, the effect differences corresponding to all trajectory segments under that category are extracted. The mean of all effect differences is calculated, and the calculated mean is used as the average change in emotional changes under that interaction scenario category. This average change is the strength of the user causal effect that can quantify the influence of the interaction scenario category on the user's emotional evolution.

[0106] The effect difference for the i-th trajectory segment under the interaction scenario category is obtained by taking the absolute value of the dimension-wise difference between the subsequent sentiment vector of the corresponding trajectory segment and the sentiment base of that trajectory segment. The maximum effect difference under the interaction scenario category is obtained by traversing the effect differences of all trajectory segments in that category and selecting the item with the largest value. The principal component score of the dimensionality-reduced trajectory feature for the i-th trajectory segment under the interaction scenario category is obtained by performing principal component analysis on the sentiment vector sequence of that trajectory segment. This algorithm first centers the sentiment vector sequence, then calculates the covariance matrix, then performs eigenvalue decomposition on the covariance matrix, extracts the eigenvector corresponding to the largest eigenvalue as the principal component, and finally projects the sentiment vector sequence onto the direction of the principal component and calculates the score. The number of trajectory segments under the interaction scenario category is obtained by counting all trajectory segments under that category. The total number of trajectory segments is obtained by summing the number of trajectory segments under all interaction scenario categories. The number of sentiment vectors in a trajectory segment is obtained by counting the number of sentiment vectors contained in a single trajectory segment.

[0107] The causal effect strength of the interaction scenario category is obtained by multiplying the normalized mean of the effect difference under the interaction scenario category, the principal component score of the dimensionality reduction trajectory feature of the corresponding trajectory segment, and the proportion of the number of trajectory segments of that category in turn. This is used to quantify the degree of influence of the interaction scenario category on the user's emotional evolution.

[0108] The larger the normalized mean of the effect difference under the interaction scenario category, the higher the principal component score of the dimensionality reduction trajectory feature of the corresponding trajectory segment, and the larger the proportion of the number of trajectory segments of this category, the greater the numerical value of the causal effect strength of the interaction scenario category.

[0109] The beneficial effects are that principal component analysis achieves efficient dimensionality reduction of trajectory segment features, which not only preserves the core emotional evolution information but also reduces the computational complexity of subsequent calculations. Combined with mean clustering, it completes the accurate classification of interaction scenario categories, which can clearly distinguish the user's emotional evolution patterns under different interaction scenarios. Furthermore, by quantifying the effect difference and solving the mean, the causal effect strength can be obtained, which can intuitively quantify the driving role of each interaction scenario on user emotional changes, providing a reliable quantitative basis for subsequent interaction scenario map construction and emotional evolution inference.

[0110] Step d: Construct the user's interaction scenario graph using the interaction scenario category as nodes and the causal effect strength as edges;

[0111] In this embodiment of the invention, constructing the user's interaction scenario graph using the interaction scenario category as nodes and the causal effect strength as edges includes:

[0112] The interaction scene category is mapped to an independent node in the interaction scene graph, and the independent node is assigned a category identifier;

[0113] Based on the location of the emotional node of the causal effect strength, determine the two interaction scene categories associated with the causal effect strength, and use the causal effect strength as the directed edge weight of the interaction scene category;

[0114] A topological association is performed between the category identifier and the directed edge weight to obtain the user's interaction scenario graph.

[0115] Each independent interaction scenario category is mapped to a graph node. A unique independent node within the interaction scenario graph is matched for each interaction scenario category. During the mapping process, all feature information and clustering identifiers of the interaction scenario category are retained. At the same time, a unique category identifier is configured for each mapped independent node. This category identifier forms a unique correspondence with the feature attributes of the interaction scenario category, ensuring that the node and the interaction scenario category can be accurately matched and quickly traced.

[0116] Extract the emotional evolution correlation information corresponding to the causal effect intensity, locate the spatial positions of the starting and ending emotional nodes corresponding to the occurrence of the causal effect intensity, determine the two corresponding interaction scene categories based on the attributes of the two emotional nodes, clarify the emotional evolution correlation between the two interaction scene categories, and directly assign the value of the causal effect intensity to the connection edge between the two interaction scene categories as the directed edge weight of the connection edge. The weight value is consistent with the causal effect intensity, and the direction of emotional evolution is marked on the directed edge to match the actual emotional migration direction.

[0117] Based on the topological structure specification of the interactive scene graph, the category identifiers of all independent nodes and the directed edge weights of the connecting edges between nodes are processed for global topological association. First, a topological correspondence between category identifiers and node positions is established, and then the directed edge weights are bound to the connection relationships between nodes. This ensures that each category identifier can accurately point to the corresponding node, and each directed edge weight can accurately match the connecting edge between the corresponding node. At the same time, it ensures that the topological relationship between all nodes and edges in the graph conforms to the actual logic of emotional evolution. After completing the topological integration of all information, a complete user interaction scene graph is obtained.

[0118] The beneficial effects are that by mapping interactive scene categories to graph nodes and assigning them unique identifiers, a graph-based concrete representation of interactive scenes is achieved. By combining the strength of causal effects to determine related scenes and configuring directed edge weights, the degree of emotional evolution correlation between different interactive scenes is accurately quantified. Then, the graph is constructed through topological association, so that the interactive scene graph can fully reflect the distribution of each scene and the emotional driving relationship between scenes. This provides a structured and logical graph foundation for subsequent real-time emotional graph reasoning and target emotional evolution trajectory deduction, improving the efficiency and accuracy of emotional evolution reasoning.

[0119] Step e: Embed the user's real-time sentiment joint vector as a node into the interaction scene graph, and perform association reasoning on the user's neighboring nodes to obtain the user's target sentiment evolution trajectory;

[0120] In this embodiment of the invention, the step of embedding the user's real-time sentiment joint vector as a node into the interaction scene graph, and performing association reasoning on the user's neighboring nodes to obtain the user's target sentiment evolution trajectory includes:

[0121] The real-time sentiment joint vector is encapsulated into nodes to obtain the user's real-time graph nodes;

[0122] The real-time graph node is inserted into the interactive scene graph, and the spatial proximity between the real-time graph node and the nodes in the interactive scene graph is evaluated to obtain the neighbor candidate nodes of the real-time graph node.

[0123] Based on the strength of the causal effect, the candidate nodes in the neighborhood are sorted and filtered to obtain the effective neighbors of the real-time graph node;

[0124] The prototype trajectories of the effective neighboring points are time-aligned to obtain the candidate trajectory segments of the user;

[0125] Based on the spatial proximity and the causal effect strength, the candidate trajectory segments are weighted and fused to obtain the user's target emotion evolution trajectory.

[0126] The step of temporally aligning the prototype trajectories of the effective neighboring points to obtain the candidate trajectory segments of the user includes:

[0127] The prototype trajectory is read from the node attribute data of the effective neighbors, and the prototype trajectory is a sequence of sentiment joint vectors arranged in chronological order;

[0128] Based on the emotional feature matching results between the real-time graph node and the prototype trajectory, the time axis mapping position of the real-time graph node on the prototype trajectory is determined;

[0129] Based on the time axis mapping position, the joint emotional vector sequence of the prototype trajectory is extracted to obtain the initial trajectory continuation segment corresponding to the effective neighbor point;

[0130] The initial trajectory continuation segment is normalized to obtain the same length trajectory continuation segment for the user.

[0131] The same length trajectory segments are smoothed and filtered to obtain the user's candidate trajectory segments.

[0132] The user's real-time sentiment joint vector is encapsulated into nodes. During the encapsulation process, all sentiment feature dimensions and real-time timestamp information of the real-time sentiment joint vector are fully preserved. At the same time, the standard data structure and attribute fields of the graph node are configured for the encapsulated carrier, so that the carrier has all the feature attributes of the node in the interaction scene graph. Finally, the user's real-time graph node that can be directly accessed by the interaction scene graph is obtained.

[0133] The generated real-time graph nodes are inserted into the feature space of the interactive scene graph according to the spatial topology rules of the graph. During the insertion process, the topological relationship between the original nodes and edges of the graph remains unchanged. Then, taking the real-time graph node as the center, the feature distance between it and all the original nodes in the interactive scene graph in the feature space is calculated. Based on the calculation results of the feature distance, the spatial proximity between nodes is quantitatively evaluated. The original nodes whose spatial proximity meets the preset judgment criteria are selected and integrated into the neighborhood candidate nodes of the user's real-time graph node.

[0134] Extract the causal effect strength value corresponding to the candidate neighbor node, use this value as the core sorting criterion, sort all candidate neighbor nodes in descending order, and then select the candidate neighbor nodes with higher rankings according to the preset screening quantity threshold, and remove the candidate neighbor nodes with lower rankings. The candidate neighbor nodes retained after screening are determined as the effective neighbors of the user's real-time graph node.

[0135] The prototype trajectory is retrieved from the node attribute data storage area of ​​the effective neighbors. This prototype trajectory is a sequence of sentiment joint vectors arranged chronologically under the interaction scenario category corresponding to the effective neighbors. During the retrieval process, the time axis identifier and the feature integrity of the sentiment vectors of the prototype trajectory are preserved. Then, based on the sentiment feature matching results between the real-time graph nodes and the prototype trajectory, the time axis mapping position of the real-time graph nodes on the prototype trajectory is determined. Based on the time axis mapping position, the sentiment joint vector sequence of the prototype trajectory is truncated backward to obtain the initial trajectory continuation segment corresponding to the effective neighbors. The initial trajectory continuation segment is subjected to length normalization processing to unify the initial trajectory continuation segments corresponding to different effective neighbors to a preset vector sequence length, resulting in the same length trajectory continuation segment for the user. Finally, the same length trajectory continuation segment is subjected to smoothing filtering processing to eliminate the sentiment feature abrupt change points in the sequence, resulting in the user's candidate trajectory segment.

[0136] The spatial proximity and causal effect intensity values ​​corresponding to each candidate trajectory segment are extracted separately. The two values ​​are weighted according to the preset weight ratio to obtain the fusion weight corresponding to each candidate trajectory segment. Then, based on the fusion weight, the sentiment joint vector of all candidate trajectory segments is weighted and fused dimension by dimension. During the fusion process, the time axis order and feature dimension integrity of the sentiment vector are preserved. The calculated sentiment joint vector sequence is arranged in chronological order to finally obtain the user's target sentiment evolution trajectory.

[0137] The beneficial effects are as follows: by encapsulating and inserting nodes into the graph of real-time emotion joint vectors, the system achieves accurate matching between the user's real-time emotional state and the interaction scene graph. By combining spatial proximity assessment and causal effect strength screening, effective neighbor points are obtained, ensuring the relevance and effectiveness of the neighbor point selection. Then, standardized candidate trajectory fragments are obtained through temporal alignment, normalization, and smoothing filtering of prototype trajectories. Finally, the target emotion evolution trajectory is derived based on the weighted fusion of spatial proximity and causal effect strength. This ensures that the derivation results not only fit the user's real-time emotional state but also conform to the historical emotional evolution pattern, significantly improving the accuracy and rationality of the inference of the target emotion evolution trajectory. This provides a precise emotional evolution basis for the generation of subsequent interaction feedback parameters.

[0138] Step f: Perform causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and use the interaction feedback parameters as incremental data to fine-tune and optimize the interaction scenario map.

[0139] In this embodiment of the invention, the step of performing causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and using the interaction feedback parameters as incremental data to fine-tune and optimize the interaction scene graph, includes:

[0140] Based on the strength of the causal effect, the target emotion evolution trajectory is weighted and modulated to obtain the modulated emotion feature sequence of the user;

[0141] The modulated emotional feature sequence is mapped with feedback parameters to obtain the user's interaction feedback parameter sequence;

[0142] Based on the interactive feedback parameter sequence, the user's interactive feedback content is collected in real time to obtain the user's instantaneous reaction data.

[0143] By comparing the instantaneous response data with the joint emotional vector in the target emotional evolution trajectory, the user's feedback response deviation is obtained.

[0144] Based on the feedback response deviation, the causal effect intensity in the interaction scene graph is incrementally updated, and the updated causal effect intensity is reassigned to the directed edges in the interaction scene graph, thereby achieving fine-tuning and optimization of the interaction scene graph.

[0145] Based on the causal effect intensity corresponding to each node in the interaction scene graph, corresponding weight coefficients are matched for the joint emotional vectors at each position in the target emotional evolution trajectory. The feature dimensions of each joint emotional vector are weighted dimension by dimension with the matched weight coefficients. Through weighted calculation, the feature enhancement and weakening processing of the target emotional evolution trajectory is realized, and the weighted modulation operation of the target emotional evolution trajectory is completed. Finally, the user modulated emotional feature sequence with the influence of causal effect intensity is obtained.

[0146] A mapping database is established between modulated emotional feature sequences and interactive feedback parameters. This database contains a one-to-one correspondence between various emotional features and interactive feedback parameters of anime characters. Each set of emotional features in the modulated emotional feature sequence is precisely matched with the content in the mapping database. Based on the matching results, the corresponding interactive feedback parameters are extracted. The extracted interactive feedback parameters are arranged in chronological order according to the modulated emotional feature sequence, and finally, a user interactive feedback parameter sequence with time series attributes is obtained.

[0147] Based on the preset execution rules of each parameter in the interaction feedback parameter sequence, the corresponding interaction feedback execution command is sent to the animation character interaction system to trigger the real-time interaction behavior between the animation character and the user. During the interaction, multi-dimensional interaction data such as the user's text, voice, and facial images are collected synchronously through multi-modal data acquisition devices. The collected multi-dimensional data is analyzed and features are extracted in real time to obtain real-time user response data that can reflect the user's real interaction state.

[0148] The collected instant response data is converted into an emotion joint vector with the same format as the target emotion evolution trajectory. Using the emotion joint vector in the target emotion evolution trajectory as a benchmark, the difference of the converted instant response data emotion joint vector is calculated dimension by dimension. The difference results of each dimension are integrated into a complete difference sequence. This difference sequence can accurately reflect the degree of deviation between the instant response and the target emotion evolution trajectory, and finally obtain the user's feedback response deviation.

[0149] The feedback response deviation is substituted into the preset update calculation rule to incrementally calculate the causal effect strength of the corresponding directed edges in the interaction scene graph, obtaining the updated causal effect strength value. The original causal effect strength value in the graph is deleted, and the updated causal effect strength value is reassigned to the corresponding directed edge, completing the update of the weight of the directed edge. At the same time, the incremental update of the causal effect strength of all related directed edges in the graph is completed according to the same rule. Through precise adjustment of the directed edge weights, the fine-tuning and optimization of the user interaction scene graph is achieved.

[0150] The beneficial effects include weighted modulation of the target emotion evolution trajectory based on the strength of causal effects, making the generation of interaction feedback parameters more closely aligned with the driving influence of the scene on emotions, thus improving the adaptability of feedback parameters. By collecting users' immediate reaction data in real time and calculating the feedback response deviation, the difference between actual interaction and expected evolution can be accurately captured. Then, based on the deviation, the causal effect strength of the interaction scene graph is incrementally updated, realizing the dynamic self-optimization of the graph. This allows the graph to continuously align with users' emotional interaction habits, continuously improving the accuracy of subsequent emotion evolution reasoning and interaction feedback parameter generation, and ensuring the continuous optimization and iterative upgrade of the animation character interaction feedback system.

[0151] like Figure 2 The diagram shown is a functional block diagram of an animation character interaction feedback optimization system based on emotional feature clustering provided in an embodiment of the present invention.

[0152] The anime character interaction feedback optimization system 100 based on emotion feature clustering described in this invention can be installed in an electronic device. Depending on the functions implemented, the anime character interaction feedback optimization system 100 based on emotion feature clustering may include a multimodal joint embedding module 101, a temporal trajectory processing module 102, an effect evaluation module 103, a scene graph construction module 104, a real-time emotion graph inference module 105, and a graph optimization module 106. The modules described in this invention can also be called units, referring to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, stored in the memory of the electronic device.

[0153] In this embodiment, the functions of each module / unit are as follows:

[0154] The multimodal joint embedding module 101 is used to perform multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's sentiment joint vector;

[0155] The temporal trajectory processing module 102 is used to perform temporal evolution encoding on the joint emotion vector to obtain the user's emotion evolution trajectory, and to perform sliding window extraction on the emotion evolution trajectory to obtain the user's trajectory segment.

[0156] The effect evaluation module 103 is used to perform feature clustering on the trajectory segment to obtain the user's interaction scenario category, and to evaluate the effect of the interaction scenario category to obtain the causal effect strength of the user.

[0157] The scene graph construction module 104 is used to construct the user's interaction scene graph with the interaction scene category as the node and the causal effect intensity as the edge.

[0158] The real-time emotion graph reasoning module 105 is used to embed the user's real-time emotion joint vector as a node into the interaction scene graph, and to perform association reasoning on the user's neighboring nodes to obtain the user's target emotion evolution trajectory.

[0159] The graph optimization module 106 is used to perform causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and use the interaction feedback parameters as incremental data to fine-tune and optimize the interaction scene graph.

[0160] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0161] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0162] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0163] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0164] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for optimizing interactive feedback of anime characters based on emotion feature clustering, characterized in that, The method includes: Step a: Perform multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's joint emotion vector. The multimodal historical interaction data includes text interaction data, voice interaction data, and facial image data. Step b: Perform temporal evolution encoding on the joint sentiment vector to obtain the user's sentiment evolution trajectory, and perform sliding window extraction on the sentiment evolution trajectory to obtain the user's trajectory segment, including: The user's original emotional evolution trajectory is obtained by sorting the joint emotional vectors in ascending order. The user's displacement vector is obtained by performing a difference operation on adjacent vectors in the original emotional evolution trajectory. The magnitude of the displacement vector is marked as the emotional fluctuation amplitude, and the direction of the displacement vector is marked as the emotional migration direction. Based on the amplitude of the emotional fluctuation and the direction of the emotional migration, the original emotional evolution trajectory is divided into trend segments to obtain the candidate trajectory segments of the user; The integrity of the candidate trajectory segments is verified to obtain the user's trajectory segments; Step c: Perform feature clustering on the trajectory segments to obtain the user's interaction scenario categories, and evaluate the effect of the interaction scenario categories to obtain the causal effect strength of the user, including: Principal component analysis was performed on the trajectory segment to obtain the user's reduced-dimensional trajectory features; The reduced trajectory features are subjected to mean clustering to obtain the user's feature clusters, and the identifiers of the feature clusters are used as the user's interaction scenario categories; Based on the interaction scenario category, the difference between the subsequent emotional vector of the trajectory segment and the emotional base of the trajectory segment is quantized to obtain the effect difference of the user; The causal effect strength of the interaction scenario category is obtained by multiplying the normalized mean of the effect difference under the interaction scenario category, the principal component score of the dimensionality reduction trajectory feature of the corresponding trajectory segment, and the proportion of the number of trajectory segments of that category in turn. This is used to quantify the degree of influence of the interaction scenario category on the user's emotional evolution. Step d: Construct the user's interaction scenario graph using the interaction scenario category as nodes and the causal effect strength as edges; Step e: Embed the user's real-time sentiment joint vector as a node into the interaction scene graph, and perform association reasoning on the user's neighboring nodes to obtain the user's target sentiment evolution trajectory, including: The real-time sentiment joint vector is encapsulated into nodes to obtain the user's real-time graph nodes; The real-time graph node is inserted into the interactive scene graph, and the spatial proximity between the real-time graph node and the nodes in the interactive scene graph is evaluated to obtain the neighbor candidate nodes of the real-time graph node. Based on the strength of the causal effect, the candidate nodes in the neighborhood are sorted and filtered to obtain the effective neighbors of the real-time graph node; The prototype trajectories of the effective neighboring points are time-aligned to obtain the candidate trajectory segments of the user; Based on the spatial proximity and the causal effect strength, the candidate trajectory segments are weighted and fused to obtain the user's target emotion evolution trajectory; Step f: Perform causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and use the interaction feedback parameters as incremental data to fine-tune and optimize the interaction scenario graph, including: Based on the strength of the causal effect, the target emotion evolution trajectory is weighted and modulated to obtain the modulated emotion feature sequence of the user; The modulated emotional feature sequence is mapped with feedback parameters to obtain the user's interaction feedback parameter sequence; Based on the interactive feedback parameter sequence, the user's interactive feedback content is collected in real time to obtain the user's instantaneous reaction data. By comparing the instantaneous response data with the joint emotional vector in the target emotional evolution trajectory, the user's feedback response deviation is obtained. Based on the feedback response deviation, the causal effect intensity in the interaction scene graph is incrementally updated, and the updated causal effect intensity is reassigned to the directed edges in the interaction scene graph, thereby achieving fine-tuning and optimization of the interaction scene graph.

2. The method for optimizing interactive feedback of anime characters based on emotion feature clustering as described in claim 1, characterized in that, The process of performing multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's joint sentiment vector includes: Obtain users' multimodal historical interaction data; The text interaction data in the multimodal historical interaction data is segmented into words to obtain the user's text sentiment vector; Acoustic feature analysis is performed on the voice interaction data in the multimodal historical interaction data to obtain the user's voice emotion vector; Facial action unit detection is performed on the facial image data in the multimodal historical interaction data to obtain the user's facial expression feature vector; The text sentiment vector, the voice sentiment vector, and the facial expression feature vector are time-aligned to obtain the user's joint sentiment vector.

3. The method for optimizing interactive feedback of anime characters based on emotion feature clustering as described in claim 1, characterized in that, The process of constructing the user's interaction scenario graph, using the interaction scenario category as nodes and the causal effect strength as edges, includes: The interaction scene category is mapped to an independent node in the interaction scene graph, and the independent node is assigned a category identifier; Based on the location of the emotional node of the causal effect strength, determine the two interaction scene categories associated with the causal effect strength, and use the causal effect strength as the directed edge weight of the interaction scene category; A topological association is performed between the category identifier and the directed edge weight to obtain the user's interaction scenario graph.

4. The method for optimizing interactive feedback of anime characters based on emotion feature clustering as described in claim 1, characterized in that, The step of temporally aligning the prototype trajectories of the effective neighboring points to obtain the candidate trajectory segments of the user includes: The prototype trajectory is read from the node attribute data of the effective neighbors, and the prototype trajectory is a sequence of sentiment joint vectors arranged in chronological order; Based on the emotional feature matching results between the real-time graph node and the prototype trajectory, the time axis mapping position of the real-time graph node on the prototype trajectory is determined; Based on the time axis mapping position, the joint emotional vector sequence of the prototype trajectory is extracted to obtain the initial trajectory continuation segment corresponding to the effective neighbor point; The initial trajectory continuation segment is normalized to obtain the same length trajectory continuation segment for the user. The same length trajectory segments are smoothed and filtered to obtain the user's candidate trajectory segments.

5. An animation character interaction feedback optimization system based on emotion feature clustering, characterized in that, The system is used to implement the animation character interaction feedback optimization method based on emotion feature clustering as described in claim 1, the system comprising: The multimodal joint embedding module is used to perform multimodal joint embedding on the user's multimodal historical interaction data to obtain the user's sentiment joint vector. The multimodal historical interaction data includes text interaction data, voice interaction data, and facial image data. The temporal trajectory processing module is used to perform temporal evolution encoding on the joint emotion vector to obtain the user's emotion evolution trajectory, and to perform sliding window extraction on the emotion evolution trajectory to obtain the user's trajectory segment, including: The user's original emotional evolution trajectory is obtained by sorting the joint emotional vectors in ascending order. The user's displacement vector is obtained by performing a difference operation on adjacent vectors in the original emotional evolution trajectory. The magnitude of the displacement vector is marked as the emotional fluctuation amplitude, and the direction of the displacement vector is marked as the emotional migration direction. Based on the amplitude of the emotional fluctuation and the direction of the emotional migration, the original emotional evolution trajectory is divided into trend segments to obtain the candidate trajectory segments of the user; The integrity of the candidate trajectory segments is verified to obtain the user's trajectory segments; The effect evaluation module is used to perform feature clustering on the trajectory segments to obtain the user's interaction scenario categories, and to evaluate the effect of the interaction scenario categories to obtain the causal effect strength of the user, including: Principal component analysis was performed on the trajectory segment to obtain the user's reduced-dimensional trajectory features; The reduced trajectory features are subjected to mean clustering to obtain the user's feature clusters, and the identifiers of the feature clusters are used as the user's interaction scenario categories; Based on the interaction scenario category, the difference between the subsequent emotional vector of the trajectory segment and the emotional base of the trajectory segment is quantized to obtain the effect difference of the user; The causal effect strength of the interaction scenario category is obtained by multiplying the normalized mean of the effect difference under the interaction scenario category, the principal component score of the dimensionality reduction trajectory feature of the corresponding trajectory segment, and the proportion of the number of trajectory segments of that category in turn. This is used to quantify the degree of influence of the interaction scenario category on the user's emotional evolution. The scene graph construction module is used to construct the user's interaction scene graph with the interaction scene category as nodes and the causal effect intensity as edges; A real-time sentiment graph inference module is used to embed the user's real-time sentiment joint vector as nodes into the interaction scene graph, and to perform association inference on the user's neighboring nodes to obtain the user's target sentiment evolution trajectory, including: The real-time sentiment joint vector is encapsulated into nodes to obtain the user's real-time graph nodes; The real-time graph node is inserted into the interactive scene graph, and the spatial proximity between the real-time graph node and the nodes in the interactive scene graph is evaluated to obtain the neighbor candidate nodes of the real-time graph node. Based on the strength of the causal effect, the candidate nodes in the neighborhood are sorted and filtered to obtain the effective neighbors of the real-time graph node; The prototype trajectories of the effective neighboring points are time-aligned to obtain the candidate trajectory segments of the user; Based on the spatial proximity and the causal effect strength, the candidate trajectory segments are weighted and fused to obtain the user's target emotion evolution trajectory; The graph optimization module is used to perform causal modulation decoding on the target emotion evolution trajectory to obtain the user's interaction feedback parameters, and to fine-tune and optimize the interaction scene graph using the interaction feedback parameters as incremental data, including: Based on the strength of the causal effect, the target emotion evolution trajectory is weighted and modulated to obtain the modulated emotion feature sequence of the user; The modulated emotional feature sequence is mapped with feedback parameters to obtain the user's interaction feedback parameter sequence; Based on the interactive feedback parameter sequence, the user's interactive feedback content is collected in real time to obtain the user's instantaneous reaction data. By comparing the instantaneous response data with the joint emotional vector in the target emotional evolution trajectory, the user's feedback response deviation is obtained. Based on the feedback response deviation, the causal effect intensity in the interaction scene graph is incrementally updated, and the updated causal effect intensity is reassigned to the directed edges in the interaction scene graph, thereby achieving fine-tuning and optimization of the interaction scene graph.

Images