Personalized emotion prediction method and system based on physiological signal calibration

Abstract

The application discloses a physiological signal calibration-based personalized emotion prediction method and system, and the method comprises the following steps: structured semantic representation and semantic feature vectors of each material in a material library are extracted through multi-modal analysis; physiological signals and subjective emotion scores of a user are synchronously collected during presentation of selected calibration materials to the user; feature sequences are extracted based on the physiological signals, and predicted emotion scores are obtained, which are compared with the subjective emotion scores and corrected to obtain calibration emotion labels corresponding to each calibration material; the semantic feature vectors are taken as inputs, and the calibration emotion labels are taken as supervision targets for training to obtain a personalized emotion mapping model that is independent of physiological signals; and the semantic feature vectors of recommended materials are input into the model to output preference prediction results of the user for the recommended materials. The application effectively calibrates subjective score noise by using physiological signals, and realizes high-precision and non-invasive personalized emotion perception.

Description

Technical Field

[0001] This invention belongs to the field of emotion recognition technology, specifically relating to a personalized emotion prediction method and system based on physiological signal calibration. Background Technology

[0002] Emotion recognition is a key technology in affective computing, with significant applications in areas such as mental health monitoring, human-computer interaction, and intelligent recommendation. Traditional emotion recognition methods primarily rely on single-modal physiological signals or behavioral features, and suffer from the following technical problems:

[0003] 1. The problem of generalization in emotion perception: Existing emotion recognition models are usually general-purpose and cannot adapt to the huge differences in physiological signals (such as EEG signals) and emotional responses among individuals. Different individuals have significantly different physiological response patterns to the same emotional stimuli, resulting in low recognition accuracy of general-purpose models for specific individuals.

[0004] 2. Subjectivity and unreliability of emotional feedback: Traditional methods mainly rely on user self-reports (such as ratings and labels) as training data. However, self-reports suffer from fatigue, faking, or unclear expression, and lack objective calibration benchmarks. Users may be unable to accurately express their true emotional state for various reasons.

[0005] 3. The static nature of emotion models: Once established, existing models cannot adapt to changes in users' emotional preferences or perceptual habits, lacking lifelong learning capabilities. Users' emotional response patterns change with time, context, and other factors, and static models cannot adapt to these dynamic changes.

[0006] 4. Label noise problem: Subjective emotion rating data contains label noise, that is, the emotion rating provided by users deviates from their actual emotional state. This noise will seriously affect the model training effect and reduce the accuracy of emotion recognition. Summary of the Invention

[0007] This invention addresses the problems of existing general emotion recognition models being unable to adapt to individual differences, unreliable self-reported data, and a lack of dynamic adaptability. It provides a personalized emotion prediction method and system based on physiological signal calibration, which calibrates subjective emotion scores using EEG signals to improve the accuracy and robustness of emotion recognition.

[0008] To achieve the above-mentioned technical objectives, the embodiments of the present invention adopt the following technical solutions.

[0009] In a first aspect, embodiments of the present invention provide a personalized emotion prediction method based on physiological signal calibration, comprising: extracting structured semantic representations of each material in a material library through multimodal analysis to obtain semantic feature vectors of the materials;

[0010] During the presentation of calibration materials selected from the material library to the user, the user's physiological signals and the user's subjective emotional rating of the calibration materials are collected simultaneously.

[0011] Feature sequences are extracted based on the physiological signals, predicted emotion scores are obtained based on the feature sequences, the predicted emotion scores are compared with the subjective emotion scores, and the subjective emotion scores are corrected based on the comparison results to obtain the calibration emotion labels corresponding to each calibration material.

[0012] Using the semantic feature vector of the calibration material as input and the corresponding calibration emotion label as the supervision target, a personalized emotion mapping model that is independent of physiological signals is obtained through training.

[0013] The semantic feature vector of the recommended material is input into the personalized sentiment mapping model, and the model outputs the user's preference prediction result for the recommended material.

[0014] Secondly, embodiments of the present invention provide a personalized emotion prediction system based on physiological signal calibration, including: a semantic feature extraction module, used to extract the structured semantic representation of each material in the material library through multimodal analysis, and obtain the semantic feature vector of the material;

[0015] The physiological signal acquisition and feature extraction module is used to simultaneously acquire the user's physiological signals while presenting calibration materials selected from the material library to the user, and extract feature sequences based on the physiological signals;

[0016] The subjective emotion rating acquisition module is used to simultaneously collect the user's subjective emotion rating for the calibration materials selected from the material library while presenting the calibration materials to the user.

[0017] The subjective emotion rating calibration module is used to extract feature sequences based on the physiological signals, obtain a predicted emotion rating based on the feature sequences, compare the predicted emotion rating with the subjective emotion rating, correct the subjective emotion rating based on the comparison results, and obtain the calibration emotion label corresponding to each calibration material.

[0018] The personalized emotion mapping model construction module is used to train a personalized emotion mapping model that is independent of physiological signals by taking the semantic feature vector of the calibration material as input and the corresponding calibration emotion label as the supervision target.

[0019] The sentiment prediction module is used to input the semantic feature vector of the recommended material into the personalized sentiment mapping model and output the user's preference prediction result for the recommended material.

[0020] It should be understood that the summary section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description.

[0021] Compared with existing technologies, the beneficial technical effects achieved by this invention are as follows: By performing multi-dimensional calibration of subjective emotion scores based on physiological signals, it effectively filters out label noise introduced by user fatigue, expression bias, or arbitrary scoring, providing high-quality and highly reliable supervision signals for training personalized emotion mapping models. It solves the two core problems of large individual differences in EEG signals and unreliable subjective scoring, achieving truly personalized and high-precision emotion perception. Attached Figure Description

[0022] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this application in any way. In the drawings:

[0023] Figure 1 A schematic diagram of a personalized emotion prediction method based on physiological signal calibration is provided for an embodiment. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0025] It should be fully understood that the EEG signals, medical images, and electrophysiological data of users involved in this application are all information and data authorized by the users or fully authorized by all parties. The use of user information should comply with the privacy policies and practices of the industry that are generally considered to meet or exceed the requirements for protecting user privacy. The collection, use, and processing of related data should comply with relevant laws, regulations, and standards, and provide corresponding operation access points for users to choose to authorize or refuse.

[0026] It should be understood that the terms "step 1", "step 2", etc. are used for descriptive purposes only and should not be construed as indicating or implying relative importance, limiting the order of steps, or implicitly specifying the number of technical features indicated.

[0027] Example 1: A personalized emotion prediction method based on physiological signal calibration, comprising:

[0028] Step 1: Extract the structured semantic representation of each material in the material library through multimodal analysis to obtain the semantic feature vector of the material;

[0029] Step 2: While presenting calibration materials selected from the material library to the user, simultaneously collect the user's physiological signals and the user's subjective emotional rating of the calibration materials;

[0030] Step 3: Extract feature sequences based on physiological signals, obtain predicted emotion scores based on feature sequences, compare the predicted emotion scores with subjective emotion scores, correct the subjective emotion scores based on the comparison results, and obtain the calibrated emotion labels corresponding to each calibration material.

[0031] Step 4: Using the semantic feature vector of the calibration material as input and the corresponding calibration emotion label as the supervision target, train the model to obtain a personalized emotion mapping model that is independent of physiological signals.

[0032] Step 5: Input the semantic feature vector of the recommended content into the personalized sentiment mapping model, and output the user's preference prediction result for the recommended content.

[0033] In this embodiment, the materials may include images, videos, audio, etc.

[0034] In some embodiments, structured semantic representations of each material in the material library are extracted through multimodal analysis to obtain semantic feature vectors of the materials. This includes: using a multimodal large model (such as an open-source large language model that meets relevant usage license agreements) to automatically analyze the images, videos, audio, and other materials in the full material library, generating natural language summaries of the materials, and converting them into token sequences. Further, a deep learning-based semantic feature mapping model is constructed to perform deep feature extraction on the token sequence, obtaining a high-dimensional dense semantic feature vector V_static, which serves as an objective numerical representation of the material content.

[0035] In this embodiment of the invention, physiological signals serve as objective biophysical indicators reflecting a user's emotional response to material, and their specific types are not limited to a single electroencephalogram (EEG) signal. Preferred physiological signals include EEG signals, which utilize their high temporal resolution to capture instantaneous emotional fluctuations. Furthermore, depending on the application scenario, physiological signals may be further included or replaced with one or more of the following: electromyography (EMG), electrocardiography / HRV, electrooculography (EOG), geosynthetic skin response (GSR / EDA), and pulse oximetry.

[0036] For example, in scenarios involving the fusion of multimodal physiological information, electroencephalogram (EEG) and electrodermal (ED) signals can be collected simultaneously. EEG characteristics reflect the valence level of emotion, while EEG characteristics reflect the arousal intensity of emotion. This constructs a more comprehensive sequence of objective physiological feature vectors, providing multidimensional objective evidence for the calibration of subjective ratings. This collaborative acquisition and analysis of multiple physiological signals can effectively compensate for potential environmental interference or individual physiological fluctuations that may exist with a single signal source, further improving the accuracy of calibrating emotion labels.

[0037] In some embodiments, the method further includes selecting calibration materials from the material library, specifically including: clustering all materials in the material library based on the word sequence of the materials to divide them into multiple content categories;

[0038] Randomly select materials from each content category according to a preset ratio as calibration materials.

[0039] As an example, the materials are clustered based on the word sequence generated by the aforementioned natural language generalization, and divided into C classes (e.g., C=20). Subsequently, k materials (e.g., k=100) are randomly and evenly selected from each class for each user and presented sequentially. Users can wear a portable EEG cap, and the system synchronously collects their electroencephalogram (EEG) signals.

[0040] In this embodiment, any of the following devices can be used to collect EEG signals, such as:

[0041] 1. Electroencephalogram (EEG) electrode arrays, including scalp electrodes, needle electrodes, flexible electrodes, etc., are used to pick up weak EEG potential signals from the scalp or cortex.

[0042] 2. EEG amplifier: This amplifier amplifies, filters, performs common-mode suppression, and noise reduction on the weak EEG signals acquired by the electrodes to improve the signal-to-noise ratio.

[0043] 3. Wearable EEG acquisition devices, such as portable EEG caps, EEG headbands, EEG earphones, and other integrated devices with built-in electrodes and signal conditioning circuits, are easy to wear for data acquisition.

[0044] 4. The multi-channel EEG acquisition system consists of multiple sets of electrodes, a signal acquisition module, and an analog-to-digital converter (ADC) module, which can simultaneously acquire multiple EEG signals and output them digitally.

[0045] 5. Invasive / semi-invasive EEG acquisition devices, such as cortical electrode arrays and microelectrode arrays, are suitable for intracranial or cortical EEG signal acquisition.

[0046] The embodiments also include preprocessing of the acquired EEG signals, such as base removal, 0.5–45Hz bandpass filtering, and artifact removal.

[0047] In some embodiments, the EEG feature vector FV_eeg of the EEG signal is extracted as an objective physiological indicator of emotional response, specifically including: 1) Frequency domain features: calculate the power spectral density (PSD) and differential entropy (DE) of five frequency bands: delta (1-4Hz), theta (4-8Hz), alpha (8-12Hz), beta (12-30Hz), and gamma (30-45Hz).

[0048] 2) Time-domain features: Extract mean, variance, standard deviation, and peak-to-peak value.

[0049] 3) Nonlinear features: Extract sample entropy and fractal dimension.

[0050] By using a sliding time window, the EEG feature sequence of the user during the period of calibration material stimulation (i.e., the period during which the calibration material is shown to the user) is obtained: EEG_sequence=[FV_eeg_t1,FV_eeg_t2,...,FV_eeg_tn];

[0051] Wherein, FV_eeg_t1 is the EEG feature vector extracted within the first time window, and FV_eeg_tn is the EEG feature vector extracted within the nth time window.

[0052] In this embodiment, during step 2, the user's subjective emotional rating of the calibration material is acquired simultaneously. While watching the calibration material, the user provides a subjective emotional rating, which, for example, may include two dimensions: valence (pleasure) and arousal, with a rating range of 1-9 points, serving as a subjective label for the emotional response.

[0053] In some embodiments, step 3 implements a subjective emotion rating calibration mechanism based on EEG signals. EEG signals are continuously acquired, while subjective emotion ratings are reported only once. This step aims to verify the temporal consistency between the two to assess whether the subjective emotion rating truly reflects the emotional dynamics during the viewing process.

[0054] Some embodiments specifically include:

[0055] 1) Construct a dynamic prediction model: Using subjective emotion ratings as labels, train a lightweight time-series prediction model (such as LSTM or regression model, which can be dynamically updated) to predict the user's overall subjective emotion rating Score_predicted_ti for the calibration material based on the EEG feature sequence EEG_sequence of the current time window, that is, learn the mapping relationship of "instantaneous EEG response → overall emotional impression".

[0056] 2) Consistency index calculation: Input the entire EEG_sequence into the time series prediction model to obtain the predicted sentiment score sequence Score_predicted_sequence=[Score_t1,Score_t2,…,Score_tn], and then analyze it; Score_t1 is the predicted sentiment score in the first time window, and Score_tn is the predicted sentiment score in the nth time window.

[0057] 3) Perform temporal consistency verification based on the predicted emotion score sequence and obtain the time weight based on the verification results; perform intra-class outlier detection based on subjective emotion scores and the emotion score baseline of the category to which the calibration material belongs, and obtain the clustering weight based on the detection results; and obtain the quality weight based on the quality of EEG signals; and calculate the comprehensive confidence weight by multiplying or weighting the three weights.

[0058] Perform the following calibration operation based on the preset interval where the comprehensive confidence weight W lies:

[0059] First confidence interval (e.g., W>0.7): Subjective emotion ratings are directly used as calibrated emotion labels;

[0060] Second confidence interval (0.3≤W≤0.7): Determine the calibrated emotion label based on the comprehensive confidence weight, subjective emotion score, and predicted emotion score sequence.

[0061] As an example, using the formula:

[0062] Calibrated_Score=W×Actual_Score+(1-W)×Mean_Predicted_Score;

[0063] Where Calibrated_Score is the calibrated sentiment label, Actual_Score is the subjective sentiment score, and Mean_Predicted_Score is the mean of the predicted sentiment score sequence.

[0064] The third confidence interval (W<0.3): The mean of the predicted sentiment rating sequence is used as the calibrated sentiment label, and the weight of the loss function of the calibrated sample in the subsequent training of the time series prediction model is reduced.

[0065] The newly calibrated sentiment labels are used to train or fine-tune the final time-series prediction model F. The improved predictive performance of the time-series prediction model F, in turn, can generate a more accurate mean of the predicted sentiment score sequence, which is used for the next round of data calibration, forming a closed loop of continuous improvement.

[0066] In this embodiment, temporal consistency verification is performed based on the predicted emotion score sequence, and the time weight W_temporal is obtained according to the verification result, including: calculating the dispersion of the instantaneous predicted score sequence and the absolute difference between the mean of the instantaneous predicted score sequence Mean_Predicted_Score and the subjective emotion score Actual_Score; if both the dispersion and the absolute difference are lower than the corresponding preset threshold, the subjective emotion score is determined to have temporal consistency, and a time weight higher than the preset time weight threshold is assigned.

[0067] The degree of dispersion can be determined by calculating the standard deviation of the instantaneous predicted rating sequence, thereby determining stability. Smaller fluctuations indicate more consistent emotional responses. For example, if the standard deviation of the predicted emotional rating sequence is compared with a stability threshold, and the standard deviation is less than the stability threshold, the predicted emotional rating is considered stable, and the absolute difference is less than the difference threshold, then the subjective emotional rating is considered reliable. If the predicted emotional rating fluctuates greatly, i.e., the standard deviation is greater than or equal to the stability threshold, or the absolute difference is greater than or equal to the difference threshold, then it is marked as "suspicious" and the predicted emotional rating sequence is discarded. The stability threshold can be set after Z-score normalization (e.g., Z ≤ -2).

[0068] Intra-class outlier detection utilizes prior information about the already clustered (C-class) content to identify anomalous ratings through intra-class comparison. In this embodiment, intra-class outlier detection is performed based on the user's subjective sentiment rating and the baseline sentiment rating of the calibrated content's category. The clustering weight W_cluster is obtained based on the detection results, including:

[0069] Based on the semantic features of the material clustering results, obtain the sentiment baseline distribution Baseline_i (mean and distribution) of the current material's category C_i; compare the current user's subjective sentiment rating of the calibration material with the sentiment baseline distribution to identify whether it is a significantly off-rating (i.e., "outlier").

[0070] For significant deviations in scores, the similarity between the EEG feature vector and the baseline of the group's physiological features corresponding to the same category of materials is calculated. If the similarity of the EEG features is higher than or equal to the similarity threshold, and the deviation of the subjective emotion score from the emotion baseline distribution is greater than the deviation threshold, it is determined that there is a deviation in the subjective evaluation, and the clustering weight W_cluster is reduced. If the EEG features and the subjective emotion score deviate from the group baseline simultaneously, the similarity of the EEG features is lower than the similarity threshold, and the subjective emotion score is a significant deviation score, it is determined that there is a real difference in individual preferences, and the clustering weight W_cluster is increased.

[0071] In the embodiments, the final weight is the product of each factor or a weighted average, such as W=W_temporal×W_cluster×W_quality, where W_quality is the quality weight.

[0072] In this embodiment, step 4 involves training a personalized emotion mapping model using the semantic feature vector of the calibration material as input and the high-quality calibrated emotion labels as the supervised target. This personalized emotion mapping model eliminates the dependence on real-time physiological signals, achieving efficient emotion prediction. Training objective: Minimize the difference between the predicted emotion score and the calibrated emotion label. The error.

[0073] In some embodiments, a personalized emotion mapping model is constructed based on a dynamic joint optimization mechanism of collaborative filtering. The core of this mechanism is to regard the semantic mapping model as a shared "item embedding model" and the personalized emotion prediction model as a "user embedding model", and to achieve collaborative evolution through joint training of multi-user data.

[0074] Personalized emotion mapping models include: Shared semantic mapping models: shared by all users, used to map the semantic feature vectors of calibration materials. The model maps shared embedding vectors to sentiment-discriminating vectors, learning general associations at the group level. Personalized sentiment prediction model: unique to each user, it maps shared embedding vectors to that user's specific preference ratings, learning individual differences.

[0075] Shared semantic mapping model (Φ): Shared parameters Shared by all users, the semantic feature vectors of the calibration materials are mapped into shared embedding vectors that are more emotionally discriminative. =Φ( ; Learn the emotional-semantic associations at the group level.

[0076] Personalized sentiment prediction model ( ): Unique parameters for each user u ,Will Mapped to a user-specific sentiment score: = ( ; ).

[0077] The complete prediction process is as follows: = (Φ( ; ); ).

[0078] Personalized sentiment prediction models can employ cross-networks or gated neural networks.

[0079] In this embodiment, multi-user collaborative initialization optimization is employed. Using calibration data from the first batch of users (e.g., 20 users), a shared semantic mapping model and all personalized sentiment prediction models are trained simultaneously.

[0080] As an example, training data: (user u, material i) , calibrate emotion labels );

[0081] Loss function of personalized sentiment mapping model:

[0082] ;

[0083] R() is the regularization function, λ1 is the shared semantic mapping coefficient, and λ2 is the personalized sentiment coefficient. Let be the parameters of the personalized sentiment prediction model for the u-th user.

[0084] Training process: Synchronous updates and{ , ,…, This enables the shared semantic space to serve all users, while personalized models adapt to individual differences.

[0085] In some embodiments, the personalized emotion mapping model supports cold start processing: when a new user is introduced, the parameters of the shared semantic mapping model are fixed. The corresponding personalized emotion prediction model is trained using only the calibrated emotion tags of the new user.

[0086] In this embodiment, based on the calibrated emotion label of the new user v (steps 2–3), only its personalized parameters are trained. The optimization objective is:

[0087] ,

[0088] The implementation example effectively solves the cold start problem.

[0089] In this embodiment, implicit user feedback (likes, favorites, etc.) is collected, and personalized parameters for the corresponding users are updated periodically. (With fixed shared parameters), the personalized emotion mapping model can adapt to short-term changes in user preferences.

[0090] In this embodiment, a simplified version of steps 2–3 can also be performed periodically (e.g., every three months) or initiated by the user to collect new EEG data, generate high-quality calibrated emotion labels, and apply personalized parameters to the corresponding user. Substantial fine-tuning is performed to combat drift in personalized emotion mapping models.

[0091] The embodiment also includes a full co-evolution step, specifically including:

[0092] Multi-source data integration: Regularly integrate calibrated emotion labels from multiple users and implicit feedback pseudo-labels generated during interactions to construct a full training dataset;

[0093] Joint parameter optimization: The shared semantic mapping model and the personalized sentiment prediction model corresponding to each user are updated synchronously using the full training dataset;

[0094] Feature space transfer: By updating the parameters of the shared semantic mapping model, the common emotional representation features extracted from the group of users are transferred to the personalized prediction logic of the target user, thereby achieving cross-user feature collaboration.

[0095] As an example, large-scale joint optimization can be performed periodically (e.g., every six months) using full user data (calibration data + implicit feedback):

[0096] Loss function for joint optimization of all data:

[0097]

[0098] in Prioritize using calibrated emotion labels When missing, pseudo-labels derived from implicit feedback are used;

[0099] Optimization process: Synchronously update shared parameters With all personalized parameters .

[0100] Synergistic effect:

[0101] 1) Evolution of shared semantic mapping models: learning more accurate emotion representations from group behavior and optimizing the semantic space;

[0102] 2) Collaborative optimization of personalized sentiment prediction models: While updating the shared model, the models of each user are adjusted, and the target user model is indirectly optimized by using similar user data, which reflects the essence of collaborative filtering.

[0103] Compared with the prior art, the embodiments of the present invention have the following significant improvements and beneficial effects:

[0104] 1. High accuracy and deep personalization in emotion recognition: Results: By performing multi-dimensional calibration of subjective ratings based on EEG signals, label noise introduced by user fatigue, expression bias, or arbitrary ratings is effectively filtered out, providing high-quality, high-reliability supervision signals for model training. Combined with a personalized emotion prediction model built independently for each user, the final emotion prediction accuracy is significantly higher than methods relying on general models or uncalibrated subjective ratings. Advantages: It solves the two core challenges of large individual differences in EEG signals and unreliable subjective ratings, achieving truly personalized and high-precision emotion perception.

[0105] 2. Effectively overcomes the cold start problem, providing a superior user experience for new users. Results: When a new user joins, rapid calibration is achieved through clustered and balanced materials. Utilizing a pre-trained shared semantic mapping model, only the parameters of their personalized sentiment prediction model need to be updated to obtain initially usable sentiment mapping capabilities, significantly shortening the model's activation time. Advantages: Decoupling the complex global model training from new user adaptation, and leveraging the "collective intelligence" (shared semantic mapping model) inherent in collaborative filtering, allows new users to quickly enjoy personalized services, improving the system's usability and user acceptance.

[0106] 3. Possesses continuous evolution capabilities, becoming increasingly intelligent with use. Effect: Through a dual mechanism of "short-term implicit feedback fine-tuning" and "long-term full-data joint optimization," the system enables the model to adapt to dynamic changes in user preferences and uncover deeper emotional-semantic connections from group behavior, continuously optimizing the shared semantic space and individual models for all users. Advantages: It breaks the limitation of traditional static models becoming fixed once trained, making the system an organism with lifelong learning capabilities. With increasing usage time, the accuracy and surprisingness of recommendations or recognition will continuously improve.

[0107] 4. Balancing scientific rigor with ease of application. Results: During initial calibration and periodic recalibration, portable EEG devices and rigorous EEG signal processing procedures ensure the objectivity and scientific validity of the data source. In daily applications, the model only needs the semantic features of the data for prediction, eliminating the need for users to wear the device long-term. Advantages: Introducing objective physiological signals for calibration at critical stages, while employing a lightweight, non-invasive prediction method in daily use, significantly lowers the barrier to long-term user adoption while maintaining core effectiveness, thus facilitating the promotion and popularization of the technology.

[0108] With the same inventive concept as the personalized emotion prediction method based on physiological signal calibration provided in the above embodiments, this application also provides a personalized emotion prediction system based on physiological signal calibration, including: a semantic feature extraction module, a physiological signal acquisition and feature extraction module, a subjective emotion score acquisition module, a subjective emotion score calibration module, a personalized emotion mapping model construction module, and an emotion prediction module.

[0109] The semantic feature extraction module is used to extract the structured semantic representation of each material in the material library through multimodal analysis, and obtain the semantic feature vector of the material.

[0110] The physiological signal acquisition and feature extraction module is used to simultaneously acquire the user's physiological signals while presenting calibration materials selected from the material library to the user, and extract feature sequences based on the physiological signals;

[0111] The subjective emotion rating acquisition module is used to simultaneously acquire the user's subjective emotion rating for the calibration materials selected from the material library while presenting the calibration materials to the user.

[0112] The subjective emotion rating calibration module is used to extract feature sequences based on the physiological signals, obtain predicted emotion scores based on the feature sequences, compare the predicted emotion scores with the subjective emotion scores, correct the subjective emotion scores based on the comparison results, and obtain calibration emotion labels corresponding to each calibration material.

[0113] The personalized emotion mapping model construction module is used to train a personalized emotion mapping model that is independent of physiological signals by taking the semantic feature vector of the calibration material as input and the corresponding calibration emotion label as the supervision target.

[0114] The sentiment prediction module is used to input the semantic feature vector of the recommended materials into the personalized sentiment mapping model and output the user's preference prediction result for the recommended materials.

[0115] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, or a tablet computer, or any combination of these devices.

[0116] The above provides a detailed description of the personalized emotion prediction method and system based on physiological signal calibration provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the concept of this application and should not be construed as limiting the scope of protection of this application.

Claims

1. A personalized emotion prediction method based on physiological signal calibration, characterized in that, include: The structured semantic representation of each material in the material library is extracted through multimodal analysis to obtain the semantic feature vector of the material; During the presentation of calibration materials selected from the material library to the user, the user's physiological signals and the user's subjective emotional rating of the calibration materials are collected simultaneously. Feature sequences are extracted based on the physiological signals, predicted emotion scores are obtained based on the feature sequences, the predicted emotion scores are compared with the subjective emotion scores, and the subjective emotion scores are corrected based on the comparison results to obtain the calibration emotion labels corresponding to each calibration material. Using the semantic feature vector of the calibration material as input and the corresponding calibration emotion label as the supervision target, a personalized emotion mapping model that is independent of physiological signals is obtained through training. The semantic feature vector of the recommended material is input into the personalized sentiment mapping model, and the model outputs the user's preference prediction result for the recommended material.

2. The personalized emotion prediction method according to claim 1, characterized in that, Structured semantic representations of each material in the material library are extracted through multimodal analysis to obtain semantic feature vectors of the materials, including: The multimodal large model is used to automatically analyze all materials in the material library, generate natural language summaries of the materials, and convert them into word sequence. A semantic feature mapping model is constructed to extract features from the word sequence and obtain semantic feature vectors.

3. The personalized emotion prediction method according to claim 1, characterized in that, The method further includes selecting calibration materials from the material library, specifically including the following steps: The material is clustered based on the word sequence of the material library to divide it into multiple content categories; calibration materials are randomly selected from each content category according to a preset ratio.

4. The personalized emotion prediction method according to claim 1, characterized in that, The physiological signals include electroencephalogram (EEG) signals; Obtain the calibration sentiment tags corresponding to each calibration material, including: Extract the EEG feature vector from the EEG signal, input it into the pre-trained time-series prediction model, and obtain the instantaneous predicted emotion score sequence based on physiological response; Temporal consistency verification is performed based on the predicted emotion rating sequence, and time weights are obtained based on the verification results; intra-class outlier detection is performed based on subjective emotion ratings and the emotion rating baseline of the category to which the calibration material belongs, and clustering weights are obtained based on the detection results; and quality weights are obtained based on EEG signal quality; the comprehensive confidence weight is calculated by multiplying or weighting the three weights. Perform the following calibration operation based on the preset interval where the comprehensive confidence weight lies: First confidence interval: The subjective emotion score is directly used as the calibrated emotion label; Second confidence interval: Determine the calibrated emotion label based on the comprehensive confidence weight, subjective emotion score, and instantaneous predicted emotion score sequence; Third confidence interval: The mean of the instantaneous predicted emotion score sequence is used as the calibration emotion label, and the weight of the loss function of the calibration material in the subsequent training of the time series prediction model is reduced; the confidence weight values ​​of the first confidence interval, the second confidence interval and the third confidence interval decrease in sequence.

5. The personalized emotion prediction method according to claim 4, characterized in that, Temporal consistency verification is performed based on the instantaneously predicted sentiment score sequence, and time weights are obtained based on the verification results, including: Calculate the dispersion of the instantaneous predicted rating sequence and the absolute difference between the mean of the instantaneous predicted rating sequence and the subjective emotion rating; If both the degree of dispersion and the absolute difference are lower than the corresponding preset threshold, the subjective emotion score is determined to have temporal consistency and is assigned a time weight higher than the preset time weight threshold.

6. The personalized emotion prediction method according to claim 4, characterized in that, Intra-class outlier detection is performed based on subjective sentiment scores and a baseline sentiment score for the category to which the calibration material belongs. Clustering weights are obtained based on the detection results, including: Based on the semantic feature clustering results of all materials in the material library, obtain the sentiment baseline distribution of the category to which the current calibration material belongs; The current user's subjective emotion rating for the calibration material is compared with the emotion baseline distribution to identify whether there is a significant deviation in the rating. For the significant deviation score, the similarity between the EEG feature vector and the group EEG feature baseline corresponding to the same category of material is calculated; If the similarity of EEG features is higher than or equal to the similarity threshold and the deviation between the subjective emotion score and the emotion baseline distribution is greater than the deviation threshold, it is determined that there is a bias in the subjective evaluation, and the clustering weight is reduced. If the similarity of EEG features is lower than the similarity threshold, and the subjective emotion score is significantly different from the score, then it is determined that there are real differences in individual preferences, and the cluster weight is increased.

7. The personalized emotion prediction method according to claim 1, characterized in that, The personalized emotion mapping model includes: Shared semantic mapping model: Shared by all users, it is used to map the semantic feature vectors of calibration materials into shared embedding vectors with emotion discrimination, and learn general associations at the group level; Personalized sentiment prediction model: unique to each user, used to map the shared embedding vector to the user's unique preference rating, learning individual differences.

8. The personalized emotion prediction method according to claim 7, characterized in that, The personalized emotion mapping model supports cold start processing: when a new user is introduced, the parameters of the shared semantic mapping model are fixed, and the parameters of the corresponding personalized emotion prediction model are trained only using the new user's calibrated emotion labels.

9. The personalized emotion prediction method according to claim 1, characterized in that, After outputting the user's preference prediction result for the recommended materials, the process also includes a dynamic iteration step: collecting the user's implicit feedback behavior on the recommended materials and periodically fine-tuning the parameters of the personalized emotion mapping model to adapt to the dynamic drift of user preferences.

10. A personalized emotion prediction system based on physiological signal calibration, characterized in that, include: The semantic feature extraction module is used to extract the structured semantic representation of each material in the material library through multimodal analysis and obtain the semantic feature vector of the material; The physiological signal acquisition and feature extraction module is used to simultaneously acquire the user's physiological signals while presenting calibration materials selected from the material library to the user, and extract feature sequences based on the physiological signals; The subjective emotion rating acquisition module is used to simultaneously acquire the user's subjective emotion rating for the calibration materials selected from the material library while presenting the calibration materials to the user. The subjective emotion rating calibration module is used to extract feature sequences based on the physiological signals, obtain a predicted emotion rating based on the feature sequences, compare the predicted emotion rating with the subjective emotion rating, correct the subjective emotion rating based on the comparison results, and obtain the calibration emotion label corresponding to each calibration material. The personalized emotion mapping model construction module is used to train a personalized emotion mapping model that is independent of physiological signals by taking the semantic feature vector of the calibration material as input and the corresponding calibration emotion label as the supervision target. The sentiment prediction module is used to input the semantic feature vector of the recommended material into the personalized sentiment mapping model and output the user's preference prediction result for the recommended material.

Images