A user emotional state management content providing method based on facial recognition and an electronic device

Abstract

The present application relates to a kind of content providing method and electronic equipment based on user emotional state management of facial recognition.The method includes: extracting facial feature points according to the video of shooting user;According to the position, distance and angle of facial feature points in single frame image, determine static expression state;According to the position variation of facial feature points between continuous multiple frames, determine expression change mode;According to static expression state and expression change mode, generate emotional feature vector and input emotional classification model, determine the current emotional state of user and its intensity level;Get target emotional state, calculate the type difference and intensity difference between them, generate feedback content accordingly;Feedback content is sent to user terminal and continuously monitors emotional state change;According to the monitoring result, dynamically adjust the intensity parameter and expression mode of feedback content, until reaching target emotional state.The present application can monitor user emotion in real time and dynamically adjust feedback content, effectively manage user emotional state.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence and image processing technology, and in particular to a method and electronic device for providing user emotion state management content based on facial recognition. Background Technology

[0002] In recent years, facial recognition technology has been widely used in various fields, playing a particularly important role in analyzing users' emotional states. Traditional emotion analysis methods mainly rely on subjective data collection through questionnaires or interviews, but these methods are time-consuming, labor-intensive, and lack objectivity. To address these issues, methods using facial recognition technology for emotion analysis are receiving increasing attention.

[0003] Facial recognition-based emotion analysis technology analyzes a user's emotional state by extracting facial feature points. It can capture subtle facial muscle movements to determine the type and intensity of emotions. Specifically, by analyzing parameters such as the position, distance, and angle of facial feature points like the eyes, mouth, and eyebrows, it quantifies facial expression changes and infers the user's emotional state. Traditional facial recognition technology is primarily used in security and identity authentication, but in recent years it has been gradually evolving towards improving user experience. For example, it can provide personalized content based on a user's emotional state, helping to stabilize or improve their mood. This technology has broad application value in areas such as stress management, mental health monitoring, education, and entertainment.

[0004] Facial recognition-based emotion analysis technology, leveraging deep learning algorithms, can achieve high-precision emotion state analysis. It boasts the advantage of stably extracting facial feature points from continuous video frames and tracking their temporal changes, thereby enabling real-time analysis of emotional states. These technological advancements make it possible to generate personalized feedback content for users, and thus effectively manage their emotional states.

[0005] However, existing technologies lack a complete solution that can dynamically generate and adjust feedback content based on the difference between the user's current emotional state and the target emotional state in order to effectively guide the user's emotions toward the target state. Summary of the Invention

[0006] The purpose of this embodiment is to determine the difference between a user's current emotional state and a target emotional state through facial recognition-based emotion state analysis, and to generate appropriate feedback content accordingly, thereby providing a content delivery method and electronic device that can effectively manage the user's emotional state.

[0007] A content provision method according to one embodiment includes: acquiring a video stream of a user; detecting a user's facial region from each frame of the video stream; extracting multiple facial feature points within the facial region, the facial feature points including feature points corresponding to the eyes, eyebrows, mouth, nose, and jawline; determining the user's static expression state in the current frame based on the position, distance, and angle of the facial feature points in a single frame; determining the user's expression change pattern based on the positional change of the facial feature points between consecutive frames; generating an emotion feature vector based on the static expression state and the expression change pattern; inputting the emotion feature vector into a pre-trained emotion classification model to determine the user's current emotion state and its intensity level; acquiring a target emotion state and its target intensity level; calculating the type difference and intensity difference between the current emotion state and the target emotion state; selecting and generating feedback content from a preset content library based on the type difference and intensity difference, the feedback content including at least one of video, audio, or text; sending the feedback content to a user terminal and continuously monitoring changes in the user's emotion state; dynamically adjusting the intensity parameters and expression method of the feedback content based on the monitoring results until the user's emotion state reaches the target emotion state.

[0008] In the content provision method of this embodiment, the emotional state is at least one of the following emotional types: joy, sadness, anger, fear, anxiety, and depression, with each emotional type corresponding to an intensity level of 1 to 5.

[0009] In the content provision method of this embodiment, the step of determining the static expression state based on the position, distance and angle of the facial feature points in a single frame image includes: calculating the vertical distance between eye feature points and quantifying the degree of eyelid opening and closing; calculating the displacement of the corner of the mouth feature points relative to the baseline and quantifying the degree of upward or downward drooping of the corner of the mouth; calculating the distance change between eyebrow feature points and quantifying the degree of eyebrow contraction or relaxation; and calculating the activity of facial motion units and quantifying the subtle movements of facial muscles.

[0010] In the content provision method of this embodiment, the step of determining the expression change pattern based on the positional change of the facial feature points between consecutive frames includes: calculating the displacement, change rate, and change acceleration of each feature point between consecutive frames; calculating the cumulative amount and change rate of expression change within a preset time window; and quantifying the temporal change features of enhanced smile or deepened frown.

[0011] In the content provision method of this embodiment, the emotion feature vector includes: the coordinate changes of the facial feature points; the activity level of the user's facial action units; and texture feature values ​​corresponding to facial skin, wrinkles, and light and dark patterns. The texture feature values ​​include Local Binary Pattern (LBP) feature values ​​and Gabor filter response values.

[0012] In the content delivery method of this embodiment, the step of dynamically adjusting the intensity parameters and expression of the feedback content based on the monitoring results includes: when the intensity difference between the current emotional state and the target emotional state is greater than a preset threshold, increasing the volume, playback speed, or screen brightness of the feedback content; when the intensity difference is less than the preset threshold, gradually decreasing the volume, playback speed, or screen brightness of the feedback content; when the current emotional state and the target emotional state belong to different emotional types, switching the type of the feedback content; and personalizing the expression of the feedback content based on the user's preference information and usage history.

[0013] In the content provision method of this embodiment, the step of generating feedback content based on the type difference and intensity difference includes: selecting a corresponding visual stimulus type, auditory stimulus type or text stimulus type based on the type difference; determining the stimulus intensity, playback duration and repetition number of the feedback content based on the intensity difference; and synchronizing the visual stimulus with the auditory stimulus in time to generate feedback content in the form of a composite stimulus.

[0014] The content provision method in this embodiment further includes: when the user's emotional state does not improve, repeatedly providing the feedback content and gradually increasing the intensity of the stimulus; when the user's emotional state reaches the target emotional state or the number of repeated provision reaches a preset upper limit, stopping the provision of the feedback content.

[0015] A content delivery system according to one embodiment includes: a facial recognition module, configured to acquire a video stream of a user, detect a user's facial region from each frame of the video stream, and extract multiple facial feature points within the facial region; an emotion state analysis module, configured to determine a static expression state based on the position, distance, and angle of the facial feature points in a single frame, determine an expression change pattern based on the positional changes of the facial feature points between consecutive frames, generate an emotion feature vector based on the static expression state and the expression change pattern, input the emotion feature vector into a pre-trained emotion classification model, and determine the user's current emotion state and its intensity level; a content generation module, configured to acquire a target emotion state and its target intensity level, calculate the type difference and intensity difference between the current emotion state and the target emotion state, generate feedback content based on the type difference and intensity difference, the feedback content including at least one of video, audio, or text; and an output module, configured to send the feedback content to a user terminal, continuously monitor changes in the user's emotion state, and dynamically adjust the intensity parameters and expression method of the feedback content based on the monitoring results until the user's emotion state reaches the target emotion state.

[0016] In the content provision system of this embodiment, the output module sends the feedback content to the user terminal through real-time streaming or local push, and dynamically adjusts the resolution, bit rate and transmission rate of the feedback content according to the network status.

[0017] An electronic device according to one embodiment includes: a memory for storing a computer program; and a processor connected to the memory for executing the computer program to implement the method described above.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: The facial recognition-based emotion analysis and feedback system provided by this invention extracts facial feature points of users and analyzes their emotional state accordingly, thereby providing feedback content that helps achieve the target emotional state, thus effectively managing the user's emotional state.

[0019] According to one embodiment, this system analyzes emotional states by subdividing them into various emotional types and different intensity levels, enabling more accurate emotional assessment. Furthermore, feedback can be presented in a user-friendly manner through video, audio, text, and other formats, thereby narrowing the gap between the user's current emotional state and the target emotional state.

[0020] By combining static facial expression analysis with dynamic facial expression change pattern analysis, this invention can more accurately capture subtle changes and trends in emotional states, significantly improving the accuracy and reliability of emotion recognition. By generating a multi-dimensional emotion feature vector that includes coordinate changes, action unit activity, and texture feature values, it can more comprehensively reflect the user's facial expression features, greatly improving the judgment accuracy of the emotion classification model.

[0021] By simultaneously calculating differences in emotion type and intensity, and adaptively selecting feedback content from a preset content library based on these dual differences, this invention enables more targeted and personalized emotion guidance. By continuously monitoring changes in emotional state and dynamically adjusting the intensity parameters of the feedback content according to preset thresholds until the target emotional state is achieved, a complete closed-loop feedback management mechanism is formed, effectively improving the efficiency of emotion management and user experience.

[0022] This invention can monitor changes in a user's emotional state in real time and dynamically adjust the intensity and expression of feedback content, thereby improving the user experience. This effect can be widely applied in fields such as stress management, mental health monitoring, education, and entertainment.

[0023] The effects of the present invention are not limited to those listed above, but also include all other effects that can be derived by those skilled in the art based on the present invention. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is an example of a face recognition-based content delivery system according to one embodiment.

[0025] Figure 2a This is a schematic diagram used to illustrate deep learning computation methods using neural networks.

[0026] Figure 2b This is a schematic diagram illustrating a learning and reasoning method of a neural network according to an embodiment.

[0027] Figure 3 This is a schematic diagram illustrating a content generation framework according to one embodiment.

[0028] Figure 4 This is a schematic diagram illustrating the operation of each module according to one embodiment.

[0029] Figure 5 This is a flowchart example of a method for providing content based on facial recognition for managing user emotional states, according to one embodiment.

[0030] Figure 6 This is an example of an electronic device according to one embodiment. Detailed Implementation

[0031] The following drawings disclose several embodiments of the present invention. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential. Furthermore, for the sake of simplicity, some conventional structures and components will be shown in the drawings in a simple schematic manner.

[0032] System Overview

[0033] Figure 1 An example of a face recognition-based content delivery system (100) according to one embodiment is shown.

[0034] refer to Figure 1 The system (100) may include an electronic device (110) and a user terminal (130). The electronic device (110) may receive facial images of the user from the user terminal (130), measure the user's emotional state in real time, and analyze the difference between the user's emotional state and the target emotional state, thereby providing feedback content. The emotional state may be designed to be managed in multiple emotion types and intensity levels.

[0035] In some embodiments of the present invention, the content provision method provided according to the embodiments of this disclosure can be applied to multiple fields such as daily emotion management, addiction and trauma treatment support, and personalized emotion management platforms. Different steps in each field, including emotion state detection, feedback content provision, real-time monitoring, and repetition control, can be implemented progressively. Specific content in each field can be customized according to user characteristics and intended use.

[0036] According to some embodiments, the user terminal (130) may be a smartphone, mobile phone, personal computer, laptop, netbook, tablet computer, personal digital assistant (PDA), digital camera, game console, MP3 player, personal multimedia player (PMP), e-book, navigation device or home appliance, but is not limited thereto.

[0037] According to some embodiments, the electronic device (110) can detect a user's facial region from a captured video and extract multiple facial landmarks within that region. The electronic device (110) can automatically correct the resolution and lighting conditions of the input video to improve the accuracy of facial region detection. The electronic device (110) employs a deep learning-based facial detection algorithm, enabling stable detection of facial regions under various facial angles and expressions. The electronic device (110) can extract multiple types of facial landmarks, such as eyes, eyebrows, mouth, nose, and jawline, frame by frame. The electronic device (110) can track temporal patterns of facial expression changes by observing changes in the positions of landmarks in consecutive video frames. The extracted facial landmark information can be used as input data for subsequent emotion state analysis.

[0038] Facial recognition module

[0039] In some embodiments of the present invention, the electronic device (110) may include a face recognition module (410), an emotion state analysis module (430), a content generation module (450), and an output module (470).

[0040] In this specification, "module" can refer to the hardware that implements the functions and operations corresponding to the names described herein, computer program code capable of performing specific functions and operations, or an electronic recording medium containing computer program code that performs specific functions and operations, such as a processor or microprocessor. In other words, "module" can be understood as a functional and / or structural combination of hardware that implements the technical concept of this invention and / or software that drives the hardware to run. For example, the facial recognition module (410), the emotion state analysis module (430), the content generation module (450), and the output module (470) can all be implemented by one or more processors integrated within the electronic device (110) (e.g., Figure 6 The processor (630) shown is used to implement this.

[0041] In some embodiments of the present invention, the face recognition module (410) is capable of detecting the user's facial region from a video captured of the user. The face recognition module (410) can apply a deep learning-based face detector to each frame of the input video. By using the deep learning-based face detector, the face recognition module (410) can automatically identify the location region of the user's face in the video. The face recognition module (410) employs multi-resolution scaling technology to improve the accuracy of face region detection in input videos of different resolutions. The face recognition module (410) also applies an illumination correction algorithm, enabling it to stably detect facial regions even in environments with large changes in illumination. Furthermore, the face recognition module (410) can reliably detect facial regions under various environmental conditions such as partial occlusion, changes in facial orientation, and wearing glasses. The detected facial region information can be used as input data for the subsequent feature point extraction stage.

[0042] In several embodiments of the present invention, the face recognition module (410) can automatically identify the region where the user's face is located in the input video. This module can detect potential candidate facial regions by analyzing various visual features such as pixel distribution, color information, and contour lines of the input video. The face recognition module (410) can apply a neural network-based classifier to the detected candidate regions to ultimately determine the actual facial region. The module can also calculate the position and size of the facial region frame by frame and use the obtained facial region coordinates as input data for a feature point extraction algorithm.

[0043] In some embodiments of the present invention, the face recognition module (410) can provide the detected facial region information as input data to the subsequent feature point extraction stage. The module can store the coordinate values ​​and size information of the detected facial region in a memory buffer, and this stored information is then passed to the feature point extraction algorithm as a reference region for keypoint extraction. In addition, the face recognition module (410) can also simultaneously provide metadata such as frame number and timestamp along with the facial region information.

[0044] According to some embodiments, the face recognition module (410) is able to extract multiple facial landmarks within the aforementioned facial region. The module employs a landmark extraction algorithm to generate multiple facial feature points within the facial region, such as eyes, eyebrows, mouth, nose, and jawline. The face recognition module (410) can calculate the position of each landmark as two-dimensional or three-dimensional coordinate values. The extracted feature point information can be used as basic data for expression analysis and emotion state estimation.

[0045] In other embodiments, the face recognition module (410) can apply a keypoint extraction algorithm to the detected facial region to generate multiple facial feature points, including eyes, eyebrows, mouth, nose, and jawline. The module utilizes a CNN-based keypoint extraction network to automatically detect feature points for each facial component. During feature point extraction, the face recognition module (410) can calculate a confidence score for each feature point, thus selecting only highly reliable feature points even in noisy environments. By analyzing the relative distances, angles, and arrangement patterns between different landmarks, the module can quantitatively capture subtle changes in facial expressions.

[0046] In some embodiments of the present invention, each facial feature point can be represented by two-dimensional or three-dimensional coordinate values. The facial recognition module (410) can record the position of each landmark as two-dimensional coordinate values ​​(x, y) or three-dimensional coordinate values ​​(x, y, z). The three-dimensional coordinate values ​​can be obtained through a stereo camera, a deep learning-based depth estimation technique, or a multi-view image processing method. The coordinate values ​​of each feature point are arranged frame by frame to facilitate tracking positional changes over time. These coordinate data can be used as input for subsequent geometric parameter calculations and the generation of emotion feature vectors.

[0047] In some embodiments of the present invention, the face recognition module (410) can integrate a deep learning-based face detector and a feature point extraction algorithm, enabling it to stably extract feature points not only within a single frame but also across consecutive video frames. This module maintains temporal consistency by tracking the changing trajectories of feature point positions across consecutive video frames. The deep learning-based face detector can detect facial regions in each frame, while the feature point extraction algorithm can generate facial landmarks within those regions. By analyzing the movement and change patterns of feature points between frames, the face recognition module (410) can stably capture subtle changes in facial expressions.

[0048] In some embodiments, a deep learning-based detector can detect facial regions frame by frame, while a feature point extraction algorithm can generate facial landmarks in each frame. The detector can employ various neural network structures such as CNN, R-CNN, and YOLO to quickly and accurately detect facial regions from the input video. The feature point extraction algorithm can predict the positions of individual facial landmarks within the detected facial regions. The landmark information generated in each frame is arranged chronologically and can be used to analyze continuous patterns of facial expression changes.

[0049] In some embodiments of the present invention, the face recognition module (410) may employ multi-resolution scaling and illumination correction algorithms to improve the accuracy of face region detection under different resolutions and illumination conditions. The face recognition module (410) can scale the resolution of the input image at multiple levels, thereby optimizing face detection performance in both low-resolution and high-resolution environments. To correct problems such as contrast and color distortion caused by changes in illumination, the face recognition module (410) may also apply image preprocessing techniques such as histogram smoothing and gamma correction.

[0050] In some embodiments of the present invention, the face recognition module (410) can still robustly detect the face region even under various environmental changes such as partial occlusion and changes in face orientation. This module can recognize the face region by utilizing the robustness of a deep learning-based detector when partially occluded by masks, hands, hair, etc. Furthermore, the face recognition module (410) uses data augmentation and a rotation-invariant neural network structure to cope with changes in face direction such as yaw, pitch, and roll, thereby maintaining stable face detection performance at different angles.

[0051] According to some embodiments, the face recognition module (410) may include steps for calculating the coordinate values ​​of the aforementioned feature points, the distance between coordinates, angles, and other geometric parameters. The face recognition module (410) can calculate geometric parameters such as the Euclidean distance, relative angle, and arrangement pattern between key points based on the key point coordinate values ​​extracted in each frame. Simultaneously, the module can also output parameters that are important for expression analysis, such as eye distance, mouth corner angle, and distance between eyebrows and eyes.

[0052] According to some embodiments, the face recognition module (410) is able to track changes in the position of each feature point in consecutive video frames, thereby maintaining temporal continuity. The module arranges the landmark coordinates of each frame in chronological order to track the motion trajectory of each feature point. The face recognition module (410) can calculate time-varying parameters such as displacement, velocity, and acceleration of landmark points between frames.

[0053] In some embodiments, the facial recognition module (410) can stably capture subtle facial expression changes by analyzing the displacement and change patterns of feature points between frames. The module can quantitatively calculate subtle facial expression changes such as minute movements, vibrations, or tremors of landmarks, and estimate the activity level of facial action units (AUs) corresponding to minute muscle movements.

[0054] In some embodiments of the present invention, the face recognition module (410) can combine and normalize multiple extracted feature values ​​to form an emotion state-related feature vector with predefined dimensions and format. The face recognition module (410) can generate a multidimensional feature vector by integrating various feature values ​​such as landmark coordinates, distance, angle, and texture. The module can also standardize and normalize each feature value to minimize the impact of scale differences in the input data. The generated feature vector can be converted into a data format with predefined dimensions and format, and then used as input for an emotion classification model.

[0055] In some embodiments, the face recognition module (410) can represent the emotion state-related feature vector as an N-dimensional real vector with a specific dimension for use as input to an emotion classification model. The face recognition module (410) can convert each element of the feature vector into a real value and store it as an N-dimensional real vector.

[0056] In some embodiments of the present invention, the facial recognition module (410) may include a step of weighting or normalizing each feature value as needed to reflect its sensitivity to different emotional states (such as joy, anger, sadness, surprise, disgust, neutrality, etc.). The facial recognition module (410) can generate differentiated input vectors for different emotional states by multiplying each emotional sensitivity weight by each feature value.

[0057] Emotional state analysis module

[0058] In some embodiments, the emotion state analysis module (430) can utilize feature point information received from the face recognition module (410) to perform an analysis function of the user's emotional state. The emotion state analysis module (430) can receive multiple facial feature point information output by the face recognition module (410) as input. The emotion state analysis module (430) can analyze the geometric parameters such as coordinate values, distances, and angles in the input feature point information. Based on the position of each feature point and their interrelationships, the emotion state analysis module (430) can evaluate the static state of the expression and provide the static analysis results as input data for the subsequent emotion classification stage.

[0059] In some embodiments, the emotion state analysis module (430) can analyze the relationships between facial feature points within a single image frame, such as position, distance, and angle, to determine the emotional state of that frame. The emotion state analysis module (430) can extract the coordinate values ​​of each feature point within a single frame, calculate the distance and angle between feature points, and thus quantitatively assess subtle changes in facial expressions. The emotion state analysis module (430) can also classify expression types (e.g., smiling, frowning, neutral, etc.) based on the obtained distance and angle information. The emotion state analysis module (430) can quantify the facial expression state of each frame and store it as a baseline value for time-varying analysis.

[0060] According to some embodiments, the emotion state analysis module (430) can use the location information of the aforementioned facial feature points and the pixel values ​​of the facial image to quantitatively calculate the degree of eyelid opening and closing, the degree of mouth corner raising or lowering, the degree of eyebrow contraction or relaxation, and the activity level of facial action units (AUs) corresponding to subtle facial muscle movements. The emotion state analysis module (430) can calculate the degree of eye opening and closing by combining the location information of facial feature points with the pixel values ​​of the facial image. This module can also analyze the changes in the coordinates of landmarks at both ends of the mouth corners, quantifying the degree of mouth corner raising or lowering into numerical values. Furthermore, the emotion state analysis module (430) can quantitatively assess the degree of eyebrow contraction or relaxation by measuring the distance changes between landmarks in the eyebrow area. This module can also convert subtle facial muscle movements into activity values ​​of facial action units (AUs). The emotion state analysis module (430) can calculate the activity level of each AU frame by frame, thereby accurately capturing subtle changes in facial expressions.

[0061] In some embodiments, the emotion state analysis module (430) can deduce the user's facial expression change pattern by tracking temporal changes between frames. This module can track the coordinate changes of various facial feature points across multiple consecutive frames. The emotion state analysis module (430) can calculate the movement of landmarks between different frames to quantify the change pattern of facial expressions over time. Simultaneously, the module can calculate statistical indicators such as the mean, variance, and maximum value of facial expression changes within a specified time window, and evaluate the dynamic characteristics of the emotional state based on these change patterns.

[0062] In some embodiments of the present invention, the emotion state analysis module (430) can track the temporal changes of the aforementioned feature points across multiple frames, performing at least one analysis of facial expression change patterns during dynamic analysis. The emotion state analysis module (430) can accumulate the coordinate changes of each landmark point across multiple frames, thereby determining the directionality and magnitude of the facial expression changes. The emotion state analysis module (430) can vectorize the movement trajectories of the landmark points, thereby assessing the continuity of facial expression changes. This module improves the reliability of complex emotion state judgment by combining dynamic analysis results with static analysis results.

[0063] In some embodiments of the present invention, the emotion state analysis module (430) may include a step of quantifying temporal change characteristics such as increased smiling and intensified frowning. The emotion state analysis module (430) can quantify the degree of smiling by measuring the rising speed and duration of landmarks at the corners of the mouth. The module can also analyze the changes in the contraction intensity of landmarks in the glabella and eyebrow area to quantitatively calculate the degree of frowning. In addition, the emotion state analysis module (430) can calculate the cumulative amount and rate of change of facial expressions over time to determine whether the emotional state has fluctuated drastically.

[0064] According to some embodiments, the emotion state analysis module (430) is capable of generating emotion feature vectors corresponding to facial expression change patterns. This module can extract multiple feature values ​​from static and dynamic analysis results and convert these feature values ​​into vectors conforming to preset dimensions and formats. The emotion state analysis module (430) can provide the generated emotion feature vectors as input data for an emotion classification model.

[0065] In some embodiments, the emotion state analysis module (430), based on the above static and dynamic analysis results, can output texture feature values ​​such as the coordinates or coordinate changes of facial feature points, the activity of facial action units (FACUs), local binary patterns (LBP), and Gabor filter responses, as well as one or more embedding vectors extracted by a deep learning neural network. The emotion state analysis module (430) can include the coordinates and coordinate changes of facial feature points as part of the vector. The module can also normalize the activity of facial action units before adding it to the vector. In addition, the emotion state analysis module (430) can generate texture feature values ​​such as LBP and Gabor filters to quantitatively reflect changes in skin, wrinkles, and light and dark patterns. By combining the embedding vectors extracted by the deep learning neural network, the emotion state analysis module (430) can vectorize high-dimensional expression features.

[0066] According to some embodiments, the emotion state analysis module (430) may include a step of combining and normalizing the multiple feature values ​​generated above to construct an emotion state-related feature vector with predefined dimensions and format. The emotion state analysis module (430) can balance the influence among elements in the vector by normalizing the scale of each feature value. The module can also apply sensitivity weights for different emotions to adjust the proportion of feature values ​​sensitive to specific emotions. The emotion state analysis module (430) represents the combined feature vector as an N-dimensional real vector, conforming to the input specifications of the emotion classification model.

[0067] In some embodiments of the present invention, the emotion state analysis module (430) can input the generated emotion feature vector into the emotion classification model to determine which emotion type and corresponding intensity level the user's current emotion state belongs to. The emotion state analysis module (430) can convert the generated emotion feature vector into a form that matches the input format of the emotion classification model. By inputting the feature vector into the emotion classification model, the module can predict the specific emotion type (e.g., joy, sadness, anger, etc.) and intensity level (1-5).

[0068] According to some embodiments, the emotion classification model may receive an emotion feature vector generated by the emotion state analysis module (430) and classify it into one of 10 emotion types (e.g., joy, sadness, anger, fear, anxiety, depression, etc.). The emotion classification model may compare each element of the input feature vector with a pre-trained classification criterion. By analyzing the patterns of the feature values ​​in the vector, the model may calculate the probability value corresponding to one of the 10 emotion types.

[0069] In some embodiments of the present invention, the emotion classification model can map different emotion types to five intensity levels. The emotion classification model can calculate an intensity score based on specific elements or combinations in the emotion feature vector. The model can then map the calculated intensity score to a level range of 1 to 5.

[0070] According to some embodiments, the sentiment classification model can output or store the classified sentiment types and intensity levels. The model can transmit the classification results to a user terminal (130) or a server in real time. Simultaneously, the sentiment classification model can save the classification results to a database for long-term historical management of sentiment states. Furthermore, the sentiment classification model can also pass the classification results to a feedback content generation module (450) to provide personalized feedback content.

[0071] Content generation module

[0072] In some embodiments, the content generation module (450) can generate corresponding feedback content based on the difference between the user's current emotional state determined by the emotional state analysis module (430) and the target emotional state input through the user terminal (130) or preset. The content generation module (450) can receive the type and intensity information of the current emotional state from the emotional state analysis module (430). It can receive the target emotional state through the input interface of the user terminal (130) or call the pre-stored target emotional state. The content generation module (450) can compare the type and intensity values ​​of the current emotional state and the target emotional state to calculate the difference index. Based on the difference index, the content generation module (450) can determine the necessity and presentation method of the feedback content. When a difference is found between emotional states, the content generation module (450) can generate corresponding feedback content to narrow the gap. During the process of generating feedback content, the content generation module (450) can apply different content type and intensity adjustment parameters according to the type and intensity of the emotional state. After generation, the content generation module (450) can send the feedback content to the output module (470) and provide it to the user.

[0073] According to some embodiments, the content generation module (450) is capable of analyzing the type and intensity differences of emotional states. This module can identify the emotion type (e.g., joy, sadness, anger, etc.) and intensity level (1 to 5) of the current emotional state and the target emotional state. When a difference exists in emotion type, the content generation module (450) can determine the type of feedback content required to achieve the emotion type conversion. If a difference exists in emotion intensity, the content generation module (450) can calculate the absolute value of the intensity difference and use it as a basis for adjusting the intensity of the feedback content.

[0074] In some embodiments of the present invention, the content generation module (450) generates feedback content to reduce the differences in the types and intensities of emotional states by analyzing these differences. When the differences in emotional types are large, the content generation module (450) can select more suitable stimuli (e.g., calm, arousal, immersion) to induce emotional shifts. When the differences in emotional intensity are large, the content generation module (450) can correspondingly increase or decrease the intensity of the feedback content (e.g., volume, brightness, playback speed). Furthermore, the content generation module (450) can combine and provide various types of feedback content, such as video, audio, and text, based on the results of the emotional gap analysis. When the emotional gap is below a certain preset threshold, the content generation module (450) can stop providing feedback content or gradually reduce its intensity.

[0075] In some embodiments of the present invention, the content generation module (450) can generate various types of feedback content, such as video, audio, and text. Based on the type and intensity of the emotional state, the content generation module (450) can select at least one of multiple feedback content types, such as video, image, music, sound, and text, for presentation. The content generation module (450) can automatically select content that matches the current emotional state by referring to a predefined emotion type matching table. This module can synchronize multiple feedback content types, providing visual, auditory, and textual stimulation simultaneously. When constructing feedback content, the content generation module (450) can provide personalized feedback content based on the user's preferences and past behavioral data.

[0076] In some embodiments of the present invention, the content generation module (450) can adjust the intensity, expression, and presentation of the content as needed to generate feedback content in a form suitable for user reception. To adjust the intensity of the feedback content, the content generation module (450) can modify parameters such as volume, brightness, playback speed, color temperature, and text length in real time. Based on the changing trends of emotional states, the content generation module (450) can dynamically adjust the expression of the feedback content (e.g., video color, music rhythm, text tone, etc.). Furthermore, the content generation module (450) can also adjust the presentation format of the feedback content, displaying it to the user through various methods such as full-screen display, pop-ups, and notifications.

[0077] In some embodiments, the content generation module (450) can use content containing video or image stimuli as feedback content. Depending on the type and intensity of the emotional state, the module can select different types of video or image stimuli. The content generation module (450) can adjust visual parameters such as resolution, color, brightness, and playback duration of the video or image stimuli.

[0078] In some embodiments of the present invention, the content generation module (450) can provide visual stimuli such as natural scenery, ocean waves, animals, and people as feedback content to the user. The content generation module (450) can select appropriate content from various visual stimuli such as natural scenery (e.g., forests, oceans, mountains), waves, animals, and people according to the user's emotional state.

[0079] In some embodiments of the present invention, the content generation module (450) can provide music or sound stimuli as feedback content. Depending on the type and intensity of the emotional state, the content generation module (450) can select parameters such as the type, rhythm, volume, and timbre of the music or sound stimulus. The content generation module (450) can also adjust detailed parameters such as the playback duration, repetition count, start and end times of the music or sound stimulus.

[0080] In some embodiments, the content generation module (450) can provide text or text stimuli as feedback content. Depending on the type and intensity of the emotional state, the content generation module (450) can select parameters such as text content, length, tone, and font. The module can also adjust the presentation method of the text stimulus (e.g., full-screen display, notification pop-up, pop-up window, etc.).

[0081] In some embodiments of the present invention, the content generation module (450) can adjust the intensity and expression of the feedback content in a suitable form according to the user's preferences. The module (450) achieves customized adjustment of personalized feedback content by analyzing the emotional state change patterns and preferences of different users. Furthermore, the module (450) can gradually increase or decrease the intensity and expression of the feedback content according to the trend of emotional changes.

[0082] Output module

[0083] In some embodiments of the present invention, the output module (470) can transmit the generated feedback content to the user terminal (130). The output module (470) can receive feedback content data from the content generation module (450). The output module (470) can determine the transmission target of the feedback content based on the network address or identifier of the user terminal (130). After determining the transmission target, the output module (470) can send the feedback content to the user terminal (130) through at least one of real-time streaming or file transfer methods.

[0084] In some embodiments of the present invention, the output module (470) may include a network interface and a transmission protocol for transmitting feedback content. The output module (470) may be connected to the user terminal (130) via a wired or wireless network interface. The output module (470) may support at least one of multiple transmission protocols such as TCP / IP, HTTP, WebSocket, and MQTT.

[0085] According to some embodiments, the output module (470) can provide the feedback content received from the content generation module (450) to the user terminal (130) in real time or asynchronously. In real-time transmission mode, the output module (470) can immediately send the generated feedback content from the content generation module (450) to the user terminal (130). In asynchronous transmission mode, the output module (470) can temporarily store the feedback content in a temporary memory and transmit it according to the request of the user terminal (130) or changes in network status.

[0086] In some embodiments of the present invention, the output module (470) may be designed to adapt to various user terminal (130) environments. The output module (470) supports various types of user terminals (130) such as smartphones, tablets, personal computers, and smartwatches. The output module (470) can automatically identify the hardware and software specifications of each terminal, such as operating system (OS), screen resolution, audio output method, etc.

[0087] According to some embodiments, the output module (470) can dynamically adjust the transmission method of the feedback content based on the network status. The output module (470) can monitor the network connection status of the user terminal (130) in real time (such as Wi-Fi, LTE, 5G, wired network, etc.). The output module (470) can measure network quality indicators such as network bandwidth, latency, and packet loss rate. When the network quality deteriorates, the output module (470) can automatically adjust parameters such as the resolution, bit rate, and transmission frequency of the feedback content.

[0088] According to some embodiments, the output module (470) can provide concurrent support for multiple user terminals (130) or multi-device environments. The output module (470) can simultaneously support the access and content transmission of multiple user terminals (130). The output module (470) can create an independent session for each terminal and independently manage the content transmission status of each session.

[0089] According to some embodiments, the output module (470) can also continuously monitor changes in the user's psychological state. After providing feedback to the user terminal (130), the output module (470) can periodically acquire facial images of the user through the terminal's camera or sensors. The output module (470) transmits the acquired facial images to the face recognition module (410) to analyze facial feature points and expression changes in real time. The output module (470) can receive real-time emotional state results from the emotion state analysis module (430) and store these results in chronological order to record the trend of emotion changes.

[0090] According to some embodiments, the output module (470) can dynamically adjust or repeatedly provide feedback content based on monitoring results, combined with the content generation module (450) or its internal logic. When the output module (470) detects that the user's emotional state has not reached the target emotion, it can request the content generation module (450) to generate additional feedback content. The output module (470) can automatically adjust the intensity, type, and expression of the feedback content according to the emotional change pattern through its own internal logic. When the emotional state does not improve or worsens, the output module (470) can repeatedly provide feedback content or gradually increase the intensity of the stimulus. When the emotional state is close to the target emotion, the output module (470) can stop providing feedback content or gradually reduce its intensity.

[0091] According to some embodiments, the output module (470) can determine whether the user's response contributes to the improvement of their emotional state. The output module (470) calculates the change in the type and intensity of emotion by comparing the emotional state data before and after the feedback content is provided. If the change in emotional state exceeds a preset threshold, it is determined that the emotion has improved.

[0092] According to some embodiments, the output module (470) can perform dynamic control of the intensity of the feedback content when necessary. When the emotion change monitoring results show that the emotional state is close to the target emotion, the output module (470) can automatically reduce the intensity parameters of the feedback content, such as volume, brightness, and playback speed. When the emotional state does not improve or worsens, the output module (470) can change the type of feedback content (e.g., video, music, text, etc.).

[0093] According to some embodiments, the output module (470) can be configured with conditions for repetitive provision or termination of feedback content. When the change in emotional state remains below a certain threshold, the output module (470) can be configured to continue repetitively providing feedback content. The output module (470) can also stop providing feedback content when the emotional state reaches a target emotional state or when the number of repetitions reaches a preset upper limit.

[0094] Target emotional state acquisition

[0095] In some embodiments of the present invention, a user terminal (130) may provide an input interface for acquiring a target emotional state. The user terminal (130) may include various input devices such as a touchscreen, physical buttons, and voice recognition. The input interface may display UI elements for selecting the desired emotion type (e.g., joy, calm, tranquility, etc.) and intensity (level 1 to 5). When the user selects the emotion type and intensity, the input interface may transmit the selected information to the system (100) in real time. The input emotional state information is converted into an internal data format (e.g., JSON, XML, etc.) and stored in the system (100).

[0096] In some embodiments of the present invention, the target emotional state acquisition step may include a series of operations to set and store the target emotional state through user input or automatic updates by the system (100). This step supports both direct input of the emotional state by the user and automatic updates of the emotional state by the system (100) through analysis of historical records and preference data.

[0097] According to some embodiments, the target emotional state set in the target emotional state acquisition step can be used as benchmark data in the subsequent emotional state analysis and feedback content generation process. The emotional state analysis module (430) can calculate the emotional gap by comparing the target emotional state with the current emotional state. The content generation module (450) can generate corresponding feedback content based on the type and intensity of the emotional gap.

[0098] In some embodiments of the present invention, the system (100) can automatically update the target emotional state by analyzing user preferences and historical data. The system (100) can collect the user's input emotional state history, response records to feedback content, and preference setting data. The system (100) can input the collected data into statistical analysis or a machine learning-based predictive model. The system (100) can prioritize recommending frequently selected emotion types and intensities based on the analysis results.

[0099] Method and Flow

[0100] Figure 5 A flowchart illustrating a content delivery method for managing user emotional states based on facial recognition, according to an embodiment, is shown.

[0101] refer to Figure 5 Actions 510 to 570 can be executed sequentially, but they do not have to be executed in order. For example, the execution order of each action (510 to 570) can be changed, and at least two actions can be executed in parallel.

[0102] In action 510, the electronic device (110) can extract one or more facial feature points from the user's facial region in the captured user image.

[0103] In action 530, the electronic device (110) can analyze the user's facial expression state based on one or more facial feature points.

[0104] In action 550, the electronic device (110) can obtain the target emotional state from the user's terminal.

[0105] In action 570, the electronic device (110) can generate feedback content based on the difference between the user's emotional state and the target emotional state.

[0106] According to some embodiments, the electronic device (110) is capable of detecting the user's facial region from captured video. For each frame of the input video, the electronic device (110) applies a deep learning-based face detector to automatically identify the location of the user's face in the image. To improve the accuracy of facial region detection under different resolutions and lighting conditions, the electronic device (110) may employ multi-resolution scaling and lighting correction algorithms. Furthermore, the electronic device (110) can stably detect facial regions under various environmental changes, such as partial occlusion and changes in facial orientation.

[0107] According to some embodiments, the electronic device (110) can extract multiple facial landmarks within the aforementioned facial region. The electronic device (110) can apply a landmark extraction algorithm from the detected facial region to that region to calculate multiple facial landmarks such as eyes, eyebrows, mouth, nose, and jawline. Each facial landmark can be represented by two-dimensional or three-dimensional coordinate values. The electronic device (110) can also calculate the coordinate values, distances between coordinates, angles, and other geometric parameters of each landmark.

[0108] According to some embodiments, the electronic device (110) may include a deep learning-based face detector and a feature point extraction algorithm to stably extract facial feature points in a single image frame and consecutive video frames. The electronic device is able to track changes in the position of feature points between frames in consecutive video frames, thereby maintaining temporal consistency. The deep learning-based detector detects facial regions frame by frame, while the feature point extraction algorithm generates landmarks in each frame. By analyzing the amount of movement and change patterns of feature points between frames, the electronic device (110) is able to stably capture subtle changes in facial expressions.

[0109] According to some embodiments, facial feature point information provided by the electronic device (110) can serve as the basis data for subsequent expression analysis and emotion state estimation. The feature point information generated by the electronic device (110) can be transmitted to the electronic device (110) and used to analyze the static state of expressions and their temporal change patterns. This feature point information can also serve as the basis data for subsequent processing steps, such as generating emotion feature vectors and inputting them into emotion classification models.

[0110] According to some embodiments, the electronic device (110) can determine the user's facial expression change pattern based on the change in the user's facial expression state determined within a single image frame over time. The electronic device (110) arranges the facial expression states generated in each frame in chronological order, and then analyzes the continuous pattern of facial expression changes. The facial expression change pattern can be quantified by various indicators such as the amount of movement of feature points between frames, the rate of change, and minute muscle twitches.

[0111] According to some embodiments, the electronic device (110) can perform an analysis function of the user's emotional state using feature point information received from itself. The electronic device (110) can analyze the geometric parameters such as the position, distance, and angle of the input feature points, thereby quantitatively deducing the user's facial expression state. The electronic device (110) combines static and dynamic analysis, and can simultaneously evaluate the facial expression state and its temporal change pattern within a single frame. The analysis results are presented in the form of emotional feature vectors, which can be used as input to an emotional classification model.

[0112] In some embodiments of the present invention, the electronic device (110) determines the facial expression state of a single image frame by analyzing the relationships between the positions, distances, and angles of facial feature points. The electronic device (110) quantifies facial expression elements such as the degree of eyelid opening and closing, the upward / downward movement of the corners of the mouth, and the contraction / relaxation of the eyebrows using feature point coordinate values, distances between coordinates, and angle information extracted from each frame.

[0113] In some embodiments of the present invention, the electronic device (110) can generate a user's facial expression change pattern by tracking temporal changes between frames. The electronic device (110) can calculate temporal change information such as the displacement of feature points, the rate of change, and minute muscle twitches in consecutive frames. This temporal change pattern can be used to quantify facial expression change features such as enhanced smiles and deepened frowns.

[0114] According to some embodiments, the electronic device (110) can determine a user's emotional state based on facial expression change patterns. The electronic device (110) inputs the facial expression change patterns into an emotion classification model to determine the user's emotion type (e.g., joy, sadness, anger, etc.) and its intensity level.

[0115] According to some embodiments, the electronic device (110) can generate a corresponding emotion feature vector based on the user's facial expression change pattern during the process of judging the user's emotional state. The electronic device (110) can combine information such as facial expression change pattern, feature point coordinate change, action unit activation degree, and texture feature value to generate an N-dimensional real vector emotion feature vector. This emotion feature vector can be normalized to reflect the sensitivity weight of different emotion types.

[0116] According to some embodiments, the electronic device (110) can determine the user's current emotional state based on an emotion feature vector. The electronic device (110) inputs the emotion feature vector into a pre-trained emotion classification model, thereby outputting the emotion type and intensity level.

[0117] In some embodiments of the present invention, the emotion feature vector may include changes in facial feature point coordinates, activation levels of facial action units, and texture features such as facial skin, wrinkles, and light and dark patterns. The emotion feature vector may include texture-based feature values ​​such as changes in feature point coordinates extracted in each frame or frame sequence, facial action unit activity, and LBP·Gabor filter responses.

[0118] According to some embodiments, the electronic device (110) can generate feedback content based on the difference between a determined current emotional state of the user and a target emotional state input through a user terminal (130) or set separately. The electronic device (110) can quantitatively calculate the difference in type and intensity between the current emotional state and the target emotional state. This calculated emotional gap can serve as a reference standard for determining the type, intensity, and expression of the feedback content.

[0119] According to some embodiments, the electronic device (110) can generate feedback content such as videos, audio, and text to reduce the differences in the type and intensity of emotional states. The electronic device (110) can select various predefined feedback content from a database, such as videos, music, images, and text of different emotional types, and adjust the stimulation intensity, playback duration, and repetition frequency of the content according to the differences in emotional intensity.

[0120] According to some embodiments, the electronic device (110) can adjust the intensity, expression, and presentation of the content as needed to generate a feedback format suitable for the user. The electronic device (110) can adjust parameters such as the volume, brightness, color temperature, playback speed, and tone of voice of the content in real time, taking into account the user's preferences, usage history, and environmental information.

[0121] In some embodiments of the present invention, the electronic device (110) may send the generated feedback content to the user terminal (130). The electronic device (110) may send the feedback content in a streaming or local push manner, depending on network status, terminal compatibility, and QoS conditions.

[0122] According to some implementations, the electronic device (110) can also continuously monitor changes in the user's emotional state. After providing feedback, the electronic device (110) can capture subsequent facial images in real time and re-analyze changes in the user's emotional state. The electronic device (110) works in conjunction with itself to continuously assess whether the user's emotions have improved.

[0123] In some implementations, the electronic device (110) can dynamically adjust or repeatedly provide feedback content based on monitoring results and its own internal logic. When insufficient improvement in user sentiment is detected, the electronic device (110) can send a reset request to itself, thereby changing the type, intensity, and expression of the feedback content.

[0124] In some embodiments of the present invention, the electronic device (110) can determine whether the emotion has improved based on the user's response, and perform dynamic control as necessary, such as reducing or changing the intensity of the feedback content. When the emotion state analysis result shows that the emotion is close to the target emotion, the electronic device (110) can gradually reduce the intensity of the feedback content; if the emotion does not improve, the type of feedback content can be switched or the intensity of the stimulus can be increased.

[0125] Electronic device structure

[0126] Figure 6 This is a schematic diagram of the operation of an electronic device according to an embodiment.

[0127] Reference Figure 6 The electronic device (110) may include a memory (610) and a processor (630).

[0128] The memory (610) can be used to store instructions (e.g., programs) that the processor (630) can execute. For example, these instructions may include instructions for controlling how the processor (630) operates or the operation of its components.

[0129] The memory (610) can be implemented as a volatile memory device or a non-volatile memory device.

[0130] Volatile storage devices can be implemented in the form of DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), T-RAM (Thyristor RAM), Z-RAM (Zero Capacitor RAM) or TTRAM (Dual Transistor RAM).

[0131] Non-volatile memory devices can be implemented in the form of EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque MRAM (STT-MRAM), CBRAM (Bridge-Conducting RAM), FeRAM (Ferroelectric RAM), PRAM (Phase Change RAM), Resistive RAM (RRAM), Nanotube RRAM, Polymer RAM (PoRAM), Nano Floating Gate Memory (NFGM), Holographic memory, Molecular Electronic Memory Device, or Insulator Resistance Change Memory, etc.

[0132] The processor (630) can process data stored in the memory (610). The processor (630) is capable of executing computer-readable code (e.g., software) stored in the memory (610) and instructions issued by the processor (630).

[0133] The processor (630) can be a hardware-implemented data processing device that includes circuitry with a physical structure to perform specific operations. For example, these specific operations may include code or instructions contained in a program.

[0134] For example, hardware-implemented data processing devices may include microprocessors, central processing units, processor cores, multi-core processors, multiprocessors, ASICs (Application-Specific Integrated Circuits), and FPGAs (Programmable Gate Arrays).

[0135] The processor (630) enables the electronic device (110) to perform one or more operations by executing code and / or instructions stored in memory (610).

[0136] The above embodiments can be implemented by hardware components, software components, and / or a combination of hardware and software components. For example, the apparatus, methods, and components described in the embodiments can be implemented by a general-purpose computer or a special-purpose computer, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor (DSP), microcomputer, field-programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing instructions and responding.

[0137] Software may include computer programs, code, instructions, or one or more combinations thereof, and may be configured as needed for processing devices, or may control processing devices independently, individually, or collectively. Software and / or data may be stored in any form of mechanical device, component, physical device, virtual device, computer storage medium, or device for interpretation by processing devices or for providing instructions or data to processing devices. Software may also be distributed across network-connected computer systems, stored or executed in a distributed manner.

[0138] The method according to the embodiments can be implemented in the form of program instructions by various computer means and can be recorded on a computer-readable medium. The computer-readable medium can store program instructions, data files, and data structures, either individually or in combination, and the program instructions recorded on the medium may be specifically designed and constructed for the embodiments or may be disclosed and available to those skilled in computer software. Examples of computer-readable recording media include magnetic media (such as hard disks, floppy disks, and magnetic tapes), optical disk media (such as CD-ROMs and DVDs), magneto-optical media (such as optical discs), and hardware devices dedicated to storing and executing program instructions (such as read-only memory (ROM), random access memory (RAM), flash memory, etc.).

[0139] The aforementioned hardware device can be configured to operate as one or more software modules to achieve the functions of the embodiments, and vice versa.

[0140] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A content delivery method based on facial recognition for managing user emotional states, characterized in that, include: Acquire a video stream of the user being filmed, and detect the user's facial region from each frame of the video stream; Multiple facial feature points are extracted within the facial region, including feature points corresponding to the eyes, eyebrows, mouth, nose, and jawline. Based on the position, distance, and angle of the facial feature points in a single frame image, determine the user's static facial expression state in the current frame; The user's facial expression change pattern is determined based on the positional changes of the facial feature points between consecutive frames; Based on the static facial expression state and the facial expression change pattern, an emotion feature vector is generated; The emotional feature vector is input into a pre-trained emotion classification model to determine the user's current emotional state and its intensity level. Obtain the target's emotional state and its intensity level; Calculate the type and intensity differences between the current emotional state and the target emotional state; Based on the type and intensity differences, feedback content is selected and generated from a preset content library, and the feedback content includes at least one of video, audio or text. The feedback content is sent to the user's terminal, and the user's emotional state is continuously monitored. The intensity parameters and expression of the feedback content are dynamically adjusted based on the monitoring results until the user's emotional state reaches the target emotional state.

2. The method according to claim 1, characterized in that, The emotional state is at least one of the following: joy, sadness, anger, fear, anxiety, and depression, with each emotional state corresponding to an intensity level of 1 to 5.

3. The method according to claim 1, characterized in that, The step of determining the static expression state based on the position, distance, and angle of the facial feature points in a single frame image includes: Calculate the vertical distance between eye feature points to quantify the degree of eyelid opening and closing; Calculate the displacement of the corner feature points relative to the baseline to quantify the degree of upturn or downturn of the corners of the mouth; Calculate the distance changes between eyebrow feature points to quantify the degree of eyebrow contraction or relaxation; Calculate the activity of facial motion units and quantify the subtle movements of facial muscles.

4. The method according to claim 1, characterized in that, The step of determining the expression change pattern based on the positional change of the facial feature points between consecutive frames includes: Calculate the displacement, velocity of change, and acceleration of each feature point between consecutive frames; Calculate the cumulative amount and rate of change of facial expressions within a preset time window; Numericalize the temporal variation characteristics of smile enhancement or frown deepening.

5. The method according to claim 1, characterized in that, The emotion feature vector includes: The coordinate changes of the facial feature points; The activity level of the user's facial motion units; and Texture feature values ​​corresponding to facial skin, wrinkles, and light and shadow patterns.

6. The method according to claim 5, characterized in that, The texture feature values ​​include local binary mode feature values ​​and Gabor filter response values.

7. The method according to claim 1, characterized in that, The step of dynamically adjusting the intensity parameters and expression method of the feedback content based on the monitoring results includes: When the intensity difference between the current emotional state and the target emotional state is greater than a preset threshold, increase the volume, playback speed, or screen brightness of the feedback content. When the intensity difference is less than a preset threshold, the volume, playback speed, or screen brightness of the feedback content are gradually reduced. When the current emotional state and the target emotional state belong to different emotional types, switch the type of the feedback content; The way the feedback content is expressed is personalized based on the user's preference information and usage history.

8. The method according to claim 1, characterized in that, The step of generating feedback content based on the type and intensity differences includes: Select the corresponding visual stimulus type, auditory stimulus type, or text stimulus type based on the differences in the types described; The stimulation intensity, playback duration, and repetition frequency of the feedback content are determined based on the intensity difference. The visual stimulus and the auditory stimulus are synchronized in time to generate feedback content in the form of a composite stimulus.

9. A content delivery system for managing user emotional states based on facial recognition, characterized in that, include: The facial recognition module is used to acquire the video stream of the user, detect the user's facial region from each frame of the video stream, and extract multiple facial feature points in the facial region. The emotion state analysis module is used to determine the static expression state based on the position, distance and angle of the facial feature points in a single frame image, determine the expression change pattern based on the position change of the facial feature points between consecutive frames, generate an emotion feature vector based on the static expression state and the expression change pattern, and input the emotion feature vector into a pre-trained emotion classification model to determine the user's current emotion state and its intensity level. The content generation module is used to obtain the target emotional state and its target intensity level, calculate the type difference and intensity difference between the current emotional state and the target emotional state, and generate feedback content based on the type difference and intensity difference. The feedback content includes at least one of video, audio or text. as well as The output module is used to send the feedback content to the user terminal, continuously monitor the changes in the user's emotional state, and dynamically adjust the intensity parameters and expression of the feedback content according to the monitoring results until the user's emotional state reaches the target emotional state.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, connected to the memory, is configured to execute the computer program to implement the method as described in any one of claims 1 to 8.

Images