A two-way emotional accompanying system and method for left-behind children

Abstract

The application relates to a bidirectional emotional accompanying system and method for left-behind children, and belongs to the technical field of artificial intelligence. The system comprises a data acquisition module, an emotion recognition module, a feedback module, a digital image generation module and an emotion and preference learning module. The data acquisition module is used for collecting emotional state data of children in real time. The emotion recognition module is used for analyzing and recognizing the emotional state data to obtain the emotional state of the children. The feedback module is used for analyzing the emotional needs of the children according to the emotional state and providing personalized feedback results. The digital image generation module is used for generating a virtual digital image according to the feedback results. The emotion and preference learning module is used for collecting the emotional state, the feedback results and the interactive information between the children and the virtual digital image in real time, and extracting emotional expression features and preference features of the children. The emotion and preference learning module comprises a data integration submodule, a time sequence recording submodule, a feature extraction submodule and a preference model submodule. The application improves the accuracy and diversity in emotion recognition.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a two-way emotional companionship system and method for left-behind children. Background Technology

[0002] Currently, AI emotion recognition technology mainly relies on the analysis of facial expressions, voice tone, and text content. However, this technology has shortcomings in accuracy and diversity, especially when dealing with children's emotional expressions. Children, especially left-behind children, often express emotions in ways that do not conform to conventional models due to prolonged loneliness or lack of channels for emotional expression. This makes it difficult for existing AI technology to accurately capture subtle changes in emotions, thus affecting the precise response to children's emotional needs.

[0003] Most AI-powered emotional companionship products on the market focus on simple interactions and companionship, such as conversational AI and gamified guidance. These products fall short in providing deep emotional care and social recognition. Left-behind children face complex psychological issues, requiring not only simple companionship but also in-depth emotional guidance and support for social development. Furthermore, existing AI products also have limitations in personalized companionship, lacking a deep understanding of each child's individual background, growth environment, and psychological state, making it difficult to provide targeted emotional guidance based on each child's unique experiences.

[0004] Although AI technology has shown potential in addressing the psychological problems of left-behind children, it still has significant shortcomings in terms of the accuracy of emotion recognition, in-depth companionship, personalized interaction, sustained long-term companionship, and social interaction capabilities. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the low accuracy and variability in emotion recognition in the prior art.

[0006] Firstly, to solve the aforementioned technical problems, the present invention provides a method for two-way emotional companionship for left-behind children, comprising:

[0007] The data acquisition module is used to collect children's emotional state data in real time;

[0008] An emotion recognition module is used to analyze and recognize the emotion state data to obtain the child's emotion state;

[0009] The feedback module is used to analyze the child's emotional needs based on the emotional state and provide personalized feedback results; wherein the feedback results are optimized using a reward function and time difference error, and the calculation formula of the reward function is:

[0010]

[0011] The formula for calculating the time difference error is:

[0012]

[0013] Where R1 represents the immediate feedback reward, R2 represents the long-term effect reward, f1(·) and f2(·) represent nonlinear functions, t represents time, γ represents the discount factor, and w1, w2, w3 represent weighting coefficients; R t+1 Let θ represent the reward at the current time t. - S represents the parameters of the target network, θ represents the parameters of the policy network, and S represents the parameters of the policy network. t+1 S represents the state at time t+1. t A represents the state at time t. t Let represent the action at time t; 'a' represent the action parameter; 'λ' represent the weight; Q(·) represent the value function; and A(·) represent the advantage function.

[0014] A digital avatar generation module is used to generate a virtual digital avatar based on the feedback results;

[0015] The emotion and preference learning module is used to collect the emotional state, the feedback results, and the interaction information between the child and the virtual digital image in real time, and to extract the child's emotional expression characteristics and preference characteristics.

[0016] In one embodiment of the present invention, the emotion and preference learning module includes: a data integration submodule, used to collect the emotional state, the feedback results, and the interaction information between the child and the virtual digital image in real time, and generate multi-source data; a time series recording submodule, used to record the multi-source data to form time series data; a feature extraction submodule, used to extract emotional expression features and preference features from the time series data; and a preference model submodule, used to establish an emotion and preference model based on the emotional expression features and the preference features.

[0017] In one embodiment of the present invention, the data acquisition module further includes a data privacy and security submodule, which is used to encrypt and transmit the emotional state data.

[0018] In one embodiment of the present invention, the emotion recognition module includes an emotion model submodule, which is used to capture the child's emotional characteristics and establish an emotion model based on the emotional state data.

[0019] In one embodiment of the present invention, the method for establishing the sentiment model is as follows: performing time series alignment and labeling on the sentiment state data to obtain a dataset organized and labeled according to time series; performing multimodal feature extraction and feature filtering dimensionality reduction processing on the dataset to obtain a feature subset; and establishing the sentiment model using a long short-term memory network combined with a convolutional neural network based on the feature subset.

[0020] Secondly, to solve the above-mentioned technical problems, the present invention provides a method for two-way emotional companionship for left-behind children, comprising:

[0021] Real-time collection of children's emotional state data;

[0022] The emotional state data is analyzed and identified to obtain the child's emotional state;

[0023] Based on the emotional state, the child's emotional needs are analyzed, and personalized feedback is provided; wherein the feedback is optimized using a reward function and time difference error, and the calculation formula for the reward function is:

[0024]

[0025] The formula for calculating the time difference error is:

[0026]

[0027] Where R1 represents the immediate feedback reward, R2 represents the long-term effect reward, f1(·) and f2(·) represent nonlinear functions, t represents time, γ represents the discount factor, and w1, w2, w3 represent weighting coefficients; R t+1 Let θ represent the reward at the current time t. - S represents the parameters of the target network, θ represents the parameters of the policy network, and S represents the parameters of the policy network. t+1 S represents the state at time t+1. t A represents the state at time t. t Let represent the action at time t; 'a' represent the action parameter; 'λ' represent the weight; Q(·) represent the value function; and A(·) represent the advantage function.

[0028] Based on the feedback results, a virtual digital avatar is generated;

[0029] The system collects the emotional state, feedback results, and interaction information between the child and the virtual digital avatar in real time, and extracts the child's emotional expression characteristics and preference characteristics.

[0030] In one embodiment of the present invention, before the real-time acquisition of the child's emotional state data, the child's identity information is identified and stripped to generate an anonymized identifier of a preset length.

[0031] In one embodiment of the present invention, the step of extracting the emotional expression features and preference features of children includes establishing or updating an emotion and preference model; when the emotion and preference model does not exist, modeling is performed based on the emotional expression features and preference features; when the emotion and preference model exists, the emotional expression features and preference features are analyzed, and the parameters of the emotion and preference model are updated based on the analysis results.

[0032] Thirdly, in order to solve the above-mentioned technical problems, the present invention provides an electronic device, including the above-mentioned two-way emotional companionship system for left-behind children.

[0033] Fourthly, to solve the above-mentioned technical problems, the present invention provides a computer program product, including computer program instructions, which, when executed on a computer, cause the above-mentioned method for two-way emotional companionship for left-behind children to be executed.

[0034] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0035] (1) The two-way emotional companionship system and method for left-behind children described in this invention utilizes the feedback results optimized by reward functions and time difference errors. Through personalized analysis of children's emotional states, it can provide customized emotional support and feedback. This not only enhances the system's understanding of children's emotions and its self-regulation ability but also improves learning efficiency and interactive experience by adjusting feedback strategies in real time. Simultaneously, through interaction with virtual digital avatars, children can explore and express their emotions in a safe and controlled environment, which contributes to the development of their social skills and emotional intelligence. Furthermore, the design of the emotion recognition module ensures high accuracy and diversity in emotion recognition. This invention, through this personalized interactive experience, helps children grow in emotional cognition, expression, and management, while also providing a powerful auxiliary tool for parents and educators.

[0036] (2) This invention enables the system to process and analyze data from different sources through a data integration submodule, a time series recording submodule, and a feature extraction submodule, forming a comprehensive data view, which helps to understand children’s behavioral and emotional changes more deeply.

[0037] (3) The feedback module possesses self-evolutionary characteristics, enabling it to continuously learn and adapt to changes in children's emotions. Over time, the system gradually improves the accuracy of its identification of emotional states and the precision of its responses by accumulating and analyzing children's emotional data. This self-optimization mechanism endows the AI ​​system with the ability to continuously improve, allowing it to tailor emotional support solutions for each child. Attached Figure Description

[0038] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein...

[0039] Figure 1 This is a structural diagram of a two-way emotional companionship system for left-behind children in a preferred embodiment of the present invention;

[0040] Figure 2 This is a structural diagram of the data acquisition module in a preferred embodiment of the present invention;

[0041] Figure 3 This is a structural diagram of the emotion and preference learning module in a preferred embodiment of the present invention;

[0042] Figure 4 This is a flowchart of a two-way emotional companionship method for left-behind children in a preferred embodiment of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0044] Example 1

[0045] Reference Figure 1 As shown, this embodiment of the invention provides a two-way emotional companionship system for left-behind children, comprising:

[0046] The data acquisition module is used to collect children's emotional state data in real time;

[0047] The emotion recognition module is used to analyze and identify emotional state data to obtain the child's emotional state;

[0048] The feedback module analyzes children's emotional needs based on their emotional state and provides personalized feedback results; it optimizes the feedback results using a reward function and time difference error.

[0049] The digital avatar generation module is used to generate virtual digital avatars based on feedback results;

[0050] The Emotion and Preference Learning Module is used to collect emotional states, feedback results, and information on children's interactions with virtual digital avatars in real time, and to extract children's emotional expression characteristics and preference characteristics.

[0051] This invention provides a two-way emotional support system for left-behind children. Utilizing a reward function and feedback optimized by time difference error, it provides customized emotional support and feedback through personalized analysis of children's emotional states. This not only enhances the system's understanding of children's emotions and its self-regulation capabilities but also improves learning efficiency and interactive experience by adjusting feedback strategies in real time. Simultaneously, through interaction with a virtual digital avatar, children can explore and express their emotions in a safe and controlled environment, which contributes to the development of their social skills and emotional intelligence. Furthermore, the design of the emotion recognition module ensures high accuracy and diversity in emotion recognition. This invention, through this personalized interactive experience, helps children grow in emotional cognition, expression, and management, while also providing a powerful auxiliary tool for parents and educators.

[0052] Specifically, in the data acquisition module, this embodiment of the invention employs a diversified perception technology strategy, encompassing multiple dimensions such as voice, facial expressions, and body movements. This multimodal data acquisition method comprehensively captures children's emotional states, thereby achieving more accurate and nuanced emotion recognition. The advantage of this multidimensional emotional data acquisition approach is that it significantly reduces the biases and errors that may arise from single-modal data acquisition. For example, relying solely on voice analysis may not accurately determine emotional states, as changes in voice can be influenced by various factors. Similarly, observing only facial expressions or body movements may not fully reveal an individual's emotional experience. By integrating multiple perceptual data, children's emotional states can be analyzed from different angles and levels, thereby improving the accuracy of emotion recognition. Furthermore, multimodal data acquisition enhances the richness of emotion recognition. It can not only identify basic emotional states such as happiness, sadness, or anger, but also capture more subtle emotional changes such as excitement, curiosity, or confusion. This rich emotional data is crucial for understanding children's emotional development and needs.

[0053] For diversified perception technology strategies, it relies on speech recognition, image and facial expression recognition, and behavioral action analysis technologies to acquire multi-dimensional data. For speech recognition, by deeply analyzing key parameters such as pitch, speech rate, and volume in children's speech, it is possible to accurately infer their emotional state. These parameters are closely related to children's emotions; for example, a high pitch may indicate anxiety, while a slow speech rate and soft volume may be associated with fatigue. Analysis of these speech features allows the system to gain a deeper understanding of children's emotional experiences. For image and facial expression recognition, computer vision technology is used to meticulously identify and analyze children's facial expressions. Facial expressions are a direct reflection of emotional state; by recognizing changes in expression such as smiles, frowns, and blinks, the system can further enrich its judgment of children's emotional state. This image recognition technology provides intuitive and powerful data support for emotion analysis. For behavioral action analysis, deployed sensors can monitor children's actions and behaviors in real time. Activity level, frequency, and patterns of movement are all important indicators of emotional state. For example, frequent activity may indicate that a child is in a state of excitement, while abnormal behavior may be a signal of emotional fluctuation. Behavioral analysis provides a dynamic and comprehensive perspective for the identification of emotional states.

[0054] Data security and privacy protection are paramount in the collection and processing of children's data. Therefore, the data collection module includes a data anonymization submodule and a data privacy and security submodule, which can be referenced... Figure 2 The two sub-modules employ anonymization and multi-layered encryption methods respectively when collecting, storing, and processing children's emotional data to fully ensure that children's privacy and security are not violated.

[0055] Specifically, the anonymization process used in the data anonymization submodule involves the following steps:

[0056] (1) At the beginning of data collection, information that can directly identify a child, such as their real name, ID number, and home address, is stripped away. A hash function is used to perform an irreversible hash operation on the child's unique identification number to generate an anonymized identifier of a preset length. All subsequent data associations are based on this anonymized identifier, which ensures traceable data ownership and avoids the leakage of sensitive identity information. The preset length can be flexibly set according to the actual situation.

[0057] (2) Desensitize parts of the collected voice, image, and behavioral data that may indirectly expose identity or privacy. For example, in voice data, specific names and place names mentioned in the voice are converted to text through speech recognition and then replaced with general words using keyword replacement algorithms; in image data, background parts other than key facial features are processed using image blurring and pixelation techniques, and if there are identifiable scenes in the background, they are also blurred to reduce the risk of privacy exposure; for behavioral data, location information such as the location where the action occurs is anonymized and gridded, and only core analysis data such as action type and frequency are retained.

[0058] (3) Leveraging the decentralized, immutable, and traceable characteristics of blockchain, the anonymized data is packaged and stored in blocks. Each block contains information such as the hash value of the previous block, the hash value of the current data batch, and a timestamp, ensuring the integrity of the data storage chain. On-chain nodes only store the data hash value associated with the anonymity identifier, while the actual data is stored in an encrypted cloud. When nodes interact to verify the integrity and authenticity of the data, there is no need to expose the original data, further enhancing anonymity and privacy security. Access rules are set through smart contracts, and only authorized data analysis and modules can access the corresponding anonymized data for subsequent sentiment analysis and interaction processes if they comply with the rules.

[0059] Specifically, the multi-layered encryption method used in the data privacy and security submodule involves the following steps:

[0060] (1) After obtaining anonymized data from the data anonymization processing submodule, the data privacy and security submodule uses a symmetric encryption algorithm to initially encrypt the anonymized emotional state data with a random key generated by the device, thus obtaining the initially encrypted data. This step aims to prevent the data from being stolen and deciphered during temporary storage or initial transmission on the device. The encrypted data packet is accompanied by metadata such as the collection timestamp and device identification number, which facilitates subsequent traceability and management.

[0061] (2) When the initially encrypted data is transmitted to the system backend over the network, an encrypted channel is established based on a secure transmission protocol. Simultaneously, an asymmetric encryption algorithm is used, employing a public key pre-allocated and stored in the trusted storage area of ​​the local device to perform secondary encryption on the key used in the symmetric encryption algorithm. This encrypted data is then transmitted along with the data previously processed using the symmetric encryption algorithm (i.e., the initially encrypted data). This layer of encryption ensures that even if the data is intercepted during network transmission, it is difficult for a third party to obtain the decryption key and the original data content. This double encryption ensures the security of the transmission link.

[0062] (3) When the encrypted data first arrives at the cloud server, the server uses the private key to decrypt the symmetric encryption key, and then uses the symmetric key to unpack and obtain the original anonymized emotional state data. Then, according to the cloud storage strategy, the data is encrypted again by data blocks based on block encryption technology and stored in a specific area of ​​the distributed storage architecture. During storage, an index is built based on information such as data classification and child identifier hash value to ensure the security of persistent data storage in the cloud and prevent unauthorized access and viewing by cloud storage administrators and other internal personnel.

[0063] In the data transmission and storage stages, advanced encryption technology is used to build a protective barrier for sensitive data, effectively resisting external attacks and preventing the risk of data leakage. Simultaneously, adhering to the principles of anonymization and data minimization, the system only collects necessary emotional data and implements strict management measures for this sensitive information to ensure that all operations comply with relevant privacy protection regulations. This comprehensive data security and privacy protection solution not only effectively safeguards children's privacy rights but also significantly enhances the trust of parents and the public in the AI ​​system.

[0064] Furthermore, the emotion recognition module includes an emotion model submodule. This submodule, based on the continuous collection and analysis of children's emotional and behavioral data, constructs a refined, personalized emotion model for each child. The specific method for building this emotion model is as follows:

[0065] (1) Data cleaning and normalization: After continuously collecting children's multimodal emotional and behavioral data (including speech tone, speech rate, volume parameters, facial expression feature values, frequency and activity of body movements, etc.), invalid data is first removed, such as abnormal speech spikes caused by sensor failure and data points that cannot be recognized due to image blur. Then, normalization processing is performed on different types of data to obtain normalized data. Speech parameters are mapped to the [0,1] interval according to the set normal range (such as tone in a certain frequency range, speech rate in a certain number of words per second range). Features extracted by facial expression recognition (such as the angle of mouth corners, the proportion of changes in the distance between eyebrows and eyes, etc.) are normalized according to the statistical maximum and minimum values. Behavioral movement data are also normalized according to the quantitative value of movement activity to ensure that the data of each dimension are at the same comparable level, which is convenient for subsequent comprehensive analysis.

[0066] (2) Time series alignment and labeling: Since emotional and behavioral data are continuously collected over time, after normalization, the voice, facial expression, and action data are precisely aligned according to timestamps to construct a dataset organized by time series. At the same time, combining manual labeling and intelligent algorithm pre-labeling, basic emotional labels (happiness, sadness, anger, calmness, etc.) are assigned to each time segment of data. Manual labeling is completed by professionally trained personnel based on comprehensive observation and judgment of the data, while intelligent pre-labeling relies on pre-trained emotional classification models for initial identification. Subsequent manual review and calibration ensure the accuracy of the labeling.

[0067] (3) Multimodal feature extraction: extract spectral features and prosodic features (in addition to pitch, speech rate, and volume, pauses, stress distribution, etc.) from speech data to reflect emotional tendencies; extract facial action unit (AU) combination features from facial expression data of images, with different AUs corresponding to different muscle movements to reflect changes in expression; extract action trajectory features (such as limb swing amplitude, walking path complexity, etc.) and action transition frequency features (the frequency of switching from stillness to activity, and from one action to another) from behavioral action data to comprehensively capture children's emotional and behavioral characteristics.

[0068] (4) Feature selection and dimensionality reduction: Principal component analysis (PCA) and mutual information methods are used to select highly relevant and low-redundancy features. PCA projects multidimensional features onto a low-dimensional principal component space based on the data covariance matrix, retaining principal components with a cumulative contribution rate exceeding a certain threshold (e.g., 90%). Mutual information methods measure the information correlation between each feature and the labeled sentiment tags, removing features with low correlation, reducing the computational load and overfitting risk of model training, and extracting the most representative feature subset for modeling.

[0069] (5) Select the basic model architecture and build the basic framework of the emotion model by combining the Long Short-Term Memory Network (LSTM) and Convolutional Neural Network (CNN) architecture in deep learning. LSTM is good at processing time series data and can capture the dynamic changes and correlations of emotions over time, while CNN has efficient feature extraction and combination capabilities for extracted local features (such as facial expression image features and behavioral action segment features). The combination of the two can comprehensively analyze the emotional information contained in multimodal data.

[0070] (6) During model training, the selected feature data is divided into a certain proportion (e.g., 80% as the training set, 10% as the validation set, and 10% as the test set). The training set data is input into the model, and the model parameters (e.g., the weight matrix in the LSTM unit, the parameters of the CNN convolution kernel, etc.) are adjusted based on the backpropagation algorithm and stochastic gradient descent optimizer. The goal is to minimize the cross-entropy loss function, and training is iterated continuously to gradually improve the model's accuracy, recall, and other evaluation metrics on the validation set while avoiding overfitting. Training is stopped when the metrics do not improve significantly after several consecutive iterations. In addition, the number of iterations or conditions can be preset to determine whether to stop training.

[0071] (7) Model optimization and evaluation: After training, the generalization performance of the model is evaluated using a test set, and the reasons for misclassification and missed classification are analyzed. By introducing an adversarial training mechanism, a discriminator network is added to distinguish between the emotional classification output by the model and the real label, prompting the generator (the main body of the emotional model) to generate more accurate results. Furthermore, the transfer learning fine-tuning strategy can be used to learn from the common and mature features and parameters in the emotional models of other similar groups of children, optimize and adjust the current personalized model for children, improve its performance and adaptability, and finally obtain a personalized emotional model that can accurately reflect the emotional characteristics and behavioral habits of specific children.

[0072] The emotional model established in this invention not only captures children's emotional characteristics but also reflects their behavioral habits, providing a foundation for personalized interaction. During interactions with children, the system can dynamically adjust its companionship strategy based on these customized emotional models to meet the children's immediate needs. Considering the individual differences in children's emotional states, each child possesses unique emotional expressions and needs. This system, through personalized emotional models, can more accurately understand and adapt to these differences, thereby providing more attentive companionship. The establishment of this model ensures that the system can accurately meet each child's specific emotional needs when accompanying them, providing them with tailor-made emotional support.

[0073] Specifically, the feedback module can generate customized feedback based on the child's current emotional state. For example, when a child feels anxious or frustrated, the module can provide timely comfort and encouragement; when a child makes progress, it can offer positive affirmation and suggestions. This personalized feedback ensures that the interactive content is highly aligned with the child's emotional needs, thereby enhancing the quality and effectiveness of the interaction. To further improve the quality of companionship, the module employs advanced algorithms such as reinforcement learning or transfer learning to continuously learn and optimize its feedback results.

[0074] For example, the specific steps for optimizing the feedback module based on reinforcement learning are as follows:

[0075] (1) The interaction scenario with the child is defined as the "environment" for reinforcement learning, and the environment state S is represented by a combination of multi-dimensional information. Specifically, it includes the child's current emotional state (categories such as happiness, sadness, and anxiety determined by the emotion recognition module, and their corresponding confidence scores), recent emotional state trends (e.g., a shift from calm to gradual anxiety over the past hour), interactive behaviors (e.g., whether the child responded positively, remained silent, or resisted in the past 5 interactions), and the type of interaction scenario (game scenario, tutoring scenario, etc.). For example, the state vector can be represented as S = [s1, s2, s3, s4, s5], where s1, s2, s3, s4, and s5 represent the emotion category code, emotion confidence score, emotion trend code, interaction behavior frequency code, and scenario type code, respectively. Through such a detailed and quantified state vector, the system can accurately grasp the child's comprehensive state at the moment of interaction.

[0076] (2) Define the "Action (A)" of the feedback module, which is the set of possible feedback strategies. Actions include verbal comforting phrases (such as "Don't worry, the difficulty is only temporary, we'll solve it together" for anxiety), encouraging statements (such as "You've improved a lot this time compared to last time, keep up the good work and you'll definitely do even better" for progress scenarios), and guiding questions (such as "What did you find most interesting about this assignment?" for stimulating expression in a learning scenario); it also covers non-verbal actions, such as displaying friendly expressions (smiles, encouraging eye contact) of virtual digital avatars, playing soothing or cheerful background music to match the emotional atmosphere, etc. The action space is discrete, and each action corresponds to a number for the algorithm to choose and execute.

[0077] (3) Construct a reward function R to evaluate the quality of each feedback action and guide the feedback module to learn better strategies. When a child's emotional state changes from negative (anxiety, frustration) to positive (calm, happiness), a high positive reward (e.g., +5) is given; if the child responds positively to the feedback verbally or actively shares their thoughts, a reward (e.g., +3) is given; conversely, if the child continues to have negative emotions or remains silent and resistant, a negative reward (e.g., -2) is given. At the same time, additional rewards are given based on long-term indicators, such as increased interaction frequency over a period of time or sustained progress in learning or life tasks. The comprehensive reward calculation formula is as follows:

[0078]

[0079] Where R1 represents the immediate feedback reward; R2 represents the long-term effect reward; f1(·) and f2(·) represent nonlinear functions used to adjust the reward, which may include thresholds, activation functions, or squared terms. For example, f1(R1) = R1 2(Squaring the immediate reward to emphasize the dramatic change in emotion) or f2(R2) = log(R2+1) (logarithmic adjustment for long-term rewards to avoid over-punishment). γ t Let γ represent the discount factor, indicating the attenuation of the impact of long-term rewards on future feedback, γ∈(0,1), where t is the time step at the current moment. This factor attenuates past rewards to emphasize recent learning outcomes. w1, w2, and w3 are weighting coefficients used to balance the effects of immediate, long-term, and historical rewards. The contributions of different factors can be controlled by adjusting these weighting coefficients. This represents cumulative rewards, and incorporating historical, real-time feedback rewards helps reflect long-term performance improvements. The reward function is designed to ensure that feedback focuses on both immediate emotional improvement and the effectiveness of long-term support.

[0080] (4) A Deep Q-Network (DQN) or its variants (such as Dual DQN or Priority Experience Replay DQN for improved stability and efficiency) is used as the policy network. The input is a state vector, and the output is the Q-value (estimated action value) of each action A. The action with the highest Q-value is selected for feedback based on probability. Initially, the network weights are randomly initialized. As experience data accumulates through interaction, the state S... t Action A t Reward R t+1 Next state S t+1 The quadruple is represented as: (S t A t ,R t+1 ,S t+1 The data is stored in the experience playback buffer. Batch data is periodically sampled from the buffer based on the time difference error TD, the mathematical expression of which is:

[0081]

[0082] Where γ is the discount factor used to measure the importance of future rewards, and t represents time. R t+1 The reward at the current time t is the reward obtained from the environment, and it represents the reward at state S. t Take action A t Then, the system provides immediate feedback to the agent. max(Q(S) t+1 ,α,θ - )+λ·A(S t+1 ,α,θ - The expression ) represents the Q-value calculated by the target network, incorporating the advantage function. Q(·) represents the value function, A(·) represents the advantage function, a represents the action parameters, and λ represents the weights. Q(S) t A t ;θ) represents the state S in the current state. tNext, take action A. t At that time, the policy network (parameter A) t The Q-value estimated by θ is the output of the current Q-network. This value reflects the expected reward of the current policy network for taking a specific action in a given state. t This represents the current state, indicating the environment or situation. At each time step, an action is selected based on the current state. A t This represents the action selected at the current time t. It is in state S. t The action taken based on the current policy. θ represents the parameters of the policy network, i.e., the model's weights. By training and updating these weights, the behavioral policy can be gradually improved. - This represents the parameters of the target network. The network weights are updated using backpropagation with gradient descent, optimizing the policy network so that it gradually learns to provide optimal feedback actions in different child states, thus continuously upgrading the companionship method.

[0083] (5) To avoid getting stuck in local optima, an exploration mechanism is introduced, such as a greedy strategy. The probability ∈ is used to randomly select actions to explore new feedback possibilities, and the probability 1-∈ is used to select the optimal action according to the network Q value. As the number of training rounds increases, the Q value is gradually reduced (e.g., from the initial 0.9 to 0.1 exponentially), balancing the "utilization" of known effective feedback and the "exploration" of unknown better feedback strategies. This ensures that the feedback module adapts to the dynamic emotional needs and growth changes of children in long-term interaction, and steadily improves the quality of companionship.

[0084] The feedback module possesses self-evolving capabilities, enabling it to continuously learn and adapt to children's emotional changes. Over time, by accumulating and analyzing children's emotional data, the system gradually improves the accuracy of its emotional state recognition and response. This self-optimization mechanism endows the AI ​​system with the ability to continuously improve, allowing it to tailor emotional support plans for each child.

[0085] Building upon emotion recognition and feedback, interactive scenarios can be designed to create a multi-dimensional interactive experience for children. Therefore, the digital avatar generation module can simulate diverse everyday scenarios, including learning, games, and social interactions. These simulations provide children with a safe and educational environment, allowing them to receive emotional support and guidance in various situations. Through interaction with children, the system can not only identify their emotional states but also encourage and guide them to express their emotions. When children encounter emotional distress, the system can help them learn emotional regulation through appropriate guidance and suggestions, thereby alleviating emotional stress. This intelligent interactive design significantly enhances the system's companionship capabilities, making it a partner that not only recognizes emotions but also promotes children's emotional expression and self-regulation through interaction. Furthermore, the digital avatar generation module can utilize technologies such as Generative Adversarial Networks (GANs) to generate realistic digital avatars and engage in emotional interaction with children.

[0086] Furthermore, the Emotion and Preference Learning module possesses self-learning and adaptive capabilities. As interaction with children deepens, this module continuously learns children's emotional expression patterns and preferences, thereby providing more personalized and precise emotional support. The Emotion and Preference Learning module includes: a data integration submodule, used to collect emotional states, feedback results, and information on children's interactions with virtual digital avatars in real time, generating multi-source data; a time-series recording submodule, used to record multi-source data to form time-series data; a feature extraction submodule, used to extract emotional expression features and preference features from the time-series data; and a preference modeling submodule, used to build an emotion and preference model based on the emotional expression features and preference features, which can be referenced... Figure 3 As shown.

[0087] Specifically, the data integration submodule collects various data points generated during each interaction between the system and the child. This includes real-time emotional state data of the child determined by the emotion recognition module (such as emotion category and intensity), feedback strategies adopted by the feedback module and the child's immediate reactions to these responses (such as verbal responses, facial expressions, and behavioral changes), and interactive scene information simulated by the digital avatar generation module (scene type, duration, etc.). These diverse data sources comprehensively reflect the child's performance in different situations.

[0088] Specifically, the time-series recording submodule is used to accurately record the multi-source data generated by the data integration submodule, forming time-series data. Each timestamp corresponds to a complete interaction event, ensuring the continuity and traceability of the data, so as to facilitate subsequent analysis of changes in children's emotional expression patterns and preferences.

[0089] Specifically, for the feature extraction submodule, key features of children's emotional expression are extracted from the time-series data generated by the time-series recording submodule. For example, for speech data, features such as tone, speech rate, and frequency of use of interjections are extracted to reflect the speech characteristics under different emotions; from facial expression data, the frequency, duration, and expression transition patterns of specific expressions are analyzed; for behavioral action data, features such as the activity level, repetition, and correlation between specific actions and emotions are extracted. Through the extraction of these features, children's emotional expression patterns can be more meticulously depicted. In addition, children's preferred responses to different feedback strategies (such as different types of comforting words, encouragement methods, and interactive scenario themes) are analyzed. Observing whether children show more positive responses (such as actively sharing more thoughts, significant improvement in mood, etc.) or relatively negative attitudes (such as silence, avoidance, etc.) when faced with different feedback, the types of feedback and interactive scenarios that children prefer to accept are identified as preference features.

[0090] Specifically, for the preference model submodule, the extracted emotional expression features and preference features are comprehensively analyzed, and data analysis techniques (such as cluster analysis and association rule mining) are used to find patterns and regularities. For example, cluster analysis is used to group children's emotional expression features under similar emotions to discover common emotional expression patterns in different emotional states; association rule mining is used to find the association between specific emotional expression features and preference features, such as when children are anxious, they have a higher preference for gentle comforting words and quiet interactive scenarios. Based on the emotional expression features and preference features obtained from the analysis, a model of children's emotions and preferences is established or updated. If it is the first time the model is established, the initial parameters and structure of the model are determined according to the results of the initial stage analysis; if the model already exists, the parameters of the model (such as weights, thresholds, etc.) are adjusted according to the new analysis results to make the model more accurately reflect the children's current emotional expression patterns and preferences.

[0091] Furthermore, for the sentiment and preference model built by the preference model submodule, reinforcement learning algorithms (similar to those used in the feedback module mentioned earlier) can be employed. Positive responses from children to feedback are considered positive rewards, while negative responses are considered negative rewards. Based on the reward situation, the model adjusts its emotional support strategies output under different emotional states and interaction scenarios, allowing the model to continuously learn how to provide more personalized emotional support that aligns with children's preferences. For example, if a child responds positively to a certain type of encouraging language, the model will be more inclined to output similar encouraging language in subsequent similar situations. In addition, as interactions with children deepen, new data is continuously collected, and the process of feature extraction, analysis, and model updates is repeated to achieve continuous optimization of the sentiment and preference model.

[0092] The data integration, time-series recording, feature extraction, and preference modeling submodules work together to enable the emotion and preference learning module to continuously adapt to changes in children's emotional expression patterns and preferences, consistently providing more personalized and precise emotional support. Furthermore, the system can process and analyze data from various sources to form a comprehensive data view, which helps in a deeper understanding of children's behavioral and emotional changes.

[0093] In terms of real-time processing of emotional data and rapid response to emotional support, this invention employs a system architecture combining cloud and local computing. Under this architecture, the AI ​​device handles the initial processing and analysis of emotional data locally, providing users with immediate emotional feedback and ensuring real-time interaction. Simultaneously, the cloud handles the storage and in-depth analysis of long-term emotional data, continuously optimizing the emotional model using the cloud computing platform to support the ongoing improvement of personalized emotional feedback. This design not only ensures the efficiency of the AI ​​system in providing immediate feedback but also enhances the system's intelligence level through in-depth data analysis in the cloud, achieving precise optimization of personalized feedback. Overall, this architecture effectively improves the performance of the AI ​​system in emotion recognition and response, providing users with a more intelligent and personalized service experience.

[0094] Example 2

[0095] Based on the same inventive concept, this embodiment provides a two-way emotional companionship method for left-behind children. The principle of solving the problem is similar to that of the two-way emotional companionship system for left-behind children provided in Embodiment 1, and the repeated parts will not be described again.

[0096] Reference Figure 4 As shown, this embodiment provides a method for two-way emotional companionship for left-behind children, including:

[0097] Real-time collection of children's emotional state data;

[0098] Analyze and identify emotional state data to obtain children's emotional state;

[0099] Based on emotional state, the system analyzes children's emotional needs and provides personalized feedback. The feedback is optimized using a reward function and time difference error. The formula for the reward function is as follows:

[0100]

[0101] The formula for calculating time difference error is:

[0102]

[0103] Where R1 represents the immediate feedback reward, R2 represents the long-term effect reward, f1(·) and f2(·) represent nonlinear functions, t represents time, γ represents the discount factor, and w1, w2, w3 represent weighting coefficients; R t+1 Let θ represent the reward at the current time t. - S represents the parameters of the target network, θ represents the parameters of the policy network, and S represents the parameters of the policy network. t+1 S represents the state at time t+1. t A represents the state at time t. t Let represent the action at time t; 'a' represent the action parameter; 'λ' represent the weight; Q(·) represent the value function; and A(·) represent the advantage function.

[0104] Based on the feedback results, a virtual digital avatar is generated;

[0105] It collects emotional states, feedback results, and information on children's interactions with virtual digital avatars in real time, and extracts children's emotional expression characteristics and preference characteristics.

[0106] Specifically, before collecting children's emotional state data in real time, the process includes identifying and stripping children's identity information to generate an anonymized identifier of a preset length.

[0107] Specifically, after extracting children's emotional expression characteristics and preference characteristics, the process includes establishing or updating an emotion and preference model; when an emotion and preference model does not exist, modeling is performed based on the emotional expression characteristics and preference characteristics; when an emotion and preference model exists, the emotional expression characteristics and preference characteristics are analyzed, and the parameters of the emotion and preference model are updated based on the analysis results.

[0108] Example 3

[0109] This embodiment provides an electronic device, including the two-way emotional companionship system for left-behind children provided in Embodiment 1.

[0110] Example 4

[0111] This embodiment provides a computer program product, including computer program instructions, which, when executed on a computer, cause the two-way emotional companionship method for left-behind children provided in Embodiment 2 to be executed.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A two-way emotional companionship system for left-behind children, characterized in that, include: A data acquisition module is used to collect children's emotional state data in real time; wherein, the data acquisition module includes a data anonymization processing submodule; the data anonymization processing submodule strips away the identification information, uses a hash function to perform an irreversible hash operation on the child's unique identification number, and generates an anonymized identifier of a preset length; and encrypts and transmits the collected emotional state data. An emotion recognition module is used to analyze and recognize the emotion state data to obtain the child's emotion state. The emotion recognition module includes an emotion model submodule. This submodule is used to capture the child's emotional characteristics and establish an emotion model based on the emotion state data. The method for establishing the emotion model is as follows: the emotion state data is time-series aligned and labeled to obtain a time-series organized and labeled dataset; multimodal feature extraction and feature filtering / dimensionality reduction are performed on the dataset to obtain a feature subset; and the emotion model is established based on the feature subset using a long short-term memory network combined with a convolutional neural network. The feedback module is used to analyze the child's emotional needs based on the emotional state and provide personalized feedback results; wherein the feedback results are optimized using a reward function and time difference error, and the calculation formula of the reward function is: ； ； ； The formula for calculating the time difference error is: ； in, This indicates an immediate feedback reward. This indicates a long-term reward. and Represents a nonlinear function. Indicates time, Indicates the discount factor. Indicates the weighting coefficient; Indicates the current time The reward The parameters representing the target network, The parameters representing the policy network, Indicates time state, Indicates time state, Indicates time The action; Indicates action parameters, Indicates weight; Represents the value function. Represents the dominance function; A digital avatar generation module is used to generate a virtual digital avatar based on the feedback results; The emotion and preference learning module is used to collect the emotional state, the feedback results, and the interaction information between the child and the virtual digital image in real time, and to extract the child's emotional expression characteristics and preference characteristics.

2. The two-way emotional companionship system for left-behind children according to claim 1, characterized in that, The emotion and preference learning module includes: a data integration submodule, used to collect the emotional state, the feedback results, and the interaction information between the child and the virtual digital image in real time, and generate multi-source data; a time series recording submodule, used to record the multi-source data to form time series data; a feature extraction submodule, used to extract emotional expression features and preference features from the time series data; and a preference model submodule, used to establish an emotion and preference model based on the emotional expression features and the preference features.

3. The two-way emotional companionship system for left-behind children according to claim 1, characterized in that, The data acquisition module also includes a data privacy and security submodule, which is used to encrypt and transmit the emotional state data.

4. A method for providing two-way emotional companionship to left-behind children, implemented using the two-way emotional companionship system for left-behind children as described in any one of claims 1 to 3, characterized in that, include: Real-time collection of children's emotional state data; The emotional state data is analyzed and identified to obtain the child's emotional state; Based on the emotional state, the child's emotional needs are analyzed, and personalized feedback is provided; wherein the feedback is optimized using a reward function and time difference error, and the calculation formula for the reward function is: ； The formula for calculating the time difference error is: ； in, This indicates an immediate feedback reward. This indicates a long-term reward. and Represents a nonlinear function. Indicates time, Indicates the discount factor. Indicates the weighting coefficient; Indicates the current time The reward The parameters representing the target network, The parameters representing the policy network, Indicates time state, Indicates time state, Indicates time The action; Indicates action parameters, Indicates weight; Represents the value function. Represents the dominance function; Based on the feedback results, a virtual digital avatar is generated; The system collects the emotional state, feedback results, and interaction information between the child and the virtual digital avatar in real time, and extracts the child's emotional expression characteristics and preference characteristics.

5. A method for two-way emotional companionship for left-behind children according to claim 4, characterized in that, Before collecting children's emotional state data in real time, the process includes identifying and stripping the children's identity information to generate an anonymized identifier of a preset length.

6. A method for two-way emotional companionship for left-behind children according to claim 4, characterized in that, After extracting children's emotional expression features and preference features, the process includes establishing or updating an emotion and preference model; when the emotion and preference model does not exist, modeling is performed based on the emotional expression features and preference features; when the emotion and preference model exists, the emotional expression features and preference features are analyzed, and the parameters of the emotion and preference model are updated based on the analysis results.

7. An electronic device, characterized in that, This includes a two-way emotional companionship system for left-behind children as described in any one of claims 1 to 3.

8. A computer program product, characterized in that, It includes computer program instructions that, when executed on a computer, cause the method of any one of claims 4 to 6 to be performed.

Images