Emotion accompanying processing system and method based on multi-dimensional information

Abstract

The invention discloses an emotion accompanying processing system and method based on multi-dimensional information, and the system comprises a cloud processing system, and a signal collection module, an emotion recognition module, a generative large model emotion dialogue module fused with positive psychology, a personalized and anthropomorphic robot interaction module, and an emotion analysis suggestion and health early warning module which are in communication connection. And the webpage client display support module is used for providing personalized emotional dialogues for the user in a real-time analysis and active dialogue generation mode. According to the invention, personalized and flexible emotion support is realized.

Description

Technical Field

[0001] The present invention relates to the field of applications related to emotional companionship, and particularly to an emotional companionship processing system and method based on multi-dimensional information. Background Art

[0002] With the development of society and the progress of technology, emotional companion robots, as a new type of intelligent hardware, have gradually been widely applied to the lives of the elderly, children, and groups with emotional needs (such as the elderly living alone, patients with depression, teenagers, etc.).

[0003] Existing emotional companion robot systems usually focus on a single technical field, such as emotional recognition based on speech recognition, facial recognition, or sensor data, or simple emotional conversations through preset scripts. However, single technical means often struggle to comprehensively and accurately identify complex emotional states. The goal of emotional companion robots is not only to conduct simple interactions, but also to help alleviate loneliness, mood swings, and even provide mental health support through intelligent technology, promoting the mental health and emotional stability of users. Especially in an environment where emotional needs are highly diverse and situations change rapidly, the intelligence level and personalized services of existing systems are still insufficient to comprehensively address the emotional fluctuations and changing needs of users. Summary of the Invention

[0004] The purpose of the present invention is to provide an emotional companionship processing system based on multi-dimensional information, to solve the problems of the prior art, and to improve the intelligence level of emotional companionship services, further improving the emotional health and quality of life of users.

[0005] To achieve the above technical purpose, the present invention provides an emotional companionship processing system based on multi-dimensional information, which includes a cloud processing system, and a signal acquisition module, an emotional recognition module, a generative large model emotional dialogue module integrating positive psychology, a personalized and personified robot interaction module, an emotional analysis and advice and health warning module, a web client display support module, and a historical emotional companionship archive that are communicatively connected; the cloud processing system internally has the following data exchange connection and processing method modules and executes: S1: Collect multi-dimensional biological information of the user's body and perform fusion processing; S2: Perform emotional analysis and calculation on the fused data to analyze, identify, and capture the emotional state of the user; S3: Provide personalized emotional conversations and psychological counseling, emotional analysis advice, health analysis advice, and warnings for the user through real-time analysis and active dialogue generation based on the emotional state.

[0006] As a further improvement, the emotion recognition module, the generative large model emotion dialogue module integrating positive psychology, the personalized and personified robot interaction module, and the emotion analysis advice and health warning module are arranged in the cloud processing system. The signal acquisition module includes a robot for collecting emotion data and a wearable device for collecting physiological data. The web client display support module is arranged in the user terminal device. The signal acquisition module is in bidirectional data communication connection with the user terminal device. The user terminal device is in bidirectional data communication connection with the cloud processing system. The robot is in unidirectional data communication connection with the cloud processing system and receives the instruction information of the cloud processing system. The historical emotion companionship file is arranged in at least one of the user terminal device and the cloud processing system.

[0007] As a further improvement, the instruction of the cloud processing system received by the robot is processed and issued by the generative large model emotion dialogue module integrating positive psychology, the personalized and personified robot interaction module, and the emotion analysis advice and health warning module.

[0008] As a further improvement, it further includes a guardian terminal device. The guardian terminal device is in unidirectional data communication connection with the cloud processing system and receives the data information of the cloud processing system. The data information received by the guardian terminal device from the cloud processing system is processed and issued by the emotion analysis advice and health warning module.

[0009] As a further improvement, the user terminal is provided with an emotion diary report module and a health report module. The sub-modules included in the emotion diary report module are: emotion trend, conversation summary, reminder and advice. The sub-modules included in the health report module are: health data visualization, health reminder and advice. The downstream data output of the conversation summary is connected to the user web page. Its upstream data input is connected to the historical emotions, conversations, user portraits and historical health in the historical emotion companionship file. And between the conversation and the conversation summary, there are also arranged a processing module and an information extraction module in sequence. The processing module is also connected to the historical health module. The processing module includes a text preprocessing module and a biological data association module. The biological data association module has an internal time series analysis machine modeling and a causal association model. The information extraction module has a natural language understanding NLP technology module inside. The conversation summary has a large model generation module and a Prompt engineering technology module inside. The conversation summary is first based on: the historical emotions, conversations, user portraits in the historical emotion companionship file, and then summarizes the real-time and historical conversations through natural language understanding NLP technology, large model generation, and Prompt engineering technology to generate keyword sentences.

[0010] As a further improvement, the dialogue first combines the historical health in the historical emotional companionship file, then performs text preprocessing and physiological data association, and then conducts information extraction; the physiological data association uses time series analysis machine modeling and causal association models, and the information extraction uses the natural language understanding NLP technology.

[0011] As a further improvement, it further includes: S4: Conduct data analysis and trend prediction based on the historical emotional companionship file, actively perceive and judge whether to initiate a dialogue, provide personalized and personalized interaction, and integrate psychological and health analysis suggestions and warnings to enhance the user's physical and mental health management; S5: Generate emotional analysis reports and health reports, and provide suggestions and warnings to the user; S6: Display text voice conversations to the user's guardians, so that the guardians can flexibly and intelligently master the emotional and basic health status information of the ward at any time and place.

[0012] Correspondingly, the present invention also provides an emotional companionship processing method based on multi-dimensional information, which is characterized in that it includes S1: Collecting multi-dimensional biological information of the user's body and performing fusion processing; S2: Performing emotional analysis calculation on the fusion data to analyze, identify and capture the user's emotional state; S3: According to the emotional state, providing personalized emotional dialogue and psychological counseling, emotional analysis suggestions, health analysis suggestions and warnings for the user through real-time analysis and active dialogue generation methods.

[0013] As a further improvement, in the step S3, the positive psychology processing method and the natural language generation dialogue of the generative large model are used to understand and adapt to the user's psychological needs in different situations, and provide emotional support for the user through emotional regulation and emotional counseling, and help the user perform emotional regulation, psychological comfort, cognitive reconstruction, etc.

[0014] As a further improvement, in the step S3, by real-time analyzing the user's health, emotional state and psychological needs, the dialogue is actively initiated to provide emotional counseling, psychological suggestions and health management.

[0015] As a further improvement, it further includes establishing and accumulating for a long time to form a historical emotional companionship file, and the historical emotional companionship file is adopted in the step S2 and / or S3; the historical emotional companionship file at least includes: the fusion data, the emotional state, the emotional dialogue and the psychological counseling.

[0016] The present invention realizes a method for emotion recognition and proactive dialogue generation based on multi-dimensional information fusion in the fields of artificial intelligence, affective computing, psychology, health monitoring, big data analysis, and generative dialogue technology, and is applied to an emotional companion robot system. The system fuses the user's voice, behavior, facial expressions, and physiological signals (such as heart rate, skin conductance response, etc.) to perform emotion recognition, and based on the theory of positive psychology, provides the user with emotional conversations, emotional analysis suggestions, health analysis suggestions, and warnings through a generative large model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the basic principle flow of the present invention; Figure 2 It is a schematic connection diagram of the basic principle of the present invention; Figure 3 It is a schematic diagram of the web page dialogue module of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] As Figures 1 to 3 shown, the present invention provides an emotional companion processing method based on multi-dimensional information, which includes S1: collecting and fusing multi-dimensional biological information of the user's body; S2: performing emotional analysis and calculation on the fused data to analyze, identify, and capture the emotional state of the user; S3: providing personalized emotional conversations and psychological counseling, emotional analysis suggestions, health analysis suggestions, and warnings for the user through real-time analysis and proactive dialogue generation methods.

[0020] The present invention provides an emotional companion processing method based on multi-dimensional information. Through the fusion of biological signals and emotional calculation, it accurately identifies the emotional state of the user and generates personalized conversations based on real-time analysis. This emotion recognition system can not only capture the emotional fluctuations of the user, but also understand and adapt to the psychological needs in different situations, and provide emotional support for the user through emotional regulation and mood counseling, so as to capture and analyze the emotional state of the user more accurately and comprehensively, and provide personalized and flexible emotional support.

[0021] As a further improvement, in the step S3, adopt positive psychology processing methods and natural language generation conversations of generative large models to understand and adapt to the psychological needs of the user in different situations, and provide emotional support for the user through emotional regulation and emotional counseling, helping the user with emotional regulation, psychological comfort, cognitive restructuring, etc.

[0022] The present invention combines generative large models (such as Transformer, GPT, etc.) with psychological theories. The system can generate conversations through natural language to achieve positive psychology guidance, helping users with emotional regulation, psychological comfort, cognitive restructuring, etc. By real-time analyzing the user's health, emotional state, and psychological needs, the robot can actively initiate conversations, provide emotional counseling, psychological suggestions, and health management.

[0023] As a further improvement, in the step S3, by real-time analyzing the user's health, emotional state, and psychological needs, actively initiate conversations, provide emotional counseling, psychological suggestions, and health management.

[0024] As a further improvement, it also includes establishing and accumulating over a long term to form a historical emotional companionship file, which is adopted in the step S2 and / or S3.

[0025] As a further improvement, the historical emotional companionship file at least includes: the fusion data, the emotional state, the emotional conversation, and the psychological counseling.

[0026] As a further improvement, it also includes S4: Conduct data analysis and trend prediction based on the historical emotional companionship file, actively perceive and judge whether to initiate a conversation, provide personalized and personified interactions, and integrate psychological and health analysis suggestions and warnings to enhance the user's physical and mental health management.

[0027] As a further improvement, in S4: If abnormal emotional fluctuations or changes in health conditions are detected, actively issue a warning reminder to assist the user in taking countermeasures as early as possible.

[0028] As a further improvement, in S4, it also includes: comparing and judging the historical emotional companionship file with the multi-dimensional biological information of the user's body in real time.

[0029] As a further improvement, it also includes S5: Generate emotional analysis reports and health reports, and provide suggestions and warnings to the user.

[0030] As a further improvement, an emotional diary report module and a health report module are arranged on the user terminal of the user. The emotional diary report module includes emotional trends, conversation summaries, reminders, and suggestions. The health report module includes health data visualization, health reminders, and suggestions.

[0031] As a further improvement, on the web page of the user terminal, text or visual conversation is used to generate real-time associated displays of emotions and health. The real-time associated displays include one or more of the following combinations: emotion - health - conversation, health - emotion - event, emotion - health - event, health and emotion, health and conversation summary.

[0032] As a further improvement, the conversation summary is first based on: the historical emotions, conversations, and user portraits in the historical emotional companionship file, and then real-time and historical conversations are summarized through natural language understanding (NLP) technology, large model generation, and Prompt engineering technology to generate keyword sentences. The keyword sentences include one or more of the following combinations: health, event, emotion, need, and suggestion.

[0033] As a further improvement, the conversation first combines the historical health in the historical emotional companionship file, then performs text preprocessing and physiological data association, and then performs information extraction. The physiological data association uses time series analysis machine modeling and causal association models, and the information extraction uses the natural language understanding (NLP) technology.

[0034] As a further improvement, it also includes S6: showing text - to - speech conversations to the guardian of the user, so that the guardian can flexibly and intelligently master the emotional and basic health status information of the ward at any time and anywhere.

[0035] The present invention also combines the user's historical health records and emotional data. When the system detects fluctuations in the user's emotional state or health data, the robot can actively initiate a conversation and provide personalized health and emotional support. This mechanism of combining historical data and real - time emotional analysis enables the robot not only to respond to the user's immediate needs but also to actively intervene when the user may be in an emotional trough or have health abnormalities.

[0036] In addition, the present invention also has a health monitoring and warning function. By combining the user's physiological data (such as heart rate, blood pressure, etc.) with emotional state analysis, it can monitor the user's health status in real - time and give corresponding health suggestions and warnings to ensure that the user can receive timely care and intervention in both emotion and health.

[0037] The technical solution of the present invention integrates cutting-edge technologies in multiple fields, including emotion computing, artificial intelligence, big data, psychology, and health monitoring technologies, providing users with comprehensive emotional companionship, mental health management, and health warning services, and having broad application prospects, especially suitable for the elderly, solitary groups, autistic patients, teenagers with large emotional fluctuations, and groups in need of long-term emotional support.

[0038] As a further improvement, in the S1, the collection of multi-dimensional biological information of the user's body includes: emotional data and physiological data; the emotional data includes: speech recognition, analysis, and collection of the user's speech intonation, speech rate, tone, etc., to evaluate their emotional state; facial expression recognition, analysis, and collection of the user's facial expressions to identify their emotional expressions; behavior recognition, analysis of the user's behavior patterns such as touch, body movements, sitting postures, walking styles, etc., to further infer their emotional state; the physiological data includes: heart rate, blood pressure, skin conductance response, brain waves, body temperature, blood oxygen, etc.

[0039] The present invention realizes an emotion recognition and active dialogue generation method based on multi-dimensional information fusion through the fields of artificial intelligence, emotion computing, psychology, health monitoring, big data analysis, and generative dialogue technology, and is applied to an emotional companion robot system. The system fuses the user's voice, behavior, facial expressions, physiological signals (such as heart rate, skin conductance response, etc.) for emotion recognition, and based on the theory of positive psychology, provides the user with emotional conversations, emotional analysis suggestions, health analysis suggestions, and warnings through a generative large model.

[0040] The present invention effectively integrates psychological theories, especially the application of positive psychology in emotional companionship. In the process of emotional analysis, meaningful information is extracted from complex physiological data and accurately interpreted. The system (robot) based on the active perception and emotional analysis function of historical data can provide timely emotional guidance or intervention when the user needs it, adjust the dialogue content and emotional expression according to the user's emotional needs, provide scientific and effective emotional counseling and psychological support in different situations, effectively fuse these data, and provide accurate emotional feedback. In addition, historical data (such as the user's past emotional fluctuations, health status, etc.) is used to trigger active conversations and provide effective solutions for emotional support.

[0041] Correspondingly, the present invention also provides an emotional companionship processing system based on multi-dimensional information, which executes a method for emotional companionship processing based on multi-dimensional information provided by the present invention. The system includes a cloud processing system, and a signal acquisition module, an emotion recognition module, a generative large model emotion dialogue module integrating positive psychology, a personalized and personified robot interaction module, an emotion analysis and advice and health warning module, and a web client display support module that are communicatively connected. The emotion recognition module, the generative large model emotion dialogue module integrating positive psychology, the personalized and personified robot interaction module, and the emotion analysis and advice and health warning module are arranged in the cloud processing system. The signal acquisition module includes a robot for collecting emotional data and a wearable device for collecting physiological data. The web client display support module is arranged in the user terminal device. The signal acquisition module is bidirectionally communicatively connected to the user terminal device. The user terminal device is bidirectionally communicatively connected to the cloud processing system. The robot is unidirectionally communicatively connected to the cloud processing system and receives the instruction information of the cloud processing system. The historical emotional companionship archive module is arranged in at least one of the user terminal device and the cloud processing system. Correspondingly, the cloud processing system internally has the following data exchange connection and processing method modules and executes: S1: Collect multi-dimensional biological information of the user's body and perform fusion processing; S2: Perform emotion analysis and calculation on the fusion data to analyze, identify, and capture the user's emotional state; S3: Provide personalized emotional dialogue and psychological counseling, emotion analysis advice, health analysis advice, and warnings for the user through real-time analysis and active dialogue generation methods.

[0042] As a further improvement, the instructions of the cloud processing system received by the robot are processed and issued by the generative large model emotion dialogue module integrating positive psychology, the personalized and personified robot interaction module, and the emotion analysis and advice and health warning module.

[0043] As a further improvement, it further includes a guardian terminal device. The guardian terminal device is unidirectionally communicatively connected to the cloud processing system and receives the data information of the cloud processing system. The data information received by the guardian terminal device from the cloud processing system is processed and issued by the emotion analysis and advice and health warning module.

[0044] As a further improvement, the user terminal is provided with an emotional diary report module and a health report module. The sub-modules included in the emotional diary report module are: emotional trend, dialogue summary, reminder and advice. The sub-modules included in the health report module are: health data visualization, health reminder and advice.

[0045] As a further improvement, the downstream data output of the dialogue summary is connected to the user's web page, and its upstream data input is connected to the historical emotions, conversations, user portraits, and historical health in the historical emotional companionship file. A processing module and an information extraction module are sequentially arranged between the conversation and the dialogue summary.

[0046] As a further improvement, the processing module is also connected to the historical health module. The processing module includes a text preprocessing module and a biological data association module. The biological data association module has an internal time series analysis machine modeling and a causal association model.

[0047] As a further improvement, the information extraction module has a natural language understanding NLP technology module inside, and the dialogue summary has a large model generation module and a Prompt engineering technology module inside. As a further improvement, the dialogue summary is first based on: the historical emotions, conversations, and user portraits in the historical emotional companionship file, and then summarizes real-time and historical conversations through natural language understanding NLP technology, large model generation, and Prompt engineering technology to generate keyword sentences.

[0048] As a further improvement, the conversation first combines the historical health in the historical emotional companionship file, then performs text preprocessing and physiological data association, and then performs information extraction; the physiological data association uses time series analysis machine modeling and causal association models, and the information extraction uses the natural language understanding NLP technology.

[0049] As a further improvement, the user's web page is also connected to the historical log for data.

[0050] As a further improvement, it also includes S4: Perform data analysis and trend prediction based on the historical emotional companionship file, actively sense and judge whether to initiate a conversation, provide personalized and personified interactions, and integrate psychological and health analysis suggestions and warnings to enhance the user's physical and mental health management.

[0051] As a further improvement, it also includes S5: Generate an emotional analysis report and a health report, and provide suggestions and warnings to the user.

[0052] As a further improvement, it also includes S6: Display text and voice conversations to the user's guardian, so that the guardian can flexibly and intelligently master the emotional and basic health status information of the ward at any time and place.

[0053] In the preferred embodiment of the present invention, its technical solution includes: 1. Physiological signal acquisition module: Using wearable devices (such as bracelets, necklaces, headphones, glasses, rings, etc.) as "robot nerve connectors" to collect users' physiological signals in real time, including but not limited to heart rate, galvanic skin response, body temperature, blood oxygen, brain waves, etc. The device automatically adjusts the sensor sensitivity according to the wearing position and signal acquisition method to ensure the accuracy and real-time nature of the data.

[0054] 2. Emotion Recognition Module: Speech Recognition: Analyze the user's speech intonation, speech rate, tone, etc. through speech recognition technology to evaluate their emotional state. Facial Expression Recognition: Use computer vision technology to collect the user's facial expressions through a camera and recognize their emotional expressions. Behavior Recognition: Further infer the user's emotional state by analyzing the user's behavior patterns such as touch, body movements, sitting postures, walking styles, etc. Multidimensional Signal Fusion Recognition: Integrate multi-dimensional information such as the user's speech, facial expressions, behavior patterns, physiological signals, etc., and use advanced emotion analysis algorithms for real-time emotion recognition. Multimodal Emotion Computing: Analyze information such as intonation and speech rate in speech through a deep learning model, analyze the user's emotional changes through facial expression recognition technology, analyze the user's body postures and activity states through behavior monitoring technology, and further verify the emotional state through physiological signal monitoring. Deep learning and multimodal emotion analysis models are used to real-time recognize the user's emotional state, such as joy, anxiety, depression, anger, etc.

[0055] 3. Generative Large Model Emotion Dialogue Integrating Positive Psychology: Based on the theoretical framework of positive psychology, use an emotion dialogue generation model to generate personalized dialogue content that matches the user's emotional state, and provide services such as emotional counseling and positive motivation. Generative large models (such as GPT, BERT, etc.) will consider the user's historical emotional data and physiological signals to provide personalized emotional support and health advice for the user.

[0056] 4. Personalized and Personified Robot Interaction: Historical Data Analysis: Establish the user's emotional profile and health profile based on the user's long-term accumulated emotional data and physiological data, and conduct data analysis and trend prediction. Active Dialogue Initiation: When the user's emotional state is abnormal or the physiological signal fluctuates, the robot can actively initiate a dialogue or issue a health warning, providing personalized emotional support or health advice. Robot Intelligence and Personification: According to the user's emotional needs and historical data, conduct deep learning model learning and training, and the robot can conduct personified dialogue interactions, possess emotional intelligence, form a unique "personalized robot", and enhance the user's sense of intimacy and trust.

[0057] 5. Emotion Analysis Suggestions and Health Warnings: Health data monitoring: By continuously monitoring physiological signals (such as heart rate, skin conductance, etc.), the health status of users is monitored in real time to determine whether there are physical and mental abnormalities. Health analysis and early warning: Combining the results of sentiment analysis and physiological signals, if the system detects abnormal emotional fluctuations or changes in health status, the robot will actively issue an early warning reminder to assist users in taking countermeasures as early as possible. Report generation: By connecting to the cloud server, the robot will continuously update the user's emotional and health profiles, and the real-time large model will generate sentiment analysis reports and health reports to provide users with suggestions and early warnings.

[0058] 6. Support for web client display: Presentation of sentiment analysis and health analysis: Display the results of the user's sentiment analysis, health status, historical data, and emotional fluctuation trend chart through the web client. Personalized suggestions and early warnings: The client display provides personalized emotional support suggestions, psychological counseling plans, and health warning information based on the user's emotional and health data. Guardian conversation: The client display has a text-to-speech conversation module, enabling guardians to flexibly and intelligently master the emotional and basic health status information of the ward at any time and place.

[0059] The present invention realizes the fusion of multi-dimensional information (such as voice, behavior, facial expression, physiological signals, etc.), accurately identifies and analyzes the emotional state of users. Based on positive psychology theory and generative large models, personalized emotional conversations and psychological counseling are provided for users. Through historical health data and sentiment analysis, the robot can actively perceive and initiate conversations to achieve personalized and personified interactions. Provide analysis suggestions and early warnings that integrate psychology and health to enhance the physical and mental health management of users.

[0060] Correspondingly, the technical implementation in the system of the technical solution of the present invention includes: 1. Hardware implementation: As Figure 2 shown, the robot neural connector 1 and the robot 2. The robot neural connector (such as wearable devices: bracelets, necklaces, headphones, glasses, rings, etc.) realizes the real-time acquisition of physiological signals through sensors. At the same time, the robot 2 collects emotional data through voice recognition, behavior analysis, and facial expression recognition. All wearable devices are equipped with wireless communication modules to ensure that data can be accurately and timely transmitted to the robot system for processing.

[0061] 2. Software implementation: Deploy the emotion recognition and generative dialogue model on the cloud server 3 to process the voice, behavior, facial expression, and physiological signal data input by users. The system generates conversation content that conforms to the user's emotional state through natural language processing technology to provide personalized emotional support. The robot 2 actively initiates conversations based on historical data and real-time sentiment analysis, continuously providing emotional companionship for users.

[0062] 3. Data transmission and analysis: The physiological signals collected by the device are transmitted to the cloud server 3 or the local robot 2 in real time through wireless communication for data storage, analysis and processing. The data processing module uses deep learning and sentiment analysis algorithms for real-time analysis and generates sentiment analysis reports and health recommendations.

[0063] 4. User interaction and feedback: Users can interact with the robot 2 through voice or text, and obtain emotional analysis results, health advice, and early warning information. The client provides a real-time emotional log, and users can view their emotional changes and adjust their emotions or lifestyles based on feedback. At the same time, the system provides user terminal devices 4 and guardian terminal devices 5, which can actively and passively obtain relevant information and status in a more systematic and timely manner.

[0064] Web page: The client APP web page includes: emotional diary report module and health report module.

[0065] The emotional diary report module on the client APP web page includes emotional trends, conversation summaries, reminders and suggestions.

[0066] The dialogue summary module uses natural language understanding (NLP) technology (denoising, word segmentation, named entity recognition (NER), information extraction, etc.), large model generation (Bert, GPT, etc.), and prompt engineering technology to summarize real-time and historical dialogues and generate keyword sentences, such as: a certain fluctuation (health), because of a certain (event), feeling a certain (emotion), a certain (need), a certain (suggestion). Health reflects the fluctuation of related data. Through dialogue, we can understand the cause event, the emotional data and changes, understand and formulate its needs, and give personalized suggestions.

[0067] The health report module on the client APP webpage includes health data visualization, health reminders and suggestions. Among them, through the association model, the real-time association of health-emotion-events is realized, allowing users to provide users with more in-depth and personalized feedback and suggestions, which not only helps users understand the root causes of their emotional fluctuations, but also enables them to intervene and adjust in the first place. This precise and real-time emotional companionship support enhances users' self-management ability, establishes deeper emotional connections, and improves users' quality of life and happiness.

[0068] The web-based dialogue generation and emotion-health-dialogue generation are displayed in real time. Through the association model, the real-time association of health-emotion-event is realized, and the implementation forms are as follows: 1) Emotion-health-event association display, text or visualization; 2) Health and emotion association display, text or visualization; 3) Health and dialogue summary association display, text or visualization.

[0069] Those skilled in the art can understand according to the disclosure of the present invention that the web page side can adopt various conversations (human-computer interaction forms), visualizations or texts, as well as their combinations, and can also adopt other human-computer interaction output forms to facilitate people with limited mobility or physical disabilities. The associated display can also be one or more combinations to meet various needs. Similarly, those skilled in the art can understand according to the disclosure of the present invention that the robot, user terminal, cloud processing system, etc. in the present invention are only distinctions of devices, and those skilled in the art can completely replace them with other devices, or use one device to cover another device. For example, the robot and the terminal mobile phone are combined into one. The relevant output information, reports, etc. in the present invention are closely related to other technical features as part of the technical features of the present invention, and they can be one or more combinations to meet different needs under the purpose and technical means of the present invention. These different technical features and their various combined features are both the creative technical features of the present invention and the scope of the solutions protected and covered by the present invention.

[0070] The present invention combines multi-dimensional user data (including voice, behavior, facial expressions, physiological signals, etc.) and fuses them, as well as emotion recognition methods. By wearing a robot neural connector, it can collect the user's physiological signals (such as heart rate, skin conductance response, etc.) in real time and analyze them together with other emotion data (voice, behavior, expression, etc.), so as to provide more accurate emotion recognition. The accurate emotion recognition of the present invention: the fusion of multi-dimensional information provides more accurate emotion recognition, solving the problem that a single technical means cannot accurately recognize complex emotional states.

[0071] At the same time, the present invention combines positive psychology theory and generative large models to provide customized emotion conversations and psychological counseling for users. These conversations are not just simple responses, but are based on the principles of positive psychology, aiming to improve the user's emotional health and psychological well-being, and help users regulate their emotions and relieve stress. The personalized emotion support of the present invention: the combination of generative large models and positive psychology theory enables the robot to provide more personalized and emotional conversations for users, improving the effect of emotional companionship.

[0072] In addition, the robot can initiate conversations proactively based on historical data and emotional health analysis, identify the user's emotional needs in advance, and provide emotional support and suggestions in a timely manner. Through intelligent emotional analysis and prediction, the robot can not only provide psychological counseling proactively when the user is in a low mood, but also provide health analysis suggestions and warnings based on the user's health data to ensure that the user receives timely attention and support in terms of emotion and health. The proactive conversation and health support of the present invention: based on historical health data and emotional analysis, the robot can actively perceive the user's emotional fluctuations and health changes, and provide timely emotional support and health warnings.

[0073] Correspondingly, the present invention also provides a data communication architecture for an emotional companionship system, which includes: real-time physiological data is exchanged between the physiological perception terminal and the emotional interaction entity through a dynamically configurable short-range communication link; the emotional interaction entity performs two-way data synchronization with the distributed cloud brain through the main communication channel and can relay and transmit through the user control interface; multiple emotional interaction entities share environmental and user status information through the group interaction protocol.

[0074] As a further improvement, the dynamic short-range communication link automatically selects Bluetooth, Wi-Fi direct connection, or ultrasonic communication according to data priority.

[0075] As a further improvement, the group interaction protocol supports cross-device emotional state migration in physical spaces and virtual environments.

[0076] As a further improvement, the personality model update generated by the distributed cloud brain includes differential privacy protection parameters.

[0077] Data communication solutions among robots, mobile terminals, the cloud, and wearable devices cover all possible communication solution variants. For example, direct communication, mobile phone relay, hybrid mode, offline, multi-robot collaboration, etc., considering edge computing solutions. Adapt to future communication technologies, such as 6G, terahertz band, or quantum encryption channels. Declare adaptation to cutting-edge technologies such as biocommunication and quantum transmission. The group interaction protocol supports personality trait migration in virtual reality environments. The communication architecture can adapt to future communication technologies, including: Body Area Network (BAN) for wearable devices, quantum encryption channels for cloud instruction transmission, neuromorphic computing for local data processing, etc.

[0078] As the first preferred embodiment (direct connection): The emotional interaction entity is built-in with a multi-mode communication chip and directly connects to the distributed cloud brain through the 5G network.

[0079] As the second preferred embodiment (edge computing): When the network is interrupted, the emotional interaction entity generates a temporary response strategy based on the historical interaction data cached locally.

[0080] As the third preferred embodiment (hybrid networking): Multiple emotional interaction entities form a Mesh network, and one of the nodes serves as a gateway to connect to the cloud.

[0081] The present invention covers all communication solution variants, and any replacement of protocols or adjustment of paths belongs to the scope of the present invention. At the same time, the present invention also lays a foundation for subsequent patents such as robot personality migration and swarm intelligence.

[0082] It should be understood that the scope to be protected by the present invention is not limited to the non-limiting embodiments, and it should be understood that the non-limiting embodiments are merely illustrative examples. The substantial scope of protection required by this application is more reflected in the scope provided by the independent claims and their dependent claims.

Claims

1. An emotional companionship processing system based on multi-dimensional information, characterized by: It includes a cloud processing system, as well as a signal acquisition module for communication connection, an emotion recognition module, a generative large-model emotion dialogue module integrating positive psychology, a personalized and personified robot interaction module, an emotion analysis suggestion and health warning module, a web client display support module, and a historical emotion companionship archive; The cloud processing system has the following data exchange connection and processing method modules and executes them: S1: Collect multi-dimensional biological information of the user and perform fusion processing; S2: Perform sentiment analysis calculation on the fused data to analyze, identify and capture the emotional state of the user; S3: Based on the emotional state, provide the user with personalized emotional dialogue and psychological counseling, emotional analysis suggestions, health analysis suggestions and early warnings through real-time analysis and active dialogue generation.

2. The emotional companionship processing system based on multi-dimensional information according to claim 1, characterized in that: The emotion recognition module, the generative large model emotion dialogue module integrating positive psychology, the personalized and personified robot interaction module and the emotion analysis suggestion and health warning module are arranged in a cloud processing system, the signal acquisition module includes a robot for collecting emotion data and a wearable device for collecting physiological data, the web client display support module is arranged in a user terminal device, the signal acquisition module is connected to the user terminal device for two-way data communication, the user terminal device is connected to the cloud processing system for two-way data communication, the robot is connected to the cloud processing system for one-way data communication and receives command information from the cloud processing system, and the historical emotion companionship file is arranged in at least one of the user terminal device and the cloud processing system.

3. The emotional companionship processing system based on multi-dimensional information according to claim 2, characterized in that: The instructions of the cloud processing system received by the robot are processed and issued by the generative large model emotional dialogue module integrating positive psychology, the personalized and personified robot interaction module and the emotional analysis suggestion and health warning module.

4. The emotional companionship processing system based on multi-dimensional information according to claim 2, characterized in that: It also includes a guardian terminal device, which is connected to the cloud processing system in a one-way data communication manner and receives data information from the cloud processing system. The data information received by the guardian terminal device from the cloud processing system is processed and issued by the emotion analysis suggestion and health warning module.

5. The emotional companionship processing system based on multi-dimensional information according to claim 4, characterized in that: The user terminal is provided with an emotional diary report module and a health report module, wherein the emotional diary report module includes submodules such as emotional trend, conversation summary, reminder and suggestion, and the health report module includes submodules such as health data visualization, health reminder and suggestion; The downstream data output of the dialogue summary is connected to the user web page, and the upstream data input is connected to the historical emotions, dialogues, user portraits and historical health in the historical emotional companionship archive, and a processing module and an information extraction module are sequentially arranged between the dialogue and the dialogue summary; The processing module is also connected to the historical health module, and the processing module includes a text preprocessing module and a biological data association module, and the biological data association module has an internal time series analysis machine modeling and a causal association model; The information extraction module has a natural language understanding (NLP) technology module inside, and the conversation summary has a large model generation module and a prompt engineering technology module inside; The conversation summary is first based on: historical emotions, conversations, and user portraits in the historical emotion companionship archive, and then summarizes real-time and historical conversations through natural language understanding (NLP) technology, large model generation, and Prompt engineering technology to generate keyword sentences.

6. The emotional companionship processing system based on multi-dimensional information according to claim 5, characterized in that: The conversation is first combined with the historical health in the historical emotional companionship archive, and then text preprocessing and physiological data association are performed, and then information extraction is performed; the physiological data association adopts time series analysis machine modeling and causal association model, and the information extraction adopts the natural language understanding NLP technology.

7. An emotional companionship processing system based on multi-dimensional information according to any one of claims 1 to 6, characterized in that: It also includes: S4: Perform data analysis and trend prediction based on the historical emotional companionship profile, actively sense and judge whether to initiate a conversation, provide personalized and personal interaction, and integrate psychological and health analysis suggestions and early warnings to enhance the physical and mental health management of the user; S5: Generate sentiment analysis report and health report, and provide suggestions and warnings to the user; S6: Displaying text and voice dialogues to the user's guardian, so that the guardian can flexibly and intelligently grasp the emotions and basic health status information of the ward anytime and anywhere.

8. An emotional companionship processing method based on multi-dimensional information, characterized in that: It includes S1: Collect multi-dimensional biological information of the user and perform fusion processing; S2: Perform sentiment analysis calculation on the fused data to analyze, identify and capture the emotional state of the user; S3: Based on the emotional state, provide the user with personalized emotional dialogue and psychological counseling, emotional analysis suggestions, health analysis suggestions and early warnings through real-time analysis and active dialogue generation.

9. The method for processing emotional companionship based on multi-dimensional information according to claim 8, characterized in that: In the S3 step, the natural language generation dialogue of the positive psychology processing method and the generative large model is adopted to understand and adapt to the psychological needs of the user in different situations, provide emotional support to the user through emotional regulation and emotional guidance, and help the user to regulate emotions, psychologically comfort, and reconstruct cognition.

10. The method for processing emotional companionship based on multi-dimensional information according to claim 9, characterized in that: In the step S3, by real-time analysis of the user's health, emotional state and psychological needs, a dialogue is actively initiated to provide emotional counseling, psychological advice and health management.

11. The method for processing emotional companionship based on multi-dimensional information according to any one of claims 8 to 10, characterized in that: It also includes establishing and accumulating over a long period of time to form a historical emotional companionship file, wherein the historical emotional companionship file is adopted in the steps S2 and / or S3; The historical emotional companionship archive includes at least: the fusion data, the emotional state, the emotional dialogue and the psychological counseling.

Images