A method and system for emotional counseling based on virtual reality

Abstract

The present invention discloses a method and system for emotional counseling based on virtual reality. The method includes: presenting an initial scene and a digital counselor located in the scene to a user through a virtual reality device; collecting the user's physiological data in real time through a physiological data sensor; guiding the user to share their emotional state through the digital counselor, and obtaining conversation data between the user and the digital counselor; evaluating the user's emotional state based on the conversation data, and evaluating the user's emotional fluctuations based on the physiological data; displaying a customized scene corresponding to the user's current emotional state through the virtual reality device to provide psychological counseling to the user; and replacing the virtual scene when the user's mood improves. The present invention utilizes virtual reality technology, combined with physiological data monitoring and multimodal conversation data analysis, to innovatively provide each user with a personalized and precise emotional counseling solution.

Description

[0001] Priority application

[0002] This application claims priority to the Chinese invention patent application [Application Number: 2024114337705] [A method and system for emotional counseling based on virtual reality] filed on October 14, 2024, which is incorporated by reference in its entirety. Technical Field

[0003] The present invention relates to the field of virtual reality technology, and in particular to a virtual reality-based emotion counseling method and system. Background Art

[0004] Psychological counseling uses professional psychological techniques and communication skills to help users relieve stress, regulate emotions, and resolve psychological issues. Originating from psychological counseling and psychotherapy, with the development of modern society, psychological counseling has gradually become a key means of maintaining mental health. Related techniques include cognitive behavioral therapy (CBT), mindfulness-based cognitive therapy (MBCT), and positive psychology.

[0005] Virtual reality (VR), a cutting-edge technology, shows great potential for making emotional intervention more accessible and personalized. Guided by specialized methodologies, it can precisely implement control procedures, presenting highly controlled immersive scenarios to help users experience and process emotions more realistically. Furthermore, VR system applications allow for repeated access and immediate use, reducing labor and time costs and providing a more convenient and flexible experience.

[0006] For example, Chinese invention patent publication number CN109432567A discloses an emotion regulation system based on an adaptive virtual reality scene, comprising: a virtual reality presentation unit for acquiring initial relaxation scene data and presenting the initial relaxation scene; a stress assessment unit for assessing stress to obtain stress information when the virtual reality presentation unit presents the initial relaxation scene; and the virtual reality presentation unit for acquiring relaxation scene data based on the stress information and presenting the relaxation scene. The initial relaxation scene data and the relaxation scene data are pre-stored locally or in a cloud storage unit. This system has the advantage of providing an emotion regulation system that can match relaxation scenes to the user's current emotional state in real time, eliminating the need for time and space constraints associated with psychotherapy courses or psychological counseling, allowing for emotional relief or release anytime and anywhere.

[0007] For example, the Chinese invention patent with publication number CN113975583A discloses an emotion counseling system based on virtual reality technology, which includes: a virtual reality display device displays at least one emotion regulation scene; in response to a selection operation of a target emotion regulation scene in at least one emotion regulation scene, outputs emotion regulation information corresponding to the target emotion regulation scene; based on user action data collected by the virtual reality display device, interacts with the emotion regulation information; a physiological testing device monitors the user's physiological index in real time and sends the physiological index to the virtual reality display device; the virtual reality display device displays the physiological index; when the physiological index meets a first condition, the emotion regulation information is adjusted to a preset emotion calming information; when the physiological index meets a second condition, the emotion regulation information is adjusted to a preset reality awakening information; the device can be used to counsel emotions and improve the efficiency of emotion counseling.

[0008] For example, Chinese invention patent publication number CN113485557A discloses a method and apparatus for virtual reality emotion control. The method comprises: outputting scene content through virtual reality and obtaining the user's facial expression and voice volume after the scene content is output; determining the user's emotion type based on the facial expression and obtaining a standard facial expression and standard voice volume for the emotion type; comparing the facial expression with the standard facial expression and the voice volume with the standard voice volume to determine the user's current emotion level and determine whether the current emotion level is within the emotion level range corresponding to the scene content; if it is not within the emotion level range corresponding to the scene content, determining an emotion control scheme based on the current emotion level, adjusting the scene content according to the emotion control scheme, and outputting the adjusted scene content through virtual reality. This method can timely control the user's emotions when the user's emotions are significantly affected, preventing further emotional impact, and improving the user experience.

[0009] However, existing psychological counseling services are severely limited by time and space, making them unable to provide timely and effective assistance to those living in remote areas or with limited economic resources. Furthermore, the effectiveness of psychological therapy is highly dependent on the counselor's ability and experience. Most psychological counseling services are conducted through standardized treatment processes, making it difficult to tailor them to the individual's needs. Summary of the Invention

[0010] The purpose of the present invention is to provide a method and system for emotional counseling based on virtual reality, which can partially solve or alleviate the above-mentioned deficiencies in the prior art.

[0011] In order to solve the above-mentioned technical problems, the present invention specifically adopts the following technical solutions:

[0012] An emotional counseling method based on virtual reality, comprising:

[0013] Presenting the initial scene and the digital counselor located in the scene to the user through a virtual reality device;

[0014] Collect the user's physiological data in real time through physiological data sensors;

[0015] The digital counselor guides users to share their emotional state and obtains conversation data between users and the digital counselor;

[0016] evaluating the user's emotional state based on the conversation data, and evaluating the user's emotional fluctuations based on the physiological data;

[0017] Based on the user's current emotional state, customized scenes corresponding to the emotional state are displayed through virtual reality devices to provide psychological counseling for the user;

[0018] When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene;

[0019] When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is changed to the emotional state consolidation scene;

[0020] The second emotion fluctuation threshold is smaller than the first emotion fluctuation threshold.

[0021] As an improvement, in the emotional state customization scenario, natural landscapes are used to map the user's emotional state, and weather changes in the scenario are used to map the user's emotional fluctuations.

[0022] As an improvement, the scene color tone, light intensity and sound effects in the emotional state customized scene are adjusted according to the user's emotional state.

[0023] As an improvement, the change of the natural landscape in the emotional state customization scene to the natural landscape in the emotional state adjustment scene is mapped to the change of the user's emotional state.

[0024] As an improvement, the emotional state consolidation scene is built based on the emotional state adjustment scene. Users interact with the newly added virtual entity in the emotional state consolidation scene and review the counseling process together with the song.

[0025] As an improvement, the lyrics of the song are generated according to the transition process of the user's emotional state and the conversation data, and the melody of the song is generated according to the transition process of the emotional state and the lyrics.

[0026] As an improvement, after obtaining the conversation data, several emotion cards are generated for users to draw. The emotion cards drawn by the user are generated based on the user's emotional state; before being drawn by the user, the content on the card is hidden; after being drawn by the user, the card shows the user the user's current emotional state.

[0027] As an improvement, the card uses colors and text to express the user's current emotional state.

[0028] As an improvement, the physiological data is heart rate data; and the emotional fluctuation is evaluated based on the user's current emotional state and heart rate data.

[0029] The present invention also provides an emotional counseling system based on virtual reality, comprising: a virtual reality output module, a physiological evaluation module, an interactive evaluation module, an emotional state judgment module, and a VR scene adjustment module;

[0030] The virtual reality output module is used to present the initial scene and the digital counselor located in the scene to the user;

[0031] The physiological evaluation module is used to collect the user's physiological data in real time through the physiological data sensor;

[0032] The interactive evaluation module is used to guide the user to share his or her emotional state through the digital counselor and obtain the conversation data between the user and the digital counselor; evaluate the user's emotional state based on the conversation data, and evaluate the user's emotional fluctuations based on the physiological data;

[0033] The VR scene adjustment module is used to display a customized scene corresponding to the user's current emotional state through the virtual reality device to provide psychological counseling for the user;

[0034] When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene;

[0035] When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is changed to the emotional state consolidation scene;

[0036] The second emotion fluctuation threshold is smaller than the first emotion fluctuation threshold.

[0037] The benefits of the present invention lie in that it utilizes virtual reality technology, combined with physiological data monitoring and multimodal conversation data analysis, to innovatively provide each user with a personalized and precise emotional counseling solution. Through a real-time feedback mechanism, the system can dynamically adjust the virtual scene based on immediate physiological and emotional data, more effectively meet the user's emotional needs, and enhance the interactivity and attractiveness of emotional counseling through the combination of technology and psychology. In addition, the system also applies advanced machine learning technology to extract audio and text features, comprehensively analyze and ensure accurate assessment of emotional states, thereby greatly improving the effectiveness of emotional counseling and user experience. The examples of the present invention not only provide an effective means of emotional counseling, but also make the emotional treatment process more dynamic and personalized, greatly enhancing the user's sense of participation and counseling experience.

[0038] This application identifies eight basic emotion categories and innovatively visually displays their connections, thereby visualizing emotions and their dynamics. Based on the theory of the emotion wheel, EmoVR offers eight unique meditation scenes (customized emotional state scenes), each pre-designed to reflect a specific emotional state. These scenes incorporate dynamic weather systems and visual landscapes, serving as visual cues of the user's emotional state and demonstrating the emotional transformations during the treatment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the embodiments or the description of the prior art. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the various elements or parts are not necessarily drawn according to the actual scale. Obviously, the drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without inventive work.

[0040] Figure 1 This is a flow chart of Example 1 of the present invention;

[0041] Figure 2 This is a structural diagram of Example 2 of the present invention;

[0042] Figure 3 This is an example of a VR scenario of the present invention: users share their emotional state;

[0043] Figure 4 This is an example of a VR scenario of the present invention: the user takes an emotion card;

[0044] Figure 5 and Figure 6 They are respectively examples of VR scenes of the present invention: the change from the emotional state customized scene to the scene before and after the emotional state adjustment;

[0045] Figure 7 This is an example of a VR scene of the present invention: an emotional state consolidation scene;

[0046] Figure 8 This is an example of a VR scenario of the present invention: interacting with a virtual entity (butterfly) within an emotional state consolidation scenario. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0048] Herein, suffixes such as "module," "component," or "unit" used to represent elements are only used to facilitate description of the present invention and have no specific meaning. Therefore, "module," "component," or "unit" may be used interchangeably.

[0049] As used herein, terms such as "upper," "lower," "inner," "outer," "front," "back," "one end," and "the other end" indicate positions or locations based on those shown in the accompanying drawings. These terms are intended solely to facilitate and simplify the description of the present invention and are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0050] As used herein, unless otherwise expressly specified or limited, the terms "installed," "provided with," and "connected" should be understood broadly. For example, "connected" may refer to a fixed connection, a detachable connection, or an integral connection; it may refer to a mechanical connection, a direct connection, an indirect connection via an intermediate medium, or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention on a case-by-case basis.

[0051] As used herein, "and / or" includes any and all combinations of one or more of the associated listed items.

[0052] Herein, "plurality" means two or more than two, ie, it includes two, three, four, five, etc.

[0053] Example 1: Figure 1As shown, the present invention provides an emotional counseling method based on virtual reality, comprising:

[0054] S1 presents the initial scene and the digital counselor in the scene to the user through virtual reality equipment.

[0055] Virtual reality (VR) technology is a promising tool for assessing and treating mental health. VR's simulation potential allows it to create realistic, immersive experiences that can elicit emotional responses. Many VR therapeutic applications design realistic, immersive simulations to evoke representative real-life experiences. One of the most popular treatment approaches in this context is virtual reality exposure therapy (VRET). This allows patients to gradually face fearful stimuli or stressful situations in a safe environment, with the presence of a counselor guiding and controlling psychological and physiological responses. These new treatment strategies have demonstrated effectiveness in real-life settings. Furthermore, personalized treatment experiences can better address specific individual needs, thereby enhancing the effectiveness of emotional interventions. Virtual humans are already widely used for self-counseling. This is particularly helpful for those who fear seeking support from others.

[0056] However, the quality and effectiveness of treatment in this setting still vary. Research suggests that using avatars with human characteristics can better build trust and connection, providing a sense of comfort and acceptance within VR environments. Therefore, integrating virtual humans with natural language and nonverbal interactions, as well as facial expressions, into VR holds promise for better supporting mental health.

[0057] While numerous studies have demonstrated that VR (Virtual Reality) therapy is an effective emotional intervention that can promote mental health, there is a lack of research focusing on a reliable and personalized treatment experience to maintain user engagement and effectiveness. This invention addresses the lack of humanistic care in existing emotional intervention methods by using an AI-powered digital counselor to guide users in emotional relief.

[0058] Building user expectations is crucial during treatment. Previous research has demonstrated that users with high outcome expectations are more likely to complete treatment and achieve desired outcomes, whereas users with lower expectations are more likely to drop out. In the context of VR therapy, using specific techniques to effectively build user expectations is crucial. Furthermore, providing positive feedback can also be beneficial. Previous research has demonstrated the potential use of visual or interactive elements, particularly those incorporating gameplay, to enhance feelings of control and willingness to engage in treatment.

[0059] In the initial scene, users are presented with a modern meeting place with private space, bright and soft lighting, and clean and tidy layout, which makes users feel professional without the tension of a hospital.

[0060] Taking into account the user's appropriate input in VR and the user's expectations of the counselor, this invention creates a virtual counselor that is more like a real person in terms of characteristics and behavior to provide a more natural interaction, thereby gaining trust and open communication. Voice input and recognition are implemented in conversations with the AI-driven counselor to support a high-quality consultation experience.

[0061] The image of the digital counselor can be changed according to the user's own situation, such as the image of an anime girl, the image of a professional woman, the image of a mature and stable man, etc.

[0062] A good counselor should demonstrate empathy, understanding, objectivity, patience, and a willingness to listen, qualities that contribute to the effectiveness of therapy. Furthermore, they should possess the ability to adapt their strategies to the client's needs and possess a deep understanding of treatment methods and techniques.

[0063] An effective way to achieve this is to use AI-powered digital humans as counselors. A study using AI-assisted avatars to treat schizophrenia confirmed the effectiveness of digital humans as counselors. As virtual counselors, digital humans excel at complex tasks that require immediate responses.

[0064] S2 collects the user's physiological data in real time through physiological data sensors.

[0065] Numerous studies have shown that physiological and psychological data can also be reflected in avatars and virtual environments, representing individual user performance through visual cues. To further analyze more complex mental health issues, previous work has focused on embedding physiological measurements within VR experiences. By tracking physiological parameters such as heart rate, electrodermal activity, respiratory rate, and eye movements, valuable real-time behavioral, emotional, cognitive, and physiological response feedback can be obtained. With the support of virtual humans and combined with physiological data tracking, VR can facilitate emotional interventions. It provides valuable insights for effective treatment outcomes and personalized user experiences.

[0066] The physiological data described in this embodiment mainly includes heart rate data, and the user's emotional fluctuations can be evaluated through the user's current emotional state and heart rate data.

[0067] Specifically, the heart rate monitoring device is a heart rate sensor, which is used to:

[0068] When the user's arteries contract and relax periodically with the heart, the LED light in the heart rate sensor shines through the skin into the blood, and the photosensor receives the corresponding light reflection signal (PPG);

[0069] After the light reflection signal is converted into an electrical signal, the circuit is filtered and amplified, and an analog voltage signal is output. The heart rate data is then obtained through a heart rate sampling data processing algorithm.

[0070] S3 guides users to share their emotional state through digital counselors and obtains conversation data between users and digital counselors.

[0071] In the usage environment, the user wears a head-mounted display to enter the virtual reality environment. Figure 3 As shown in the figure, users are guided into a private virtual space (initial scene) to meet with a digital counselor. Users are then encouraged to share their stories, thoughts, and feelings with the digital counselor. The virtual counselor can participate in the conversation and ask follow-up questions.

[0072] The conversation data can be text data or voice data. In order to facilitate subsequent processing, the voice data can be converted into text.

[0073] S4 evaluates the user's emotional state based on the conversation data, and evaluates the user's emotional fluctuation based on the physiological data.

[0074] In this implementation, the Convai plug-in integrated in UE5 can be used to perform real-time recognition of conversations in user interviews (based on NLP), convert the user's natural language into text, and analyze the user's semantics to extract keywords and their weights (TF-IDF approach).

[0075] Then, using the sentiment analysis model built into Convai, we generate probability distribution vectors of different sentiment categories based on the text content of the user’s answers in the interview, thereby identifying the current user’s sentiment type.

[0076] In the present invention, emotion types include joy, trust, fear, surprise, sadness, disgust, anger, and hope.

[0077] However, according to previous interviews, directly pointing out users' emotional problems may have unforeseen consequences. Instead, convey emotional states through euphemistic expressions and fun activities.

[0078] In this embodiment, after obtaining the conversation data, the system generates several emotion cards for the user to draw. The emotion card drawn by the user is generated based on the user's emotional state; before being drawn by the user, the content on the card is hidden; after being drawn by the user, the card shows the user the user's current emotional state.

[0079] In other words, the system has actually analyzed the user's emotional type through the above-mentioned emotion analysis model, but based on the premise that it cannot be directly pointed out, the above-mentioned method cleverly allows the user to reveal his or her own emotional type.

[0080] Specifically, if Figure 4 As shown, the system presents several emotion cards (e.g., eight cards corresponding to emotion types) in the initial scene. In the initial state, all eight cards face back toward the user, making the contents of the eight cards invisible.

[0081] The counselor guides the user to randomly draw an emotion card. This card reflects the user's current emotional state. In fact, no matter which emotion card the user draws, the corresponding emotion is their current emotion.

[0082] For users, drawing a card that corresponds to their current mood feels like providence, increasing their trust. Compared to bluntly pointing out the user's emotion type, this approach is more subtle and easier for users to accept.

[0083] In addition, although the texture on the back of each card is the same, the front uses different colors (of course there will be text) to represent different emotional states. For example, the eight main emotions of joy, trust, fear, surprise, sadness, disgust, anger, and hope correspond to the eight main emotional colors: yellow, grass green (Pantone 15-0343, Greenery), forest green (Pantone18-6024, Amazon), aqua blue (Pantone 17-4041, Marina), azure blue (Pantone 19-4057, 1Blue), purple (Pantone 19-3542, Pansy), flaming red (Pantone 18-1664, Fiery Red), and persimmon orange (Pantone 16-1356, Persimmon Orange).

[0084] The eight main emotional colors mentioned above adjust the color depth, that is, the emotional color brightness, according to the user's emotional fluctuations.

[0085] S5 uses virtual reality devices to display customized scenes corresponding to the user's current emotional state to provide psychological counseling to the user.

[0086] According to the user's current emotional state, a corresponding emotional state customized scene is matched in the database. The emotional state customized scene is a new virtual scene and various guidance elements therein, including any one or more combinations of colors, music, dynamic images, etc.

[0087] In this embodiment, in the emotional state customization scene, natural landscapes are used to map the user's emotional state, and weather changes in the scene are used to map the user's emotional fluctuations. In addition, the scene color, light intensity, and sound effects in the emotional state customization scene can be adjusted according to the user's emotional state.

[0088] The above eight customized emotional state scenarios can be mapped one by one, and the specific corresponding relationships are pre-set based on psychological knowledge and experience, for example:

[0089] A custom scene for the joyful emotional state might be "Quiet Dawn."

[0090] The customized scene for the emotional state of trust may be "Sunny Meadow".

[0091] The customized scene for the emotional state of fear may be "rainy night forest".

[0092] The customized scene for the surprised emotional state may be "a bare snowy mountain slope with snow falling".

[0093] The customized scene for a sad emotional state can be "Heavy Rain by the Lake".

[0094] The custom scene for the disgusted emotional state can be a "gray-purple cave".

[0095] The customized scene for the angry emotional state can be "volcanic magma".

[0096] The desired emotional state customized scene may be "Nighttime Dunes".

[0097] It is understandable that, in order to save system overhead, the above emotional state customization scenarios are all preset and can be called through the database.

[0098] For users in different emotional states, different customized emotion state scenarios may have different emotional relief effects. When the virtual output module presents customized emotion state scenarios, the heart rate sensor monitors the user's heart rate data in real time. For example, the high and low heart rate data within a unit of time can reflect the user's emotional fluctuations, that is, the effectiveness of the solution in relieving the user's emotions.

[0099] The system can dynamically present customized scenarios tailored to the user's mood swings. During the user's emotional guidance process, the heart rate assessment module collects the user's heart rate data in real time and calculates their mood swings. Whenever emotions fluctuate, the scene imagery is adjusted based on the real-time emotional fluctuations.

[0100] Virtual counselors provide customized counseling programs to help users improve their emotional state. They can also integrate appropriate rituals into the coaching process, adhering to the fundamental principles of emotional counseling. During the counseling process, relaxation techniques such as deep breathing and meditation are added to the library of techniques for combating negative emotions. Users are encouraged to maintain positivity during practice, and counseling is personalized based on their specific emotional state, thereby helping them gradually face and process their emotions in a safe and controlled environment.

[0101] This step constructs different immersive spaces to convey and modulate corresponding emotions. Dynamic environmental elements, such as lighting, weather, and other landscapes, can reflect emotional fluctuations. For example, when an emotional state improves, such as fear or anger, the heart rate will gradually calm down. Conversely, when sadness improves, the heart rate will increase.

[0102] S6 When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene.

[0103] The first emotion fluctuation threshold represents that the user's emotions have been correspondingly channeled, and the specific adjustment relationship of the threshold is preset based on psychological knowledge and experience.

[0104] In step S5, the user's emotional state is mapped by the natural landscape in the scene. Therefore, in this step, the natural landscape in the scene is customized by the emotional state to the change of the natural landscape in the scene adjusted by the emotional state, and the change of the user's emotional state is mapped, such as Figure 5 and Figure 6 shown.

[0105] For example, from a lonely night to a warm day, the sky and grass become brighter and richer, bringing an emotional lift and a greater sense of confidence and security.

[0106] More specifically, in this implementation, the joyful emotional state adjustment scene can gradually change from "quiet dawn" to "early morning sun, with sunlight filling the earth."

[0107] The emotional state adjustment scenario of trust can be gradually transformed from "sunny meadow" to "sunny flower field".

[0108] The scene for adjusting the emotional state of fear can be "a forest on a rainy night" gradually changing into "a forest on a sunny day".

[0109] The emotional state adjustment scene for surprise can be "a bare snowy mountain slope gradually changing to a snowy mountain slope covered with red leaves."

[0110] The scene for adjusting a sad emotional state can be gradually changed from "lakeside in heavy rain" to "lakeside in sunny day".

[0111] The disgust emotional state adjustment scene can be a "gray-purple cave" that gradually changes to a "sunlit cave."

[0112] The scenario for adjusting the angry emotional state can be "a volcano with lava flowing" gradually transforming into "a crater sprouting in spring."

[0113] The desired emotional state adjustment scene can gradually change from "Night Dunes" to "Dune Meteors".

[0114] S7: When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is changed to an emotional state consolidation scene.

[0115] The second emotion fluctuation threshold indicates that the user's emotions have been further relieved. Therefore, it can be understood that the second emotion fluctuation threshold should be smaller than the first emotion fluctuation threshold, and the specific adjustment relationship of the threshold is also pre-set based on psychological knowledge and experience.

[0116] In this embodiment, the emotional state consolidation scene is built based on the emotional state adjustment scene, and some interactive virtual entities such as butterflies and birds that symbolize beauty are added to the original emotional state adjustment scene. Figure 7 and Figure 8 shown.

[0117] In the emotional state consolidation scene, users can interact with the newly added virtual entity and review the counseling process together with the song. The lyrics of the song are generated based on the user's emotional state transition process and conversation data, and the melody of the song is generated based on the emotional state transition process and the lyrics.

[0118] In this embodiment, Convai is used to generate lyrics. Specifically, the key sentences and emotion probability vectors of the user's answers in the interview are taken as input, and these contents are introduced into the Convai dialogue system with a set context to generate lyrics.

[0119] That is, based on key sentences and emotion probability vectors (or emotional state and / or emotional intensity), corresponding emotional counseling measures are generated. For example, lyrics are generated, and based on the lyrics, corresponding customized songs are generated to conduct emotional counseling and / or map the user's current emotional state.

[0120] The lyrics are generated based on two aspects: 1) conversational language; 2) positive guidance of user emotions. The processing of emotional guidance and conversational language is based on the relevant experience and domain knowledge of psychology and linguistics experts.

[0121] This content is introduced into the Convai dialogue system, which has a pre-defined context. Specifically, through the unofficial SunoAI API, Convai-generated lyrics and their corresponding emotions are read, and instructions containing musical style, lyrics, and vocal requirements are input to generate a customized song audio file. The generated audio is then sent back to the Convai interface to drive the character model to sing the song, thus achieving emotional expression and integration of lyrics, music, and vocals.

[0122] In an example of an emotional state consolidation scenario, the conversation between the counselor and the user may be:

[0123] Teacher Shu: Hello, you seem to be feeling a little heavy. Would you like to share your feelings with me?

[0124] User: Well, when I think about what happened recently, I always feel empty inside.

[0125] Master Shu: I can relate to the feeling of emptiness you're describing, especially after experiencing some change or loss, when it feels like all the sounds and colors of the world have vanished. What causes you to feel this way?

[0126] User: I recently lost a very close friend who provided me with a lot of support both personally and emotionally. The fond memories of her and I keep stirring in my mind like ripples on the water.

[0127] Teacher Shu: These memories are like ripples on a lake, flooding your heart with waves, seemingly endlessly. The loneliness feels especially heavy. Do you feel like these memories and emotions have trapped you on this "lake"?

[0128] User: Yes, that's exactly how it feels. Everything in the past seems like a heavy, dark cloud hanging over me, and I feel like I'm trapped in a place without light.

[0129] Teacher Shu: I understand. It seems like you're struggling in the darkness, trying to find a glimmer of light. We often feel this kind of oppression when we experience loss or face profound emotional change. It's like when night falls, the shadows grow especially large, filled with uncertainty and fear.

[0130] User: Yeah, I always feel like these shadows are getting deeper and deeper, and every step I take is like being dragged into a darker place, with no way to escape.

[0131] Teacher Shu: That feeling is very real, but sometimes, even in the deepest darkness, dawn still comes. You mentioned sitting by the lake, and the memories made you feel painful, but did you notice that perhaps light would slowly spread to the lake, bringing about some changes?

[0132] User: Maybe. I know I can't be trapped in these memories forever, but it's really hard to move on. It feels like every step is so heavy, and I don't know how to move on.

[0133] Master Shu: I understand. Every feeling you have is natural. Pain and memories can haunt you like shadows in the night. But sometimes, when we take one small step forward, light gradually illuminates the world around us. Your friends will also bless you.

[0134] User: Maybe, I need more time to get out of these shadows, but I am willing to try.

[0135] Teacher Shu: You've taken a very important step. Although the night is long, every glimmer of light will slowly illuminate the path ahead. You don't need to rush to find all the answers, but every attempt is an opportunity to move toward light and hope.

[0136] Judging from the above conversation, the user's emotional state is the fifth emotion type: sad.

[0137] Based on the emotion type and the key sentences of the dialogue, the lyrics may be: "A silent melody echoes / Water drops fall on the lake / Thoughts echo in my mind / Sitting alone by the lake / Feeling abandoned / Since the day you left / You have been lingering in my heart / Every moment is filled with heavy memories / Ah, this lonely night / Where have you been / Heart filled with sorrow / Struggling in the whirlpool / Tears in my eyes / Dreams forgotten / We are drifting apart / My thoughts are confused and I have lost myself / But the dawn is gradually breaking / The light shines on the calm lake / I will take back everything that belongs to me / Step by step / I will find the way forward / Leave the shadows behind / In this new beginning / I will find strength and hope / Ah, a brand new day / The pain is gradually dissipating / My heart is filled with courage / We have forged a new path / Tears have long dried up / Dreams are awakened again / We are drifting freely / Feeling the vitality of life / Ah, a brand new day / Hope surges in my heart / We stand side by side / Never apart / Life is like a melody / Find your own rhythm / Under the bright moonlight / Enjoy it all forever."

[0138] The lyrics primarily describe an emotional transition, from loss to renewed hope and strength. The first half depicts sitting alone by a lake, missing someone who has passed away, lost in memories and loneliness, feeling abandoned, and filled with sadness and confusion. Then, with the arrival of dawn, light slowly spreads across the tranquil lake, symbolizing the strength and hope of a fresh start. The second half depicts a positive transformation, emerging from the shadows and embarking on a new journey, filled with courage and confidence in the future, ultimately facing life together with others, following the rhythm of life, and enjoying the beauty of a new life.

[0139] After reading the generated lyrics and corresponding emotions, a customized song (including instructions for music style, lyrics, and vocal requirements) is generated. The customized song can be:

[0140] [Music theater][Female voice]:

[0141] [Verse 1]A silent melody echoes / Water drops fall on the lake / Thoughts echo in my mind / Sitting alone by the lake / Feeling abandoned / Since the day you left / You have been lingering in my heart / Every moment is filled with heavy memories.

[0142] [Chorus] Ah, this lonely night / Where have you gone / My heart is full of sadness / Struggling in the whirlpool / With tears in my eyes / Dreams are forgotten / We are drifting apart / My thoughts are confused and I have lost myself.

[0143] [Bridge] But the dawn is breaking / The light shines on the calm lake / I will take back everything that belongs to me / Step by step / I will find the way forward / Leaving the shadows behind / In this new beginning / I will find strength and hope.

[0144] [Chorus] Ah, a brand new day / The pain fades away / Our hearts are filled with courage / We forge a new path / The tears have long dried up / Dreams awaken again / We drift free / And feel the vitality of life.

[0145] [Outro] Ah, a brand new day / Hope surges in my heart / We stand side by side / Never apart / Life is like a melody / Find your own rhythm / Under the bright moonlight / Enjoy it all forever.

[0146] The audio generated by the customized song is sent back to the convai interface and sung by the character model.

[0147] Example 2: Figure 2 As shown, the present invention also provides an emotional counseling system based on virtual reality, including: a virtual reality output module, a physiological evaluation module, an interactive evaluation module, an emotional state judgment module, and a VR scene adjustment module;

[0148] The virtual reality output module is used to present the initial scene and the digital counselor located in the scene to the user;

[0149] The physiological evaluation module is used to collect the user's physiological data in real time through the physiological data sensor;

[0150] The interactive evaluation module is used to guide the user to share his or her emotional state through the digital counselor and obtain the conversation data between the user and the digital counselor; evaluate the user's emotional state based on the conversation data, and evaluate the user's emotional fluctuations based on the physiological data;

[0151] The VR scene adjustment module is used to display a corresponding customized emotional state scene through a virtual reality device according to the user's current emotional state, and provide psychological counseling to the user; when the user's emotional fluctuation reaches a first emotional fluctuation threshold, the customized emotional state scene is replaced with a corresponding emotional state adjustment scene; when the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is replaced with an emotional state consolidation scene; wherein the second emotional fluctuation threshold is less than the first emotional fluctuation threshold.

[0152] In some embodiments, the interactive evaluation module is also used to generate a number of emotion cards based on the user's conversation data, and display them to the user through the virtual reality output module for the user to draw; wherein, the emotion card drawn by the user is generated based on the user's emotional state; before being drawn by the user, the content on the card is hidden; after being drawn by the user, the card shows the user the user's current emotional state.

[0153] In some embodiments, the VR scene adjustment module is also used to generate lyrics for a corresponding song based on the user's emotional state transition process and conversation data, and the melody of the song is generated based on the emotional state transition process and the lyrics.

[0154] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0155] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a computer terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0156] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A method of emotional counseling based on virtual reality, characterized in that include: Presenting the initial scene and the digital counselor located in the scene to the user through a virtual reality device; Collect the user's physiological data in real time through physiological data sensors; The digital counselor guides users to share their emotional state and obtains conversation data between users and the digital counselor; Based on the conversation data, a number of emotion cards are generated for the user to draw. The emotion card drawn by the user is generated based on the user's emotional state. Before the user draws the card, the content on the card is hidden. After the user draws the card, the card shows the user's current emotional state to increase trust. The card uses color and text to express the user's current emotional state, and uses the brightness of the color to express the user's current emotional fluctuations. evaluating the user's emotional state based on the conversation data, and evaluating the user's emotional fluctuations based on the physiological data; Based on the user's current emotional state, a customized emotional state scene is displayed through a virtual reality device to provide psychological counseling for the user. The customized emotional state scene is a new virtual scene with various counseling elements, including any one or more combinations of color, music, and dynamic images. The customized emotional state scene uses natural landscapes to map the user's emotional state, and uses weather changes in the scene to map the user's emotional fluctuations. When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene; By customizing the natural landscape in the emotional state scene to the emotional state adjustment scene, the natural landscape changes, mapping the changes in the user's emotional state; When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is replaced with an emotional state consolidation scene; the emotional state consolidation scene is built based on the emotional state adjustment scene, and the user interacts with the newly added virtual entity in the emotional state consolidation scene, and reviews the counseling process together with the song; wherein the second emotional fluctuation threshold is less than the first emotional fluctuation threshold.

2. The method for emotional counseling based on virtual reality according to claim 1, characterized in that: Adjust the scene color tone, light intensity and sound effects in the scene according to the user's emotional state.

3. The method of emotional counseling based on virtual reality according to claim 1, characterized in that: The lyrics of the song are generated according to the transition process of the user's emotional state and the conversation data, and the melody of the song is generated according to the transition process of the emotional state and the lyrics.

4. The method of emotional counseling based on virtual reality according to claim 1, characterized in that: The physiological data is heart rate data; the emotional fluctuation is evaluated based on the user's current emotional state and heart rate data.

5. An emotional counseling system based on virtual reality, characterized in that: Includes: virtual reality output module, physiological evaluation module, interactive evaluation module, emotional state judgment module, VR scene adjustment module; The virtual reality output module is used to present the initial scene and the digital counselor located in the scene to the user; The physiological evaluation module is used to collect the user's physiological data in real time through the physiological data sensor; The interactive evaluation module is used to guide users to share their emotional state through the digital counselor and obtain conversation data between the user and the digital counselor; Based on the conversation data, a number of emotion cards are generated for the user to draw. The emotion card drawn by the user is generated based on the user's emotional state. Before the user draws the card, the content on the card is hidden. After the user draws the card, the card shows the user's current emotional state to increase trust. The card uses color and text to express the user's current emotional state, and uses the brightness of the color to express the user's current emotional fluctuations. The user's emotional state is evaluated based on the conversation data, and the user's emotional fluctuations are evaluated based on the physiological data. The VR scene adjustment module is used to display a customized emotional state scene corresponding to the user's current emotional state through the virtual reality device to provide psychological counseling for the user; the customized emotional state scene is a new virtual scene and various counseling elements therein, including any one or more combinations of color, music, and dynamic images; the customized emotional state scene uses natural landscapes to map the user's emotional state, and uses weather changes in the scene to map the user's emotional fluctuations; When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene; By customizing the natural landscape in the emotional state scene to the emotional state adjustment scene, the natural landscape changes, mapping the changes in the user's emotional state; When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is replaced with an emotional state consolidation scene; the emotional state consolidation scene is built based on the emotional state adjustment scene, and the user interacts with the newly added virtual entity in the emotional state consolidation scene, and reviews the counseling process together with the song; wherein the second emotional fluctuation threshold is less than the first emotional fluctuation threshold.

Images