Emotion counseling method and system based on virtual reality

Abstract

The invention discloses an emotion counseling method and system based on virtual reality. The method comprises the steps that an initial scene and a digital counseling teacher located in the scene are presented to a user through virtual reality equipment; collecting physiological data of a user in real time through a physiological data sensor; the method comprises the following steps: guiding a user to share own emotional state through a digital groomer, and acquiring dialogue data between the user and the digital groomer; evaluating the emotional state of the user according to the dialogue data, and evaluating the emotional fluctuation of the user according to the physiological data; according to the current emotional state of the user, a corresponding emotional state customization scene is displayed through virtual reality equipment, and psychological counseling is conducted on the user; and when the emotion of the user is improved, replacing the virtual scene. The virtual reality technology is utilized, physiological data monitoring and multi-mode dialogue data analysis are combined, and a personalized and accurate emotion counseling scheme is creatively provided for each user.

Description

[0001] Priority application This application claims priority to the Chinese invention patent application [Application Number: 2024114337705] [A method and system for emotional counseling based on virtual reality] filed on October 14, 2024, which is incorporated by reference in its entirety. Technical Field

[0002] The present invention relates to the field of virtual reality technology, and in particular to an emotion guidance method and system based on virtual reality. Background Art

[0003] Psychological counseling is a method to help users relieve psychological pressure, regulate emotions, and solve psychological problems through professional psychological techniques and communication skills. Originated from psychological counseling and psychotherapy, with the development of modern society, psychological counseling has gradually become an important means of maintaining mental health. Its related technologies include: cognitive behavioral therapy (CBT), mindfulness-based therapy (MBCT), positive psychology, etc.

[0004] As a cutting-edge technology, virtual reality (VR) technology has shown great potential in making emotional intervention more popular and personalized. Under the guidance of professional methods, it can accurately implement regulatory procedures to present highly controlled immersive scenes to help users experience and process emotions more realistically. In addition, VR system applications allow repeated access and instant use, reducing labor and time costs and bringing a more convenient and flexible experience.

[0005] For example, the Chinese invention patent with publication number CN109432567A discloses an emotion regulation system based on an adaptive virtual reality scene, including: a virtual reality presentation unit for obtaining initial relaxation scene data and presenting the initial relaxation scene; a stress assessment unit for assessing stress to obtain stress information when the virtual reality presentation unit presents the initial relaxation scene; the virtual reality presentation unit is also used to obtain relaxation scene data according to the stress information and present the relaxation scene; the data of the initial relaxation scene and the relaxation scene are pre-stored locally or in a cloud storage unit. Its advantage is that it provides an emotion regulation system that can match the relaxation scene in real time according to the user's current emotional state, without the need for time and space for psychotherapy courses or psychological counseling, and can relieve or release emotions anytime and anywhere.

[0006] For another example, a Chinese invention patent with publication number CN113975583A discloses an emotion counseling system based on virtual reality technology, the system comprising: a virtual reality display device displays at least one emotion regulation scene; in response to a selection operation of a target emotion regulation scene in at least one emotion regulation scene, outputs emotion regulation information corresponding to the target emotion regulation scene; based on user action data collected by the virtual reality display device, interacts with the emotion regulation information; a physiological testing device monitors the user's physiological index in real time and sends the physiological index to the virtual reality display device; the virtual reality display device displays the physiological index; when the physiological index meets the first condition, the emotion regulation information is adjusted to a preset emotion calming information; when the physiological index meets the second condition, the emotion regulation information is adjusted to a preset reality awakening information; the device can be used to counsel emotions and improve the efficiency of counseling emotions.

[0007] For another example, the Chinese invention patent with publication number CN113485557A discloses a method and device for virtual reality emotion control, the method comprising: outputting scene content through virtual reality, and obtaining the user's facial expression and voice volume after outputting the scene content; determining the user's emotion type according to the facial expression, and obtaining the standard facial expression and standard voice volume of the emotion type; comparing the facial expression with the standard facial expression, and the voice volume with the standard voice volume, determining the user's current emotion level, and judging whether the current emotion level is within the emotion level range corresponding to the scene content; when it is not within the emotion level range corresponding to the scene content, determining an emotion control scheme according to the current emotion level, adjusting the scene content through the emotion control scheme, and outputting the adjusted scene content through virtual reality. The method can timely control the user's emotions when the user's emotions are greatly affected, prevent the user's emotions from being further affected, and improve the user experience.

[0008] However, existing psychological counseling services are greatly limited by time and space, and cannot provide timely and effective help to people living in remote areas and those with weak economic conditions. At the same time, the effectiveness of psychological treatment is highly dependent on the ability and experience of counselors. Most psychological counseling is mainly carried out through standardized treatment processes, which is difficult to make personalized adjustments for users and cannot fully meet the unique needs of each person. Summary of the invention

[0009] The purpose of the present invention is to provide a method and system for emotional counseling based on virtual reality, which can partially solve or alleviate the above-mentioned deficiencies in the prior art.

[0010] In order to solve the above-mentioned technical problems, the present invention specifically adopts the following technical solutions: An emotional counseling method based on virtual reality, comprising: Presenting the initial scene and the digital counselor in the scene to the user through a virtual reality device; Collect the user's physiological data in real time through physiological data sensors; Guide users to share their emotional state through digital counselors, and obtain conversation data between users and digital counselors; evaluating the user's emotional state based on the conversation data, and evaluating the user's emotional fluctuations based on the physiological data; According to the user's current emotional state, the corresponding emotional state customized scene is displayed through the virtual reality device to provide psychological counseling for the user; When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene; When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is changed to an emotional state consolidation scene; Among them, the second emotion fluctuation threshold is smaller than the first emotion fluctuation threshold.

[0011] As an improvement, in the emotional state customization scene, natural landscapes are used to map the user's emotional state, and weather changes in the scene are used to map the user's emotional fluctuations.

[0012] As an improvement, the scene color tone, light intensity and sound effects in the emotional state customized scene are adjusted according to the user's emotional state.

[0013] As an improvement, the change of the user's emotional state is mapped by changing the natural landscape in the emotional state customization scene to the natural landscape in the emotional state adjustment scene.

[0014] As an improvement, the emotional state consolidation scene is built based on the emotional state adjustment scene. The user interacts with the newly added virtual entity in the emotional state consolidation scene and reviews the counseling process together with the song.

[0015] As an improvement, the lyrics of the song are generated according to the transition process of the user's emotional state and the conversation data, and the melody of the song is generated according to the transition process of the emotional state and the lyrics.

[0016] As an improvement, after obtaining the conversation data, several emotion cards are generated for users to draw. The emotion cards drawn by users are generated based on the user's emotional state. Before being drawn by the user, the content on the card is hidden; after being drawn by the user, the card shows the user the user's current emotional state.

[0017] As an improvement, the card uses colors and text to express the user's current emotional state.

[0018] As an improvement, the physiological data is heart rate data; and the emotional fluctuation is evaluated based on the user's current emotional state and heart rate data.

[0019] The present invention also provides an emotion counseling system based on virtual reality, comprising: a virtual reality output module, a physiological evaluation module, an interactive evaluation module, an emotion state judgment module, and a VR scene adjustment module; The virtual reality output module is used to present the initial scene and the digital counselor located in the scene to the user; The physiological evaluation module is used to collect the user's physiological data in real time through the physiological data sensor; The interactive evaluation module is used to guide the user to share his or her emotional state through the digital counselor and obtain the conversation data between the user and the digital counselor; evaluate the user's emotional state based on the conversation data, and evaluate the user's emotional fluctuations based on the physiological data; The VR scene adjustment module is used to display a customized scene corresponding to the user's current emotional state through a virtual reality device to provide psychological counseling for the user; When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene; When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is changed to an emotional state consolidation scene; Among them, the second emotion fluctuation threshold is smaller than the first emotion fluctuation threshold.

[0020] The benefit of the present invention lies in that the invention utilizes virtual reality technology, combined with physiological data monitoring and multimodal conversation data analysis, to innovatively provide each user with a personalized and accurate emotional counseling solution. Through the real-time feedback mechanism, the system can dynamically adjust the virtual scene according to the instant physiological and emotional data, more effectively meet the emotional needs of users, and enhance the interactivity and attractiveness of emotional counseling through the combination of technology and psychology. In addition, the system also uses advanced machine learning technology to extract audio and text features, comprehensively analyze and ensure accurate assessment of emotional states, thereby greatly improving the effectiveness of emotional counseling and user experience. The example of the present invention not only provides an effective means of emotional counseling, but also makes the emotional treatment process more dynamic and personalized, greatly enhancing the user's sense of participation and counseling experience.

[0021] This application identifies eight basic emotion categories and innovatively displays the correlation between them visually, that is, visualizes emotions and their changes. Based on the emotion wheel theory, situational counseling (EmoVR) provides eight unique meditation scenes (i.e., emotional state customized scenes), each of which is pre-designed to reflect a specific emotional state. These scenes contain dynamic weather systems and visual landscapes as visual cues of the user's emotional state, showing the transformation of emotions during the treatment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale. Obviously, the drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without paying creative labor.

[0023] Figure 1 This is a flow chart of Embodiment 1 of the present invention; Figure 2 This is a structural diagram of Embodiment 2 of the present invention; Figure 3 This is an example of a VR scenario of the present invention: the user shares his or her emotional state; Figure 4 This is an example of a VR scenario of the present invention: a user takes an emotion card; Figure 5 and Figure 6 They are respectively examples of VR scenes of the present invention: changes from emotional state customized scenes to scenes before and after emotional state adjustment; Figure 7 This is an example of a VR scene of the present invention: an emotional state consolidation scene; Figure 8 This is an example of a VR scenario of the present invention: interacting with a virtual entity (butterfly) within an emotional state consolidation scenario. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] Herein, suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings by themselves. Therefore, "module", "component" or "unit" can be used mixedly.

[0026] In this document, the terms "upper", "lower", "inner", "outer", "front", "back", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0027] In this document, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] Herein "and / or" includes any and all combinations of one or more of the associated listed items.

[0029] Herein, "plurality" means two or more than two, ie, it includes two, three, four, five, etc.

[0030] Example 1: Figure 1 As shown, the present invention provides an emotion guidance method based on virtual reality, comprising: S1 presents the initial scene and the digital counselor in the scene to the user through a virtual reality device.

[0031] Virtual reality (VR) technology is a promising tool for assessing and treating mental health. VR’s simulation potential makes it an experiential image comparable to reality that can elicit emotional responses. Many VR therapeutic applications have designed realistic immersive simulations to evoke representative real-life experiences. One of the most popular treatments in this context is virtual reality exposure therapy (VRET). Patients are allowed to gradually face fearful stimuli or stressful situations in a safe environment, and the presence of a counselor can guide and control psychological and physiological responses. Trying these new treatment strategies in real life has shown its effectiveness. On the other hand, personalized treatment experiences are more able to meet specific individual needs, thereby increasing the effectiveness of emotional interventions. Currently, virtual people are already widely used for self-counseling. This is especially helpful for those who are afraid to seek support from others.

[0032] However, there are still differences in the quality and effectiveness of treatment in this setting. Research has shown that using avatars with human characteristics can better build trust and connection, providing a sense of comfort and acceptance in VR environments. Therefore, integrating virtual humans with natural language and non-verbal interactions and facial expressions into VR is expected to better support mental health.

[0033] Although many studies have proven that VR (Virtual Reality) therapy is an effective emotional intervention method that can promote mental health, there is a lack of research focusing on reliable and personalized treatment experiences to maintain user engagement and effectiveness. The present invention uses an artificial intelligence-driven digital counselor to guide users to relieve their emotions, thereby solving the problem that existing emotional intervention methods lack humanistic care for users.

[0034] The construction of user expectations is critical in the treatment process. Previous research has demonstrated that users with high outcome expectations are more likely to complete treatment and achieve desired results, while users with lower beliefs are more likely to drop out. In the context of VR therapy, it is crucial to use specific techniques to effectively build user expectations. In addition, providing positive feedback can also be beneficial. Previous research has demonstrated the potential use of visual or interactive elements, especially those combined with gameplay, to enhance the sense of control and willingness to undergo treatment.

[0035] In the initial scene, a modern-style reception area is presented to users, which has a private space, bright and soft lighting environment, and clean and tidy layout, allowing users to feel its professionalism without the tension of a hospital.

[0036] Taking into account the user's appropriate input in VR and the user's expectations of the counselor, the present invention creates a virtual counselor that is more like a real person in characteristics and behaviors in order to provide a more natural interaction, so as to gain trust and open communication. Voice input and recognition are implemented in the conversation with the AI-driven counselor to support a high-quality consultation experience.

[0037] The image of the digital counselor can be changed according to the user's own situation, such as the image of an anime girl, the image of a professional woman, the image of a mature and stable man, etc.

[0038] A good counselor should demonstrate qualities such as empathy, understanding, objectivity, patience, and willingness to listen, which contribute to the effectiveness of treatment. In addition, it should also have the ability to adjust strategies according to the needs of the client and have a deep understanding of treatment methods and techniques.

[0039] An effective way to achieve this requirement is to let AI-driven digital humans act as counselors. A study using AI-assisted avatars to treat schizophrenia confirmed the effectiveness of digital humans as counselors. As virtual counselors, digital humans performed well in complex tasks that required instant reactions.

[0040] S2 collects the user's physiological data in real time through physiological data sensors.

[0041] Numerous studies have shown that physiological and psychological data can also be reflected in avatars and virtual environments, representing the performance of individual users through visual cues. To further analyze more complex mental health issues, previous work has focused on embedding physiological measurements in VR experiences. By tracking physiological parameters such as heart rate, electrodermal activity, respiratory rate, and eye movements, valuable real-time behavioral, emotional, cognitive, and physiological response feedback can be obtained. With the support of virtual humans and combined with physiological data tracking, VR facilitates emotional intervention. It provides valuable insights for effective treatment outcomes and personalized user experience.

[0042] The physiological data described in this embodiment mainly includes heart rate data, and the user's emotional fluctuations can be evaluated through the user's current emotional state and heart rate data.

[0043] Specifically, the heart rate monitoring device is a heart rate sensor, and the heart rate sensor is used for: When the user's arteries contract and relax periodically with the heart, the LED light in the heart rate sensor shines through the skin into the blood, and the photosensitive element receives the corresponding light reflection signal (PPG); After the light reflection signal is converted into an electrical signal, the circuit is filtered and amplified, and an analog voltage signal is output. Then, the heart rate data is obtained through the heart rate sampling data processing algorithm.

[0044] S3 guides users to share their emotional state through digital counselors and obtains conversation data between users and digital counselors.

[0045] In the usage environment, the user wears a head mounted display to enter a virtual reality environment. Figure 3 As shown in Figure 2, users are guided into a private virtual space (initial scene) to meet with a digital counselor. Users are then encouraged to share their stories, thoughts, and feelings with the digital counselor. The virtual counselor can participate in the conversation and ask follow-up questions.

[0046] The conversation data can be text data or voice data. In order to facilitate subsequent processing, the voice data can be converted into text.

[0047] S4 evaluates the user's emotional state based on the conversation data, and evaluates the user's emotional fluctuations based on the physiological data.

[0048] In this implementation, the Convai plug-in integrated in UE5 can be used to perform real-time recognition of conversations in user interviews (based on NLP), convert the user's natural language into text, and analyze the user's semantics to extract keywords and their weights (TF-IDF approach).

[0049] Then, using the built-in sentiment analysis model in Convai, we generate probability distribution vectors of different sentiment categories based on the text content of the user’s answers in the interview, thereby identifying the current user’s sentiment type.

[0050] In the present invention, the emotion types include joy, trust, fear, surprise, sadness, disgust, anger, and hope.

[0051] However, according to previous interviews, directly pointing out users' emotional problems may have unforeseen consequences. Use euphemistic expressions and fun activities to convey emotional states.

[0052] In this embodiment, after obtaining the conversation data, the system generates several emotion cards for users to draw. The emotion cards drawn by the users are generated based on the user's emotional state. Before being drawn by the user, the content on the card is hidden. After being drawn by the user, the card shows the user the user's current emotional state.

[0053] In other words, the system has actually analyzed the user's emotion type through the above-mentioned emotion analysis model, but based on the premise that it cannot be directly pointed out, the above-mentioned method cleverly allows the user to reveal his or her own emotion type.

[0054] Specifically, Figure 4As shown, the system presents several emotion cards (e.g., eight cards corresponding to emotion types) in the initial scene. In the initial state, all eight cards face back toward the user, so that the contents of the eight cards are invisible.

[0055] The instructor guides the user to randomly draw an emotion card. This card reflects the user's current emotional state. In fact, no matter which emotion card the user draws, the corresponding emotion is his or her current emotion.

[0056] From the user's perspective, if the corresponding emotion card is exactly what they are feeling, they will feel it is God's will, which increases their sense of trust. Compared with bluntly pointing out the user's emotion type, the above method is more tactful and easier for users to accept.

[0057] In addition, although the texture on the back of each card is the same, the front side uses different colors (of course there will be text) to represent different emotional states. For example, the eight main emotions of joy, trust, fear, surprise, sadness, disgust, anger, and hope correspond to eight main emotional colors: yellow, greenery (Pantone 15-0343), forest green (Pantone 18-6024, Amazon), aqua blue (Pantone 17-4041, Marina), azure (Pantone 19-4057, 1Blue), purple (Pantone 19-3542, Pansy), Fiery Red (Pantone 18-1664), and Persimmon Orange (Pantone 16-1356).

[0058] The eight main emotional colors mentioned above adjust the color depth, that is, the emotional color brightness, according to the user's emotional fluctuations.

[0059] S5 displays customized scenes corresponding to the user's current emotional state through virtual reality devices to provide psychological counseling for the user.

[0060] According to the current emotional state of the user, a corresponding emotional state customized scene is matched in the database. The emotional state customized scene is a new virtual scene and various guidance elements therein, including any one or more combinations of colors, music, dynamic pictures, etc.

[0061] In the emotional state customization scene of this embodiment, natural landscapes are used to map the user's emotional state, and weather changes in the scene are used to map the user's emotional fluctuations. In addition, the scene color, light intensity and sound effects in the emotional state customization scene can be adjusted according to the user's emotional state.

[0062] The above 8 emotional state customized scenes can be matched one by one, and the specific corresponding relationships are preset based on psychological knowledge and experience, for example: A custom scene for the emotional state of joy could be "Quiet Dawn".

[0063] The customized scene for the emotional state of trust may be "Sunny Meadow".

[0064] The customized scene for the emotional state of fear may be "Forest on a Rainy Night".

[0065] The customized scene for the surprised emotional state may be "snowing bare snowy mountain slopes".

[0066] The customized scene for a sad emotional state can be "Heavy Rain by the Lake".

[0067] The custom scene for the disgust emotional state can be "gray-purple cave".

[0068] The customized scene for the angry emotional state can be "volcanic magma".

[0069] The desired emotional state customized scene may be "Nighttime Dunes".

[0070] It is understandable that in order to save system overhead, the above emotional state customization scenarios are all preset and can be called through the database.

[0071] For users in different emotional states, different emotional state customized scenes may release different emotional guidance effects. When the virtual output module presents the emotional state customized scene, the heart rate sensor monitors the user's heart rate data in real time. For example, the high and low heart rate data in a unit of time can be used to obtain emotional fluctuations, that is, the emotional guidance effect of the solution on the user.

[0072] The system can dynamically present customized scenes corresponding to emotional states of emotional fluctuations. During the process of emotional guidance, the heart rate assessment module collects the user's heart rate data in real time and calculates the emotional fluctuations. Whenever the emotions fluctuate, the scene image is adjusted and presented according to the real-time emotional fluctuations.

[0073] Virtual counselors provide customized counseling programs to help users improve their emotional state. Reasonable rituals can also be integrated into the coaching program, following the basic principles of emotional counseling. During the counseling process, relaxation techniques such as deep breathing and meditation are added to the database for combating negative emotions. Users are encouraged to stay positive in practice and receive individual counseling based on their specific emotional state, thereby improving the ability to gradually face and deal with emotions in a safe and controllable environment.

[0074] This step constructs different immersive spaces to convey and regulate corresponding emotions. Some dynamic environmental elements, such as lighting, weather, and other landscapes, can reflect emotional fluctuations. Assume that when the emotional state improves, for example, when the emotions of fear and anger are improved, the heart rate will slowly calm down. When the sad emotions are improved, the heart rate will increase.

[0075] S6 When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene.

[0076] The first emotion fluctuation threshold represents that the user's emotions have been correspondingly channeled, and the specific adjustment relationship of the threshold is preset based on psychological knowledge and experience.

[0077] In step S5, the user's emotional state is mapped through the natural landscape in the scene. Therefore, in this step, the change of the natural landscape in the scene customized by the emotional state is also mapped to the change of the user's emotional state, such as Figure 5 and Figure 6 shown.

[0078] For example, from a lonely night to a warm day, the sky and grass become brighter and richer, bringing an emotional uplift and a greater sense of confidence and security.

[0079] More specifically, in this implementation, the joyful emotional state adjustment scene can gradually change from "quiet dawn" to "early morning sun, with sunlight shining all over the earth."

[0080] The emotional state adjustment scenario of trust can be that "sunny meadow" gradually changes into "sunny flower field".

[0081] The fear emotional state adjustment scene can be "rainy night forest" gradually changing into "sunny day forest".

[0082] The emotional state adjustment scene for surprise can be "a bare snowy mountain slope" gradually transforming into a "snowy mountain slope covered with red leaves."

[0083] The sad emotional state adjustment scene can be "rainy lakeside" gradually changed to "sunny lakeside".

[0084] The disgust emotional state adjustment scene can be a "gray-purple cave" that gradually changes into a "sunlit cave".

[0085] The scenario for adjusting the emotional state of anger can be "a volcano with lava flowing" gradually transforming into "a crater sprouting in spring."

[0086] The desired emotional state adjustment scene can be "Night Dunes" gradually changing to "Dune Meteors".

[0087] S7: When the user's emotion fluctuation reaches a second emotion fluctuation threshold, the emotion state adjustment scene is changed to an emotion state consolidation scene.

[0088] The second emotion fluctuation threshold indicates that the user's emotions are further relieved, so it can be understood that the second emotion fluctuation threshold should be smaller than the first emotion fluctuation threshold, and the specific adjustment relationship of the threshold is also preset based on psychological knowledge and experience.

[0089] In this embodiment, the emotional state consolidation scene is built based on the emotional state adjustment scene, and some interactive virtual entities such as butterflies and birds that symbolize beauty are added to the original emotional state adjustment scene. Figure 7 and Figure 8 shown.

[0090] In the emotional state consolidation scene, the user can interact with the newly added virtual entity and review the counseling process together with the song. The lyrics of the song are generated according to the emotional state transition process and dialogue data of the user, and the melody of the song is generated according to the emotional state transition process and the lyrics.

[0091] In this embodiment, Convai is used to generate lyrics. Specifically, the key sentences and emotion probability vectors of the user's answers in the interview are taken as input, and these contents are introduced into the Convai dialogue system with set context to generate lyrics.

[0092] That is, corresponding emotional guidance measures are generated based on key sentences and emotional probability vectors (or emotional states and / or emotional strengths), for example, lyrics, and corresponding customized songs are generated based on the lyrics, so as to guide the user's emotions and / or map the current emotional state.

[0093] The lyrics are generated based on the following two aspects: 1) conversation sentences; 2) positive guidance of user emotions. The processing of emotion guidance and conversation sentences is based on the relevant experience and domain knowledge of psychology and linguistics experts.

[0094] The content is introduced into the Convai dialogue system with a set context. Specifically, the lyrics text and corresponding emotions generated by Convai are read through the unofficial SunoAI API, and instructions containing music style, lyrics and vocal requirements are input to generate customized song audio files. The generated audio will be sent back to the Convai interface to drive the character model to sing the song, thereby realizing the emotional expression and integration of lyrics, music and vocals.

[0095] In an example of an emotional state consolidation scenario, the conversation between the counselor and the user may be: Teacher Shu: Hello, you seem to be feeling a little heavy. Would you like to share your feelings with me? User: Well, when I think about what happened recently, I always feel empty inside.

[0096] Master Shu: I can relate to the feeling of emptiness you're describing, especially after experiencing some change or loss, when it feels like all the sounds and colors of the world have disappeared. What is it that makes you feel this way? User: I recently lost a very close friend who had given me a lot of support both in life and in my emotions. The beautiful memories of her and I kept stirring up waves in my mind.

[0097] Teacher Shu: These memories are like ripples on the lake, surging into my heart one after another, as if there is no end. That kind of loneliness makes people feel particularly heavy. Do you feel that these memories and emotions have trapped you on this "lake"? User: Yes, that's exactly what it felt like. Everything in the past seemed like a heavy dark cloud hanging over me, and I felt like I was trapped in a place without light.

[0098] Teacher Shu: I understand. It seems like you are struggling in the darkness, trying to find a little light. We often feel this kind of oppression when we lose something or face a huge emotional change. Just like when night falls, the shadows will become particularly large, full of unknowns and fears.

[0099] User: Yeah, I always feel like these shadows are getting deeper and deeper, and every step I take is like being dragged into a darker place, with no escape.

[0100] Teacher Shu: That feeling is very real, but sometimes even in the deepest darkness, dawn will come. You mentioned that you were sitting by the lake, and the memories made you feel painful, but did you notice that maybe the light would slowly spread to the lake and bring about some changes? User: Maybe. I know I can't be stuck in these memories forever, but it's really hard to get out. It seems like every step is so heavy, and I don't know how to move on.

[0101] Teacher Shu: I understand that every feeling you have is natural. Pain and memories trap you like shadows in the middle of the night, but sometimes, when we slowly start to take a step, the light will illuminate the world around us little by little. Your friends will also bless you.

[0102] User: Maybe, I need more time to get out of this shadow, but I am willing to try.

[0103] Teacher Shu: You have taken a very important step. Although the night is long, every little light will slowly illuminate the road ahead. You don't need to rush to find all the answers, but every attempt is an opportunity to move yourself towards light and hope.

[0104] Judging from the above dialogue, the user's emotional state is the fifth emotion type: sad.

[0105] Based on the emotion type and the key sentences of the dialogue, the lyrics may be: "A silent melody echoes / Water drops fall on the lake / Thoughts echo in my mind / Sitting alone by the lake / Feeling abandoned / You have been lingering in my heart since the day you left / Every moment is filled with heavy memories / Ah, this lonely night / Where have you been / Heart is full of sadness / Struggling in the whirlpool / With tears in my eyes / Dreams forgotten / We are drifting away / Thoughts are confused and I have lost myself / But the dawn is gradually breaking / The light shines on the calm lake / I will take back everything that belongs to me / Step by step / I will find the way forward / Leave the shadows behind / In this new beginning / I will find strength and hope / Ah, a brand new day / The pain gradually dissipates / My heart is full of courage / We have opened up a new path / Tears have long dried up / Dreams awaken again / We drift freely / Feel the vitality of life / Ah, a brand new day / Hope surges in my heart / We stand side by side / Never separate / Life is like a melody / Find your own rhythm / Under the bright moonlight / Enjoy it all forever." The lyrics describe an emotional transition from loss to finding hope and strength again. The first half of the lyrics depicts sitting alone by the lake, missing someone who has left, falling into memories and loneliness, feeling abandoned, and filled with sadness and confusion. With the arrival of dawn, light slowly spreads to the calm lake, symbolizing the emergence of strength and hope to start anew. The second half of the lyrics shows a positive transformation, walking out of the shadows and heading for a new journey, with courage and confidence in the future in the heart, and finally facing life together with others, following the melody of life, and enjoying the beauty of a new life.

[0106] After reading the generated lyrics and corresponding emotions, a customized song (including instructions for music style, lyrics, and vocal requirements) is generated. The customized song can be: [Music theater][Female voice]: [Verse 1]A silent melody echoes / Water drops fall on the lake / Thoughts echo in my mind / Sitting alone by the lake / Feeling abandoned / Since the day you left / You have been lingering in my heart / Every moment is filled with heavy memories.

[0107] [Chorus] Ah, this lonely night / Where have you been / My heart is full of sadness / Struggling in the whirlpool / With tears in my eyes / Dreams are forgotten / We are drifting away / My thoughts are confused and I have lost myself.

[0108] [Bridge] But the dawn is breaking / The light shines on the calm lake / I will take back everything that belongs to me / Step by step / I will find the way forward / Leave the shadows behind / In this new beginning / I will find strength and hope.

[0109] [Chorus] Ah, a brand new day / The pain fades away / The heart is filled with courage / We forge a new path / The tears have long dried up / The dream awakens again / We drift free / And feel the vitality of life.

[0110] [Outro]Ah, a brand new day / Hope surges in my heart / We stand side by side / Never apart / Life is a melody / Find your own rhythm / Under the bright moonlight / Enjoy it all forever.

[0111] The audio generated by the customized song is transmitted back to the interface of convai and sung by the character model.

[0112] Example 2: Figure 2 As shown, the present invention also provides an emotion counseling system based on virtual reality, including: a virtual reality output module, a physiological evaluation module, an interactive evaluation module, an emotional state judgment module, and a VR scene adjustment module; The virtual reality output module is used to present the initial scene and the digital counselor located in the scene to the user; The physiological evaluation module is used to collect the user's physiological data in real time through the physiological data sensor; The interactive evaluation module is used to guide the user to share his or her emotional state through the digital counselor and obtain the conversation data between the user and the digital counselor; evaluate the user's emotional state based on the conversation data, and evaluate the user's emotional fluctuations based on the physiological data; The VR scene adjustment module is used to display the corresponding customized scene of the emotional state through the virtual reality device according to the user's current emotional state, so as to provide psychological counseling for the user; when the user's emotional fluctuation reaches a first emotional fluctuation threshold, the customized scene of the emotional state is replaced with the corresponding emotional state adjustment scene; when the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is replaced with an emotional state consolidation scene; wherein the second emotional fluctuation threshold is less than the first emotional fluctuation threshold.

[0113] In some embodiments, the interactive evaluation module is also used to generate a number of emotion cards based on the user's conversation data, and display them to the user through the virtual reality output module for the user to draw; wherein the emotion card drawn by the user is generated based on the user's emotional state; before being drawn by the user, the content on the card is hidden; after being drawn by the user, the card shows the user the user's current emotional state.

[0114] In some embodiments, the VR scene adjustment module is also used to generate lyrics of a corresponding song according to the user's emotional state transition process and dialogue data, and the melody of the song is generated according to the emotional state transition process and the lyrics.

[0115] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a computer terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0117] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for emotional counseling based on virtual reality, characterized in that include: Presenting the initial scene and the digital counselor in the scene to the user through a virtual reality device; Collect the user's physiological data in real time through physiological data sensors; Guide users to share their emotional state through digital counselors, and obtain conversation data between users and digital counselors; evaluating the user's emotional state based on the conversation data, and evaluating the user's emotional fluctuations based on the physiological data; According to the user's current emotional state, the corresponding emotional state customized scene is displayed through the virtual reality device to provide psychological counseling for the user; When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene; When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is changed to an emotional state consolidation scene; Among them, the second emotion fluctuation threshold is smaller than the first emotion fluctuation threshold.

2. The method for emotional counseling based on virtual reality according to claim 1, characterized in that: In the emotional state customization scene, natural landscapes are used to map the user's emotional state, and weather changes in the scene are used to map the user's emotional fluctuations.

3. The method for emotional counseling based on virtual reality according to claim 2, characterized in that: Adjust the scene color tone, light intensity and sound effects in the emotional state customized scene according to the user's emotional state.

4. The method for emotional counseling based on virtual reality according to claim 1, characterized in that: The changes in the natural landscape in the emotional state customization scene to the natural landscape in the emotional state adjustment scene are mapped to the changes in the user's emotional state.

5. The method for emotional counseling based on virtual reality according to claim 1, characterized in that: The emotional state consolidation scene is built based on the emotional state adjustment scene. The user interacts with the newly added virtual entity in the emotional state consolidation scene and reviews the counseling process together with the song.

6. The method for emotional counseling based on virtual reality according to claim 5, characterized in that: The lyrics of the song are generated according to the transition process of the user's emotional state and the conversation data, and the melody of the song is generated according to the transition process of the emotional state and the lyrics.

7. The method for emotional counseling based on virtual reality according to claim 1, characterized in that: After obtaining the conversation data, several emotion cards are generated for users to draw. The emotion cards drawn by users are generated based on the user's emotional state. Before being drawn by the user, the content on the card is hidden. After being drawn by the user, the card shows the user the user's current emotional state.

8. The method for emotional counseling based on virtual reality according to claim 7, characterized in that: The card uses colors and text to express the user's current emotional state, and uses the brightness of the color to express the user's current emotional fluctuations.

9. The method for emotional counseling based on virtual reality according to claim 1, characterized in that: The physiological data is heart rate data; the emotional fluctuation is evaluated based on the user's current emotional state and heart rate data.

10. An emotional counseling system based on virtual reality, characterized in that: Including: virtual reality output module, physiological evaluation module, interactive evaluation module, emotional state judgment module, VR scene adjustment module; The virtual reality output module is used to present the initial scene and the digital counselor located in the scene to the user; The physiological evaluation module is used to collect the user's physiological data in real time through the physiological data sensor; The interactive evaluation module is used to guide the user to share his or her emotional state through the digital counselor and obtain the conversation data between the user and the digital counselor; evaluating the user's emotional state based on the conversation data, and evaluating the user's emotional fluctuations based on the physiological data; The VR scene adjustment module is used to display a customized scene corresponding to the user's current emotional state through a virtual reality device to provide psychological counseling for the user; When the user's emotional fluctuation reaches a first emotional fluctuation threshold, the emotional state customized scene is replaced with a corresponding emotional state adjustment scene; When the user's emotional fluctuation reaches a second emotional fluctuation threshold, the emotional state adjustment scene is changed to an emotional state consolidation scene; Among them, the second emotion fluctuation threshold is smaller than the first emotion fluctuation threshold.

Images