Extended reality-based psychological counseling method and system, and computing device for performing same
The augmented reality-based psychological counseling system addresses the limitations of conventional techniques by using customized avatars and counselor-controlled interactions to enhance immersion and effectiveness in counseling scenarios involving difficult subjects, ensuring safety and continuity.
Patent Information
- Application Number
- PCT/KR2026/005958
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2026-05-08
- Filing Date
- 2026-05-11
- Publication Date
- 2026-11-12
AI Technical Summary
Conventional psychological counseling techniques struggle to provide a realistic and emotionally immersive experience, particularly when counseling subjects that cannot be physically faced, such as deceased family members or conflict partners, due to limitations in client imagination and emotional readiness.
An augmented reality-based psychological counseling system and method that utilizes avatars generated based on user data and psychological state indicators to create customized counseling scenarios, allowing interaction with virtual representations of subjects like deceased individuals or pets, and enabling counselors to control avatar expressions and movements.
Enhances the sense of reality and emotional immersion in counseling, providing tailored support for various psychological needs, including mourning, reconciliation with the past self, and conflict resolution, while ensuring safety and continuity through integrated management of counseling data.
Smart Images

Figure KR2026005958_12112026_PF_FP_ABST
Abstract
Description
Augmented Reality-based psychological counseling method and system and computing device for performing the same
[0001] An embodiment of the present invention relates to an extended reality-based psychological counseling technique.
[0002] With the recent advancements in Virtual Reality (VR), AI-based avatar generation technology, and speech synthesis technology, services that allow users to interact with virtual objects or virtual characters within a virtual space are being utilized in various fields. In particular, 3D avatar generation technology, voice cloning technology, Text-to-Speech (TTS) technology, and Head Mounted Display (HMD)-based immersive interface technology are showing increasing potential for use in education, training, and entertainment, as well as in psychological support and counseling.
[0003] Meanwhile, in the field of psychological counseling, counseling techniques have been utilized in which clients envision a subject they find difficult to converse with or cannot realistically meet, and are encouraged to express their emotions or unresolved thoughts to that subject. However, conventional counseling techniques have limitations; the effectiveness of the counseling can vary depending on the client's capacity for imagination, emotional readiness, or degree of immersion in the counseling setting, and it is difficult for clients to fully experience a sense of social presence as if the actual subject were present before them. In particular, it is challenging to reliably provide a sense of reality and emotional immersion in the counseling setting for subjects that cannot be physically faced, such as deceased family members, past selves, conflict partners, or objects of mourning.
[0004] The present invention is intended to provide an augmented reality-based psychological counseling method and system, and a computing device for performing the same.
[0005] A method according to one disclosed embodiment is performed in a computing device having one or more processors and a memory for storing one or more programs executed by said one or more processors, comprising: receiving a psychological counseling request including one or more of a photograph of a subject required for psychological counseling, counseling purpose information, counseling subject information, and a user's life history information; obtaining information related to the user's psychological state and determining a psychological counseling mode to be provided to the user based on said psychological counseling request and one or more of said psychological state information; generating an avatar to be used for counseling based on said determined psychological counseling mode; and executing an extended reality-based psychological counseling session between the user and the counselor based on said determined psychological counseling mode and said generated avatar.
[0006] The step of determining the psychological counseling mode may include: a step of obtaining the user's prior psychological state information and the user's biosignal information to calculate a psychological state indicator prior to counseling; and a step of determining one or more psychological counseling modes to be provided to the user among a plurality of pre-set psychological counseling modes based on the psychological counseling application and the psychological state indicator prior to counseling.
[0007] The step of determining the psychological counseling mode may include: a step of calculating a suitability score for each psychological counseling mode based on one or more of the counseling purpose information, counseling subject information, life history information, and pre-counseling psychological state indicators; and a step of selecting a psychological counseling mode among the plurality of psychological counseling modes in which the calculated suitability score is greater than or equal to a preset threshold value.
[0008] If there are multiple selected psychological counseling modes, the method further includes a step of determining the order of progress of the selected psychological counseling modes, and the order of progress can be determined according to the suitability score or the psychological state indicator prior to counseling.
[0009] If there are multiple selected psychological counseling modes, the method may further include: a step of determining the order of progress of the selected psychological counseling modes; and, when the user's psychological state indicator is updated during counseling, a step of suspending the progress of a specific psychological counseling mode or changing the order of progress of the psychological counseling modes by reflecting the updated psychological state indicator.
[0010] The above method may further include the steps of: acquiring one or more of the user’s biosignal information, verbal response information, non-verbal response information, and auxiliary interaction information during counseling to calculate a psychological state indicator during counseling; generating a state change warning event when the psychological state indicator during counseling satisfies a preset threshold condition; and performing a preset alarm measure or mitigation measure in response to the state change warning event.
[0011] The above mitigation measures may reduce the frequency of avatar speech in the augmented reality space provided to the user, mitigate the intensity of avatar movement, or pause the augmented reality scene.
[0012] The step of determining the above psychological counseling mode includes the step of selecting one or more psychological counseling modes among the first to fifth psychological counseling modes that have been set, wherein the first psychological counseling mode is a psychological counseling mode in which the user has a conversation with a user in their past form, the second psychological counseling mode is a psychological counseling mode in which the user has a conversation with a user in their future form, the third psychological counseling mode is a psychological counseling mode in which the user has a conversation with one or more of a user form with low self-esteem and a user form with high self-esteem, the fourth psychological counseling mode is a psychological counseling mode in which the user has a conversation with the object of mourning (a deceased person or a pet), and the fifth psychological counseling mode may be a psychological counseling mode in which the user has a conversation with another person who is the object of conflict.
[0013] The step of generating the above avatar may generate a user avatar if the determined psychological counseling mode is one or more of the first to third psychological counseling modes, generate a deceased person avatar or a pet avatar depending on the object of mourning if the determined psychological counseling mode is the fourth psychological counseling mode, and generate another person avatar if the determined psychological counseling mode is the fifth psychological counseling mode.
[0014] The step of generating the above user avatar may generate a first type user avatar reflecting the user's past appearance when the psychological counseling mode is a first psychological counseling mode, generate a second type user avatar reflecting the user's future appearance when the psychological counseling mode is a second psychological counseling mode, and generate one or more of a third type user avatar with low self-esteem and a fourth type user avatar with high self-esteem when the psychological counseling mode is a third psychological counseling mode.
[0015] The step of generating the above pet avatar includes the step of generating metadata for linkage with the user's auxiliary terminal, and the metadata may include one or more of a contact area corresponding to a petting input, a responsive area corresponding to a play input, a vocalization candidate file corresponding to a sound input, and a contact event identifier that can be matched with a haptic feedback intensity.
[0016] The step of generating the above-mentioned other person avatar may include setting the default expression of the above-mentioned other person avatar to a blank or neutral expression, and generating expression candidate data so that an expression candidate corresponding to one of positive, negative, listening, and apology can be selected according to the counselor's intervention.
[0017] The step of executing the above psychological counseling session is such that, when the psychological counseling mode is the fourth psychological counseling mode and the object of mourning is the deceased, the deceased avatar is placed in the extended reality space with its back or side facing the user, and when the counselor inputs a rotation command or a preset progress condition is satisfied, the deceased avatar is controlled to face the user.
[0018] The step of executing the above psychological counseling session can control the operation of the pet avatar in response to auxiliary interaction information received from the user's auxiliary terminal when the psychological counseling mode is the fourth psychological counseling mode and the object of mourning is a pet.
[0019] The above method further includes the step of generating and storing major progress events that occurred during the psychological counseling session as session event data, and the session event data may include one or more of the following: the time of start of counseling mode, the time of avatar appearance, the time of avatar rotation, the time of counselor control command input, the time of user utterance, the time of auxiliary terminal interaction, the time of receiving a state change attention event, and the time of end of counseling.
[0020] The above method may further include the step of generating a counseling session identifier based on the psychological counseling application, and storing one or more of counseling application information, counseling mode determination data, avatar-related data, counseling progress-related data, speech data, voice data, auxiliary interaction data, and psychological state indicator data in a single counseling session record based on the counseling session identifier.
[0021] The above method further includes the step of providing a record screen for viewing to a user or counselor based on the counseling session record, and the record screen for viewing may include one or more of a timeline by counseling session, an avatar control history, a list of uttered texts, a graph of changes in psychological state indicators, and a list of major events.
[0022] A computing device according to one disclosed embodiment comprises: a processor; and a memory for storing one or more programs executed by the processor, wherein the processor performs the operation of receiving a psychological counseling application including one or more of a photograph of a subject required for psychological counseling, counseling purpose information, counseling subject information, and user's life history information; the operation of acquiring information related to the user's psychological state and determining a psychological counseling mode to be provided to the user based on one or more of the psychological counseling application and the information related to the psychological state; the operation of generating an avatar to be used for counseling based on the determined psychological counseling mode; and the operation of executing an extended reality-based psychological counseling session between the user and the counselor based on the determined psychological counseling mode and the generated avatar.
[0023] A system according to one disclosed embodiment is an augmented reality-based psychological counseling system comprising a user terminal, a counselor terminal, and a psychological counseling server, wherein the user terminal transmits a psychological counseling request to the psychological counseling server, the request including one or more of a photograph of a subject required for psychological counseling, information on the purpose of counseling, information on the subject of counseling, and information on the user's life history; the psychological counseling server acquires information related to the user's psychological state, determines a psychological counseling mode to be provided to the user based on one or more of the psychological counseling request and the information related to the psychological state, generates an avatar to be used for counseling based on the determined psychological counseling mode, and executes an augmented reality-based psychological counseling session between the user terminal and the counselor terminal based on the determined psychological counseling mode and the generated avatar.
[0024] According to an embodiment of the present invention, by providing an avatar in an extended reality space to a user through a user terminal and enabling a counselor to control the speech, facial expressions, and movements of the avatar through a counselor terminal, a psychological counseling environment in which a user, a counselor, and an avatar form a triangular structure can be realized.
[0025] In addition, a psychological counseling mode suitable for the user can be determined based on the user's life history information and psychological state indicators. Accordingly, customized counseling scenarios tailored to the user's counseling goals and emotional state can be provided, and various psychological support situations, such as mourning for the deceased or pets, reconciliation with the past self, dialogue with the future self, recovery of self-esteem, and resolution of conflicting relationships, can be flexibly responded to.
[0026] Furthermore, by synchronizing user, counselor, and auxiliary terminals, it provides augmented reality counseling sessions and enables integrated management of avatar data, speech records, screen recordings, counselor operation history, and psychological state indicator data for each counseling session. Accordingly, counselors can intervene precisely via avatars while monitoring user responses, and since records generated during the counseling process can be utilized for follow-up counseling, aftercare, and service improvement, the safety, continuity, and effectiveness of augmented reality-based psychological counseling can be enhanced.
[0027] FIG. 1 is a diagram showing an augmented reality-based psychological counseling system according to one embodiment of the present invention.
[0028] FIG. 2 is a block diagram showing the configuration of a psychological counseling server according to one embodiment of the present invention.
[0029] FIG. 3 is a diagram illustrating the process of creating a first type of user avatar using a user's photograph in one embodiment of the present invention.
[0030] FIG. 4 is a drawing showing a third type of user avatar and a fourth type of user avatar in one embodiment of the present invention.
[0031] FIG. 5 is a drawing showing a deceased person avatar created using a photograph of the deceased person in one embodiment of the present invention.
[0032] FIG. 6 is a flowchart illustrating an extended reality-based psychological counseling method according to one embodiment of the present invention.
[0033] FIG. 7 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.
[0034] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely illustrative and the present invention is not limited thereto.
[0035] In describing the embodiments of the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such detailed descriptions may unnecessarily obscure the essence of the present invention. Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe the embodiments of the present invention and should not be limiting in any way. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "include" or "comprise" are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.
[0036] In the following description, terms such as "transmission," "communication," "transmission," "reception," and other terms of similar meaning regarding signals or information include not only the direct transfer of signals or information from one component to another but also the transfer through other components. In particular, "transmission" or "transmitting" a signal or information to one component refers to the final destination of the signal or information and does not mean a direct destination. The same applies to the "reception" of signals or information. Furthermore, in this specification, two or more data or information are "related" means that if one data (or information) is obtained, at least a portion of another data (or information) can be obtained based thereon.
[0037] Additionally, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.
[0038] FIG. 1 is a diagram showing an augmented reality-based psychological counseling system according to one embodiment of the present invention.
[0039] Referring to FIG. 1, a psychological counseling system (100) may include a user terminal (102), an auxiliary terminal (104), a counselor terminal (106), and a psychological counseling server (108). The user terminal (102), the auxiliary terminal (104), and the counselor terminal (106) may be connected to communicate with the psychological counseling server (108) through a communication network (150).
[0040] Here, the communication network (150) may include the Internet, one or more local area networks, wide area networks, cellular networks, mobile networks, other types of networks, or a combination of these networks.
[0041] In the disclosed embodiments, extended reality may refer to an environment equipped with a computer-generated virtual space, such as virtual reality, augmented reality, mixed reality, and holograms. In this case, the virtual space may consist only of virtual objects and backgrounds created separately from the actual reality space, or it may include objects or backgrounds from the actual reality space. Hereinafter, such a virtual space of extended reality may be referred to as an "extended reality space."
[0042] The user terminal (102) is a terminal of a user (i.e., a client) receiving psychological counseling using augmented reality. In one embodiment, the user terminal (102) may be a Head Mounted Display (HMD) or smart glasses that the user can wear on their head, but is not limited thereto, and various types of devices capable of displaying an augmented reality space to the user may be used.
[0043] The user terminal (102) can transmit a psychological counseling request to the psychological counseling server (106). Here, the psychological counseling request may include a photograph of the subject required for the user's psychological counseling, information on the purpose of counseling, information on the subject of counseling, and information on the user's life history. For example, the photograph may be a photograph of a deceased person or pet that the user wishes to mourn, a photograph of the user's past, a photograph of a person with whom the user is in conflict, etc.
[0044] Here, the information regarding the purpose of counseling may include one or more of self-understanding, self-comfort, enhancement of self-affirmation, mourning, separation, conflict resolution, relationship restoration, interpersonal relationship practice, and emotional separation regarding the loss of a pet. Additionally, the information regarding the subject of counseling may include one or more of the following: one's past self, one's future self, one's self with low self-esteem, one's self with high self-esteem, a deceased family member or pet, an acquaintance, or a party in a conflictual relationship.
[0045] The user's life history information is information related to the user's life history and may include the user's financial situation (e.g., poverty or wealth, etc.), the user's educational background, the user's history of memorable events (e.g., school bullying, surgical experience, etc.), the user's romantic relationships, the user's family history, and the user's physical changes (e.g., weight gain or loss, etc.).
[0046] An application for receiving an augmented reality-based psychological counseling service may be installed on the user terminal (102). The application may be stored on a computer-readable storage medium of the user terminal (102). The application includes a predetermined set of instructions executable by the processor of the user terminal (102). The instructions may cause the processor of the user terminal (102) to perform operations according to exemplary embodiments. The computer-readable storage medium of the user terminal (102) includes components of an operating system for executing a set of instructions such as the application on the user terminal (102). For example, such an operating system may be Apple’s iOS or Google’s Android.
[0047] The auxiliary terminal (104) may be a terminal that receives additional user interactions or provides tactile feedback, either separately from the user terminal (102) or coupled to the user terminal (102). In one embodiment, the auxiliary terminal (104) may include a controller, a haptic device, a tactile feedback device, a motion input device, or a wearable input device. The auxiliary terminal (104) may be configured to allow the user to interact emotionally and tactilely with the pet avatar, particularly when the subject of counseling is a pet avatar. In one embodiment, the auxiliary terminal (104) may receive auxiliary interaction inputs, such as petting, hugging, playing, calling, feeding, or making sounds, performed by the client. The auxiliary terminal (120) may transmit the received auxiliary interaction inputs to a psychological counseling server (108).
[0048] Additionally, the auxiliary terminal (104) can provide tactile feedback to the user corresponding to the results of interaction with the avatar in the augmented reality space. For example, the auxiliary terminal (104) can output one or more of vibration, pressure, resistance, temperature, or touch to allow the user to experience sensations similar to actually touching the pet avatar in the virtual space. Accordingly, immersive psychological support can be provided during the process of mourning and parting following the loss of a pet.
[0049] The counselor terminal (106) is a terminal of a counselor that provides psychological counseling to a user who has applied for psychological counseling. The counselor terminal (106) may be a Head Mounted Display (HMD) or smart glasses that the user can wear on their head, but is not limited thereto, and various types of devices capable of displaying an extended reality space to the counselor may be used.
[0050] Each counselor can register as a counselor for the augmented reality-based psychological counseling service provided by the psychological counseling server (108) through the counselor terminal (106). At this time, the counselor terminal (106) can transmit a counselor registration application, including the counselor's career information and the counselor's photo, to the psychological counseling server (108).
[0051] The psychological counseling server (108) may be a server computing device that provides an augmented reality-based psychological counseling service. The psychological counseling server (108) may provide an augmented reality space for psychological counseling to the user terminal (102) in response to a psychological counseling request from the user terminal (102). Additionally, the psychological counseling server (108) may provide the augmented reality space to the counselor terminal (106).
[0052] At this time, the counselor provided with the augmented reality space may be selected by user input from among the previously registered counselors, or may be selected by the psychological counseling server (108). When the psychological counseling server (108) selects a counselor, the psychological counseling server (108) may select a counselor suitable for the user's psychological counseling based on the user's life history information included in the psychological counseling application and the career information of each counselor. A detailed description of the configuration and operation of the psychological counseling server (108) will be provided later.
[0053] FIG. 2 is a block diagram showing the configuration of a psychological counseling server (108) according to an embodiment of the present invention. Referring to FIG. 2, the psychological counseling server (108) may include a psychological state measurement module (111), a counseling mode determination module (113), an avatar creation module (115), a counseling progress module (117), and a record management module (119).
[0054] When the psychological state measurement module (111) receives a request for psychological counseling from the user terminal (102), it can obtain information related to the psychological state of the user. Here, the information related to the psychological state may include the user's prior psychological state information, biosignal information, verbal response information, non-verbal response information, and auxiliary interaction information. The psychological state measurement module (111) can convert the information related to the user's psychological state into a psychological state indicator that can be used for counseling.
[0055] Specifically, the psychological state measurement module (111) can obtain the user's prior psychological state information through the user terminal (102). Here, the prior psychological state information may include response information regarding the user's counseling purpose, the degree of emotional burden regarding the counseling subject, the degree of readiness to face the counseling subject, the current emotional state, whether the user can express an intention to stop counseling, and the level of stability before counseling. For example, the psychological state measurement module (111) can obtain the user's prior response to each item through the user terminal (102) using multiple-choice questions, scale questions, text input, or voice input methods.
[0056] The psychological state measurement module (111) can acquire the user's biosignals from the user terminal (102) or a separate measurement device. Here, the user's biosignals may include one or more of heart rate, heart rate variability, pulse, respiratory rate, body temperature, skin conductivity, oxygen saturation, brain waves, and electromyography. The psychological state measurement module (111) can set the biosignals acquired during a certain period of time prior to the start of counseling as reference biosignals and compare the biosignals acquired during counseling with the reference biosignals.
[0057] The psychological state measurement module (111) can obtain the user's verbal response information from the user terminal (102) during counseling. Here, the verbal response information may include one or more of the user's voice data, text data converted by voice recognition, speech rate, volume, pitch, pause time between speeches, repeated speech, crying, or trembling. The psychological state measurement module (111) can calculate a language-based feature value from the verbal response information that includes one or more of the emotional intensity, speech duration, degree of speech avoidance, and degree of emotional agitation.
[0058] The psychological state measurement module (111) can obtain non-verbal response information of the user from an HMD sensor, camera, or motion sensor included in the user terminal (102) during counseling. Here, the non-verbal response information may include one or more of the user's gaze direction, avatar gaze time, head bowing, head avoidance direction, posture change, hand movements, gestures, facial expressions, eye blinking, and HMD wearing status. The psychological state measurement module (111) can calculate non-verbal-based feature values from the non-verbal response information, including one or more of the degree of face-to-face avoidance, degree of immersion, degree of tension, and degree of stabilization.
[0059] The psychological state measurement module (111) can obtain auxiliary interaction information of the user from the auxiliary terminal (104). Here, the auxiliary interaction information may include one or more of the number of contact inputs performed by the user through the auxiliary terminal (104), duration, input intensity, input pattern, whether the interaction was interrupted, and whether feedback was accepted.
[0060] The psychological state measurement module (111) can perform preprocessing (e.g., missing value correction, time synchronization, and normalization) on the acquired psychological state-related information (including one or more of prior psychological state information, biosignal information, verbal response information, non-verbal response information, and auxiliary interaction information).
[0061] The psychological state measurement module (111) can calculate one or more psychological state indicators based on preprocessed psychological state-related information. Here, the psychological state indicators may include one or more of emotional stability, emotional intensity, tension, anxiety, face-to-face burden, avoidance, immersion, counseling continuity, and need for stabilization. In one embodiment, the psychological state measurement module (111) can calculate each psychological state indicator in the form of a numerical value, a grade value, or a state label.
[0062] The psychological state measurement module (111) can calculate psychological state indicators before counseling and psychological state indicators during counseling, respectively. At this time, the psychological state indicator before counseling can serve as a reference indicator. The psychological state indicator before counseling can be calculated based on the user's prior psychological state information and the user's biosignal information. The psychological state indicator during counseling can be calculated based on one or more of the user's biosignal information, verbal response information, non-verbal response information, and auxiliary interaction information.
[0063] The psychological state measurement module (111) can calculate the amount of change in the user's psychological state by calculating the difference between the psychological state indicator before counseling and the psychological state indicator during counseling. For example, the psychological state measurement module (111) can set the emotional stability calculated before the start of counseling as the reference stability indicator and the emotional stability calculated during counseling as the real-time stability indicator. Additionally, the psychological state measurement module (111) can calculate the amount of change in the user's stability by calculating the difference between the reference stability indicator and the real-time stability indicator.
[0064] The psychological state measurement module (111) can determine whether the psychological state indicators calculated during counseling satisfy a preset threshold condition. For example, the psychological state measurement module (111) can generate a state change warning event if the emotional intensity, tension, or anxiety level exceeds a preset threshold value, or if the counseling continuity level drops below a preset level. Here, the state change warning event may be an event indicating that the user's psychological state has reached a state requiring attention or intervention.
[0065] The counseling mode determination module (113) can determine a psychological counseling mode for a user based on one or more of the user's application for psychological counseling and psychological state indicators (i.e., determine whether to proceed with psychological counseling modes for the user). Here, the psychological counseling mode is implemented in an augmented reality space and is a mode for psychological counseling of the user, and may include a first psychological counseling mode, a second psychological counseling mode, a third psychological counseling mode, a fourth psychological counseling mode, and a fifth psychological counseling mode.
[0066] In one embodiment, the first psychological counseling mode is a psychological counseling mode in which the user converses with a past user, the second psychological counseling mode is a psychological counseling mode in which the user converses with a future user, and the third psychological counseling mode may be a psychological counseling mode in which the user converses with a user with low self-esteem or a user with high self-esteem. Additionally, the fourth psychological counseling mode is a psychological counseling mode in which the user converses or interacts with the object of mourning, and the object of mourning may include a deceased person or a pet. The fifth psychological counseling mode may be a psychological counseling mode in which the user converses with another person who is the object of conflict.
[0067] The counseling mode determination module (113) determines to proceed with at least one of the first to fifth psychological counseling modes based on one or more of the user's counseling purpose information, counseling target information, life history information, and psychological state indicators, and can determine the order of proceeding of the determined psychological counseling modes.
[0068] In one embodiment, the counseling mode determination module (113) can calculate a suitability score for each psychological counseling mode based on one or more of the user's counseling purpose information, counseling target information, life history information, and pre-counseling psychological state indicators.
[0069] For example, the suitability score may be calculated based on one or more of the following: the degree of purpose match between the counseling purpose and the psychological counseling mode, the degree of subject match between the counseling subject information and the psychological counseling mode, the degree of event relevance between the life history information and the psychological counseling mode, and the degree of safety suitability or the need for counselor intervention based on psychological state indicators. The counseling mode determination module (113) may select one psychological counseling mode or select multiple psychological counseling modes together based on the calculated suitability score.
[0070] For example, the counseling mode determination module (113) may select a psychological counseling mode in which the calculated suitability score is greater than or equal to a preset threshold. In this case, if multiple psychological counseling modes are selected, the counseling mode determination module (113) may determine the order of progress among the multiple psychological counseling modes according to the suitability score, but is not limited thereto, and may also determine the order of progress among the multiple psychological counseling modes according to the user's psychological state indicators prior to counseling. For example, if the user's emotional stability is low or the burden of face-to-face interaction is high, the counseling mode determination module (113) may determine the order so that the first psychological counseling mode or the third psychological counseling mode proceeds before the fourth psychological counseling mode or the fifth psychological counseling mode.
[0071] The counseling mode determination module (113) can suspend the progress of a specific psychological counseling mode or change the order of psychological counseling modes by reflecting the updated psychological state indicator when the user's psychological state indicator is updated during counseling.
[0072] The avatar generation module (115) can generate one or more of a user avatar, a person avatar, and a pet avatar based on a photo included in the psychological counseling application. In one embodiment, the avatar generation module (115) can generate one or more user avatars based on the user's photo and life history information included in the psychological counseling application. The avatar generation module (115) can generate a user avatar, a person avatar, or a pet avatar to be used in the psychological counseling mode determined by the counseling mode determination module (113). At this time, the avatar generation module (115) can determine whether to generate a user avatar or a person avatar depending on the type of psychological counseling mode (i.e., depending on which psychological counseling mode it is).
[0073] The avatar generation module (115) can perform an image quality evaluation to determine whether an avatar can be generated for a photo included in a psychological counseling application. Specifically, the avatar generation module (115) can analyze one or more of the resolution of the photo, the frontality of the face, lighting conditions, occlusion areas, the exaggeration of facial expressions, the possibility of face detection, and background clutter. If the quality of the photo satisfies a preset standard, the avatar generation module (115) sets the photo as a reference image for avatar generation, and if the quality of the photo does not satisfy the standard, it can generate input request information indicating that a supplementary image or additional explanatory information is required and transmit it to the user terminal (102).
[0074] Specifically, if the psychological counseling mode determined by the counseling mode determination module (113) is one or more of the first to third psychological counseling modes, the avatar creation module (115) may determine to create a user avatar. If the psychological counseling mode determined by the counseling mode determination module (113) is the fourth psychological counseling mode, the avatar creation module (115) may determine to create a third-person avatar or a pet avatar. If the psychological counseling mode determined by the counseling mode determination module (113) is the fifth psychological counseling mode, the avatar creation module (115) may determine to create a third-person avatar.
[0075] When creating a user avatar, the avatar creation module (115) can create different types of user avatars depending on the type of psychological counseling mode. When the psychological counseling mode is the first psychological counseling mode, the avatar creation module (115) can create a first type of user avatar based on the user's photo and life history information. The first type of user avatar may be an avatar that reflects the user's past appearance. At this time, the first type of user avatar may be created to reflect one or more of the user's past age group, past hairstyle, past body shape, past clothing, or appearance features of a specific period included in the life history information. FIG. 3 is a diagram showing the process of creating a first type of user avatar (an avatar that reflects the user's past appearance) using a user's photo in an embodiment of the present invention.
[0076] When the psychological counseling mode is the second psychological counseling mode, the avatar creation module (115) can create a second type of user avatar based on the user's photos and life history information. The second type of user avatar may be an avatar that reflects the user's future appearance. For example, the avatar creation module (115) can create a second type of user avatar by referring to future occupation, living environment, health status, or self-growth goals included in the life history information or counseling purpose information. Here, the first type of user avatar or the second type of user avatar is created by reflecting the user's life history information based on the user's photos, and can be created using image aging techniques and image modeling techniques.
[0077] When the psychological counseling mode is the third psychological counseling mode, the avatar generation module (115) can generate a third type of user avatar and a fourth type of user avatar based on the user's photo and life history information. The third type of user avatar and the fourth type of user avatar may be avatars representing different fragmented selves of the user. The third type of user avatar and the fourth type of user avatar may each be avatars representing users with different facial expressions, confidence, and health conditions.
[0078] The third type of user avatar may be a user avatar with low self-esteem. For example, the third type of user avatar may be created to have one or more of the following: a posture with the head positioned low, withdrawn shoulders, a low gaze, a slurred expression, or dark clothing tones, in order to express a state of low self-efficacy, withdrawal, or self-blame. The fourth type of user avatar may be a user avatar with high self-esteem. For example, the fourth type of user avatar may be created to have one or more of the following: a relatively upright posture, a steady gaze, a natural smile, bright clothing, or a steady expression, in order to express a state of self-acceptance, self-confidence, or self-affirmation. FIG. 4 is a drawing showing the third type of user avatar and the fourth type of user avatar, respectively, in an embodiment of the present invention.
[0079] When the psychological counseling mode is the fourth psychological counseling mode, the avatar creation module (115) can create a deceased person avatar or a pet avatar representing the deceased person or pet that the user is mourning.
[0080] In the fourth psychological counseling mode, when the object of mourning is the deceased, the avatar creation module (115) may receive one or more of the deceased's photograph, video of life, voice sample, clothing information, speech style information, relationship information, and atmosphere information of life, and create a deceased avatar based thereon. At this time, the deceased avatar is not limited to being created to have an appearance exactly identical to the actual deceased, but may be created to have external and symbolic similarities that allow the user to feel emotionally similar to the deceased. FIG. 5 is a diagram showing a deceased avatar created using a photograph of the deceased in an embodiment of the present invention.
[0081] In the fourth psychological counseling mode, when the object of mourning is a pet, the avatar creation module (115) may receive input of one or more of the pet’s photo, species, breed, fur color, fur length, body shape, ear shape, tail shape, characteristic pattern, and behavioral characteristics remembered by the user, and create a pet avatar based thereon. The pet avatar may be created as a three-dimensional animal model that can be displayed in the user’s field of view in an extended reality space, and interaction area data for contactable areas such as the head, back, belly, legs, or tail may be created together for interaction with the auxiliary terminal (104).
[0082] The avatar generation module (115) can generate metadata for linking with the auxiliary terminal (104) for the pet avatar. For example, the avatar generation module (115) can generate metadata including one or more of a contact area corresponding to a petting input, a responsive area corresponding to a play input, a vocalization candidate file corresponding to a sound input, and a contact event identifier that can be matched with a haptic feedback intensity.
[0083] When the psychological counseling mode is the fifth psychological counseling mode, the avatar creation module (115) can create a third party avatar representing the third party who is the subject of the user's conflict. At this time, the avatar creation module (115) can create the third party avatar by receiving a photo of the third party or text describing the third party from the user terminal (102).
[0084] When the avatar generation module (115) generates an avatar of another person with whom the conflict is made, it can maintain objectivity sufficient for the user to recognize the conversation situation with the person with whom the conflict is made, while mitigating external elements that induce excessive threat or hostility so as not to contradict the purpose of counseling. For example, the avatar generation module (115) can set the default expression of the other person's avatar to a blank or neutral expression and generate expression candidate data so that expression candidates corresponding to positive, negative, listening, or apology can be selected depending on the counselor's intervention.
[0085] The avatar creation module (115) can generate rigging data that is commonly used for each generated avatar. The rigging data may include one or more of a face rig for implementing facial expressions of the avatar, a speech rig for mouth shape synchronization, a head rig for nodding or shaking the head, a body rig for waving hands or body rotation, and an eye rig for eye processing. The rigging data may be structured data that enables the consultation progress module (117) to control the avatar's expressions, gaze, posture, and movements in an extended reality space.
[0086] The avatar generation module (115) can generate avatar-specific control item mapping information so as to be linked with the control interface of the counselor terminal (106). Here, the control item mapping information may include facial expression data, gesture data, or animation clip identifiers corresponding to one or more of basic actions, listening, affirmation, negation, questioning, sadness, greeting, turning, exiting, and farewell actions.
[0087] In one embodiment, the avatar generation module (115) can generate a user avatar, another person's avatar, or a pet avatar based on deep learning technology. The avatar generation module (115) may include a first deep learning model. In one embodiment, the avatar generation module (115) can generate a user avatar of a type corresponding to the type of psychological counseling mode by inputting a user's photo, user's life history information, and the type of psychological counseling mode into the first deep learning model.
[0088] The avatar creation module (115) can generate eye alignment data for the generated avatar. The eye alignment data may include one or more of a reference point, an eye tracking range, a head rotation limit value, and an eye alignment correction parameter for aligning the viewpoint of a user wearing an HMD with the direction of the avatar's pupils.
[0089] The counseling progress module (117) can execute an augmented reality-based psychological counseling session based on a determined psychological counseling mode and a generated avatar. The counseling progress module (117) can provide an augmented reality space for psychological counseling to one or more of a user terminal (102) and a counselor terminal (106). The counseling progress module (117) can provide an augmented reality space to one or more of a user terminal (102) and a counselor terminal (106) according to the determined psychological counseling mode and the sequence of progression between psychological counseling modes. The counseling progress module (117) can load an avatar into the augmented reality space and map an avatar to be displayed on the user terminal (102) and an avatar to be controlled on the counselor terminal (106) to the same counseling session identifier.
[0090] The counseling progress module (117) can generate screen configurations corresponding to each of the user terminal (102) and the counselor terminal (106). The user terminal (102) may be provided with an extended reality space viewed by the user through an HMD, an avatar, background objects, and counseling guide objects. The counselor terminal (106) may be provided with a control screen for the counselor that includes one or more of the following: a user field of view image, avatar status information, counseling progress steps, operation buttons, a speech input window, current counseling mode information, and session status information. At this time, the counseling progress module (117) can synchronize the screen of the user terminal (102) and the screen of the counselor terminal (106) so that they correspond to the same counseling session.
[0091] The counseling progress module (117) can control the progress stages of the counseling session by dividing them into one or more of the preparation stage, entry stage, face-to-face stage, conversation stage, response stage, and termination stage. In the preparation stage, the counseling progress module (117) can provide the user terminal (102) with instructions for wearing the HMD, instructions for proceeding with the counseling, instructions for distinguishing between virtual and reality, or instructions regarding the possibility of interrupting the counseling. In the entry stage, the counseling progress module (117) can load an augmented reality background corresponding to the counseling mode and display a waiting screen or a stabilization screen before the avatar appears.
[0092] The counseling progress module (117) can set the initial display method of the avatar during the face-to-face stage differently depending on the psychological counseling mode. For example, in the fourth psychological counseling mode, if the object of mourning is the deceased, the counseling progress module (117) can control the deceased avatar so that it is not displayed facing forward from the beginning, but is placed in the extended reality space showing the user its back or side. Subsequently, when a rotation command is input from the counselor terminal (106) or when a pre-set progress condition is satisfied, the counseling progress module (117) can control the deceased avatar to gradually rotate toward the user.
[0093] In one embodiment, when the first psychological counseling mode is executed, the counseling progress module (117) may place a first type of user avatar in an extended reality space and provide a counseling scene in which the user speaks to their past self or receives a response from their past self. In this case, the counseling progress module (117) may place the first type of user avatar in a position where it is sitting in a chair or looking at the user, and may cause the first type of user avatar to perform listening actions, head nodding, short speech, or expressions of comfort according to a control command of the counselor terminal (106) or a pre-set scenario.
[0094] When the second psychological counseling mode is executed, the counseling progress module (117) may place a second type of user avatar in the extended reality space and provide a counseling scene in which the user converses with their future self. In this case, the counseling progress module (117) may sequentially execute speech events and facial expression events so that a scene in which the second type of user avatar conveys advice, questions, or encouragement toward the user is implemented.
[0095] When the third psychological counseling mode is executed, the counseling progress module (117) may place one or more of the third type user avatar and the fourth type user avatar in the augmented reality space. In one embodiment, the counseling progress module (117) may place the third type user avatar and the fourth type user avatar in different locations so that the user can look at each avatar alternately and converse. In another embodiment, the counseling progress module (117) may display only one of the third type user avatar and the fourth type user avatar and selectively execute candidates for facial expressions and postures corresponding to the third type user avatar and the fourth type user avatar according to the counseling progress stage.
[0096] When the fourth psychological counseling mode is executed, the counseling progress module (117) may place an avatar representing the user's object of mourning in the augmented reality space. If the object of mourning is a deceased person, the counseling progress module (117) may conduct a counseling session in which the deceased avatar performs one or more actions such as appearing toward the user, turning, listening, giving a short response, saying goodbye, and exiting. If the object of mourning is a pet, the counseling progress module (117) may display a pet avatar and cause the pet avatar to perform actions such as approaching, wagging its tail, making noise, sitting, lying down, or reacting in response to auxiliary interaction information received from the auxiliary terminal (104).
[0097] When the fifth psychological counseling mode is executed, the counseling progress module (117) may place a third-party avatar representing the user's conflict target in the augmented reality space and provide a counseling scene in which the user can perform utterances regarding feelings, apologies, forgiveness, refusal, or resolving the relationship that the user could not express to the conflict target. In this case, the counseling progress module (117) may set a neutral facial expression, a listening posture, a limited frequency of utterances, or counselor approval-based utterances as defaults so that the third-party avatar is not expressed in an overly threatening manner.
[0098] The counseling progress module (117) can receive avatar control commands from the counselor terminal (106). Here, the avatar control commands may include commands corresponding to one or more of basic actions, listening, affirmation, negation, questioning, sadness, nodding, shaking the head, waving, turning, entering, exiting, parting, fading out, and waiting. The counseling progress module (117) can control the actions of the avatar to be executed in the extended reality space of the user terminal (102) according to the received avatar control commands.
[0099] The counseling progress module (117) can receive speech input information from the counselor terminal (106). The speech input information may include text entered by the counselor, selected script sentences, pre-set short response, or counselor acknowledgment speech information. The counseling progress module (117) can generate synthetic speech based on the speech input information and combine the synthetic speech with the avatar's lip shape synchronization data to output it to the user terminal (102). Accordingly, the counselor can provide an answer to the user's query or a question to the user through the counselor terminal (106) via the avatar.
[0100] The counseling progress module (117) may selectively support an automatic speech mode and a counselor-controlled speech mode. In the automatic speech mode, speech candidates for the avatar may be generated according to the output of a pre-set counseling script or conversation model, and in the counselor-controlled speech mode, only sentences entered or approved by the counselor may be output through the avatar. In one embodiment, the counseling progress module (117) may set the counselor-controlled speech mode as the default value in counseling modes with high emotional sensitivity, such as those involving a subject of mourning or a subject of conflict.
[0101] In one embodiment, the consultation progress module (117) may use a second deep learning model in an automatic speech mode. The second deep learning model may be a model trained to output an appropriate answer in response to a user's query or to ask a query to the user. That is, the second deep learning model may be a model trained to enable chatting with the user.
[0102] The consultation progress module (117) can update the camera position within the extended reality space using the HMD position information, gaze direction information, or viewpoint information of the user terminal (102). Additionally, the consultation progress module (117) can use the gaze alignment data provided by the avatar creation module (115) to correct the direction of the avatar's pupils, head, or upper body so that they naturally align with the user's viewpoint. Accordingly, the user can have an experience similar to making eye contact with the avatar.
[0103] The counseling progress module (117) may receive a state change warning event from the psychological state measurement module (111) during counseling. The state change warning event may be an event generated when the user's emotional intensity, tension, anxiety, or counseling continuity satisfies a preset condition. When the state change warning event is received, the counseling progress module (117) may perform alarm or mitigation measures in response to the state change warning event, such as outputting a warning notification to the counselor terminal (106), lowering the frequency of speech of the avatar, mitigating the intensity of the avatar's movements, or pausing the extended reality scene.
[0104] The counseling progress module (117) can execute a finishing action of the avatar at the end of the counseling phase. For example, the counseling progress module (117) can control the deceased avatar or the avatar of the other person involved in the conflict to end the counseling scene by waving a hand, walking away, gradually disappearing, or the screen turning white or black out.
[0105] The counseling progress module (117) can generate major progress events that occur during the counseling session as session event data. Here, the session event data may include one or more of the following: the time when the counseling mode starts, the time when the avatar appears, the time when the avatar rotates, the time when a counselor control command is entered, the time when a user speaks, the time when the auxiliary terminal interacts, the time when a state change warning event is received, and the time when the counseling ends.
[0106] The counseling progress module (117) can apply different scene templates for each psychological counseling mode. For example, the counseling progress module (117) can apply a chair arrangement or a mirror-type arrangement suitable for the user's self-talk in the first and second psychological counseling modes, apply a background, lighting, and sense of distance suitable for a scene of parting with a deceased person or pet in the fourth psychological counseling mode, and apply a neutral counseling room background for safe conversation with the subject of conflict in the fifth psychological counseling mode.
[0107] The counseling progress module (117) can provide counseling progress assistance information to the counselor terminal (106). Here, the counseling progress assistance information may include one or more of the current counseling stage, the next recommended stage, available avatar control commands, currently utterable scripts, the auxiliary terminal connection status, the user terminal connection status, and status change event notifications. The counselor can select the avatar's speech, facial expression, or action by referring to the counseling progress assistance information.
[0108] The record management module (119) can be configured to store, classify, link, and manage counseling-related data generated or received in the augmented reality-based psychological counseling service on a counseling session basis.
[0109] The record management module (119) can generate a counseling session identifier based on a psychological counseling request received from a user terminal (102). The counseling session identifier may be identification information for distinguishing a specific user, a specific counselor, a specific counseling time and date, and a specific psychological counseling mode. Based on the counseling session identifier, the record management module (119) can store counseling request information, data related to determining the counseling mode, avatar-related data, counseling progress-related data, speech data, voice data, auxiliary interaction data, and psychological state indicator data by linking them into a single counseling session record.
[0110] The record management module (119) can manage access rights to counseling session records. For example, the record management module (119) can set different access rights for users (clients), counselors, administrators, and researchers. Clients can access records that are allowed to be viewed among their counseling records, counselors can access records of counseling sessions they handled, and researchers can access only de-identified data.
[0111] The record management module (119) may perform one or more of the following processes to secure counseling session records: encryption, de-identification, pseudonymization, and storage of access logs. For example, the record management module (119) may encrypt and store data corresponding to sensitive information among the client's name, contact information, original face image, original voice data, and life history information. Additionally, if the counseling session records are provided as research data or learning data, the record management module (119) may remove client identifiers, counselor identifiers, and counseling subject identification information, or convert them into alternative identifiers.
[0112] The record management module (119) can manage the retention period and deletion conditions of counseling session records. For example, the record management module (119) can set a retention period for each counseling record based on the scope of the client's consent, the counseling agency's policy, or legal retention obligations. When the retention period expires or the client's request for deletion is approved, the record management module (119) can delete the relevant counseling session record or some data contained in the relevant counseling session record, or process it in a state where it cannot be recovered.
[0113] The record management module (119) can provide a record screen for viewing to the counselor terminal (106) or user terminal (102) based on the counseling session record. For example, the record management module (119) can provide the counselor terminal (106) with a timeline by counseling session, avatar control history, a list of spoken texts, a graph of changes in psychological state indicators, or a list of major events.
[0114] In this specification, the term "module" may refer to a functional and structural combination of hardware for carrying out the technical concept of the present invention and software for driving said hardware. For example, the "module" may refer to a logical unit of a specific code and a hardware resource for executing said code, and does not necessarily refer to physically connected code or a single type of hardware.
[0115] FIG. 6 is a flowchart illustrating an augmented reality-based psychological counseling method according to an embodiment of the present invention. In the illustrated flowchart, the method is described by dividing it into a plurality of steps, but at least some of the steps may be performed in a different order, combined with other steps, omitted, divided into detailed steps, or performed with one or more steps not illustrated added.
[0116] Referring to FIG. 6, a psychological counseling server (108) can receive a psychological counseling request from a user terminal (102) (S 101). The psychological counseling request may include a photo of the subject required for the user's psychological counseling, information on the purpose of counseling, information on the subject of counseling, and information on the user's life history.
[0117] Next, the psychological counseling server (108) can obtain information related to the user's psychological state and calculate a psychological state indicator prior to counseling based on the information related to the psychological state (S 103). In one embodiment, the information related to the psychological state may include the user's prior psychological state information and biosignal information, etc.
[0118] Next, the psychological counseling server (108) can determine a psychological counseling mode to be provided to the user based on the psychological counseling application and the psychological state indicators prior to counseling (S 105). The psychological counseling server (108) can determine one or more psychological counseling modes to be provided to the user from a plurality of pre-set psychological counseling modes (e.g., first psychological counseling mode to fifth psychological counseling mode).
[0119] Next, the psychological counseling server (108) can generate an avatar to be used for counseling based on the determined psychological counseling mode and counseling target type (S 107).
[0120] Next, the psychological counseling server (108) can execute an augmented reality-based psychological counseling session based on the determined psychological counseling mode and the generated avatar (S 109).
[0121] Next, the psychological counseling server (108) receives input for controlling one or more of the avatar's speech, facial expressions, and movements from one or more of the user terminal (102), auxiliary terminal (104), and counselor terminal (106), and can control the avatar according to the received input (S 111).
[0122] Next, when counseling is terminated, the psychological counseling server (108) can store and manage data generated in the counseling session based on the counseling session identifier (S 113).
[0123] FIG. 7 is a block diagram illustrating a computing environment (10) including a computing device suitable for use in exemplary embodiments. In the illustrated embodiments, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.
[0124] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a device for providing an augmented reality-based psychological counseling service. The computing device (12) may be a user terminal (102). Additionally, the computing device (12) may be an auxiliary terminal (104). Additionally, the computing device (12) may be a counselor terminal (106). Additionally, the computing device (12) may be a psychological counseling server (108).
[0125] The computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) can cause the computing device (12) to operate according to the exemplary embodiment described above. For example, the processor (14) can execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause the computing device (12) to perform operations according to the exemplary embodiment when executed by the processor (14).
[0126] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by a processor (14). In one embodiment, the computer-readable storage medium (16) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other forms of storage media that are accessed by a computing device (12) and capable of storing desired information, or a suitable combination thereof.
[0127] The communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and the computer-readable storage medium (16).
[0128] The computing device (12) may also include one or more input / output interfaces (22) and one or more network communication interfaces (26) that provide interfaces for one or more input / output devices (24). The input / output interfaces (22) and network communication interfaces (26) are connected to a communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) through the input / output interfaces (22). An exemplary input / output device (24) may include an input device such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices and / or imaging devices, and / or an output device such as a display device, a printer, a speaker and / or a network card. An exemplary input / output device (24) may be included inside the computing device (12) as a component constituting the computing device (12), or it may be connected to the computing device (12) as a separate device distinct from the computing device (12).
[0129] Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof.
Claims
1. One or more processors, and A method performed in a computing device having a memory for storing one or more programs executed by the above-mentioned one or more processors, wherein A step of receiving a psychological counseling application that includes one or more of a photograph of the subject required for the user's psychological counseling, information on the purpose of counseling, information on the subject of counseling, and the user's life history information; A step of obtaining information related to the psychological state of the user and determining a psychological counseling mode to be provided to the user based on one or more of the psychological counseling application and the information related to the psychological state; A step of generating an avatar to be used for counseling based on the psychological counseling mode determined above; and A method comprising the step of executing an extended reality-based psychological counseling session between a user and a counselor based on the psychological counseling mode determined above and the generated avatar.
2. In Claim 1, The step of determining the above psychological counseling mode is, A step of obtaining prior psychological state information of the user and biosignal information of the user to calculate a psychological state indicator prior to counseling; and A method comprising the step of determining one or more psychological counseling modes to be provided to a user among a plurality of pre-set psychological counseling modes based on the above psychological counseling application and the above psychological state indicators prior to counseling.
3. In Claim 2, The step of determining the above psychological counseling mode is, A step of calculating a suitability score for each psychological counseling mode based on one or more of the above counseling purpose information, counseling subject information, life history information, and pre-counseling psychological state indicators; and A method comprising the step of selecting a psychological counseling mode among the plurality of psychological counseling modes in which the calculated suitability score is greater than or equal to a preset threshold value.
4. In Claim 3, If there are multiple psychological counseling modes selected above, The above method further includes the step of determining the proceeding order of the selected psychological counseling modes, and The above procedure is a method determined according to the above suitability score or the above pre-counseling psychological state indicator.
5. In Claim 3, If there are multiple psychological counseling modes selected above, The above method comprises the step of determining the proceeding order of the selected psychological counseling modes; and A method further comprising the step of, when the psychological state indicator of the user is updated during counseling, suspending the progress of a specific psychological counseling mode or changing the order of progress of the psychological counseling modes by reflecting the updated psychological state indicator.
6. In Claim 1, The above method is, A step of obtaining one or more of the user’s biosignal information, verbal response information, non-verbal response information, and auxiliary interaction information during counseling to calculate a psychological state indicator during counseling; A step of generating a state change warning event when a psychological state indicator satisfies a preset threshold condition during the above counseling; and A method further comprising the step of performing a preset alarm action or mitigation action in response to the above-mentioned state change warning event.
7. In Claim 6, The above mitigation measures are, A method for lowering the frequency of avatar speech, mitigating the intensity of avatar movement, or pausing the augmented reality scene in an augmented reality space provided to the user.
8. In Claim 1, The step of determining the above psychological counseling mode includes the step of selecting one or more psychological counseling modes among the previously set first to fifth psychological counseling modes, and The first psychological counseling mode is a psychological counseling mode in which the user engages in a conversation with the user in their past form, and The second psychological counseling mode is a psychological counseling mode in which the user engages in a conversation with a future version of the user, and The third psychological counseling mode is a psychological counseling mode in which the user engages in a conversation targeting one or more of the user personas of low self-esteem and high self-esteem, and The 4th psychological counseling mode is a psychological counseling mode in which the user converses with the object of their mourning (the deceased or a pet), and The 5th psychological counseling mode is a method, which is a psychological counseling mode in which the user has a conversation with another person who is the subject of conflict.
9. In Claim 8, The step of generating the above avatar is, A method for creating a user avatar when the determined psychological counseling mode is one or more of the first to third psychological counseling modes, creating a deceased person avatar or a pet avatar according to the object of mourning when the determined psychological counseling mode is the fourth psychological counseling mode, and creating another person avatar when the determined psychological counseling mode is the fifth psychological counseling mode.
10. In Claim 9, The step of creating the above user avatar is, A method for generating a first type of user avatar reflecting the user's past appearance when the above psychological counseling mode is a first psychological counseling mode, generating a second type of user avatar reflecting the user's future appearance when the above psychological counseling mode is a second psychological counseling mode, and generating one or more of a third type of user avatar reflecting low self-esteem and a fourth type of user avatar reflecting high self-esteem when the above psychological counseling mode is a third psychological counseling mode.
11. In Claim 9, The step of creating the above pet avatar is, It includes the step of generating metadata for linking with the auxiliary terminal of the above-mentioned user, and A method comprising one or more of the following metadata: a contact area corresponding to a stroking input, a responsive area corresponding to a play input, a vocalization candidate file corresponding to a sound input, and a contact event identifier that can be matched with a haptic feedback intensity.
12. In Claim 9, The step of creating the above-mentioned other person's avatar is, A method comprising the step of setting the default expression of the above-mentioned other person avatar to a blank or neutral expression, and generating expression candidate data such that an expression candidate corresponding to one of positive, negative, listening, and apology can be selected according to the counselor's intervention.
13. In Claim 9, The step of conducting the above psychological counseling session is, A method in which, when the above psychological counseling mode is the fourth psychological counseling mode and the object of mourning is the deceased, the deceased avatar is placed in an extended reality space showing its back or side to the user, and when the counselor inputs a rotation command or a pre-set progress condition is satisfied, the deceased avatar is controlled to face the user.
14. In Claim 9, The step of conducting the above psychological counseling session is, A method for controlling the operation of the pet avatar in response to auxiliary interaction information received from the user’s auxiliary terminal when the above psychological counseling mode is the fourth psychological counseling mode and the object of mourning is a pet.
15. In Claim 1, The above method is, It further includes the step of generating and saving major progress events that occurred during the above psychological counseling session as session event data, and The above session event data includes one or more of the following: the start time of the consultation mode, the time of the avatar appearance, the time of the avatar rotation, the time of the counselor control command input, the time of the user utterance, the time of the auxiliary terminal interaction, the time of the reception of a state change attention event, and the time of the end of the consultation.
16. In Claim 1, The above method is, A method further comprising the step of generating a counseling session identifier based on the above psychological counseling application, and storing one or more of counseling application information, counseling mode determination data, avatar-related data, counseling progress-related data, speech data, voice data, auxiliary interaction data, and psychological state indicator data in a single counseling session record based on the above counseling session identifier.
17. In Claim 16, The above method is, The method further includes the step of providing a record screen for viewing to a user or counselor based on the above counseling session record, The above-mentioned inquiry record screen includes one or more of a counseling session timeline, avatar control history, utterance text list, psychological state indicator change graph, and a list of major events.
18. Processor; and A computing device comprising memory for storing one or more programs executed by the above processor, wherein the processor, The action of receiving a psychological counseling application that includes one or more of a photograph of the subject required for the user's psychological counseling, information on the purpose of counseling, information on the subject of counseling, and the user's life history information; An operation to obtain information related to the psychological state of the user and to determine a psychological counseling mode to be provided to the user based on one or more of the psychological counseling application and the information related to the psychological state; The operation of generating an avatar to be used for counseling based on the psychological counseling mode determined above; and A computing device that performs the operation of executing an extended reality-based psychological counseling session between a user and a counselor based on the psychological counseling mode determined above and the generated avatar.
19. An augmented reality-based psychological counseling system comprising a user terminal, a counselor terminal, and a psychological counseling server, wherein The above user terminal transmits a psychological counseling application to the above psychological counseling server, the application including one or more of a photograph of a subject required for the user's psychological counseling, information on the purpose of counseling, information on the subject of counseling, and information on the user's life history. A system in which the psychological counseling server acquires information related to the psychological state of the user, determines a psychological counseling mode to be provided to the user based on one or more of the psychological counseling application and the information related to the psychological state, generates an avatar to be used for counseling based on the determined psychological counseling mode, and executes an extended reality-based psychological counseling session between the user terminal and the counselor terminal based on the determined psychological counseling mode and the generated avatar.
