Extended reality-based interpersonal relationship diagnosis method and computing device for performing same

Abstract

Disclosed are an extended reality-based interpersonal relationship diagnosis method and a computing device for performing same. The interpersonal relationship diagnosis method, disclosed according to one embodiment, comprises the steps of: upon receiving an interpersonal relationship diagnosis request of a user terminal, providing first extended reality content for psychological diagnosis of a user, on the basis of one or more among user-related information and social life-related information; on the basis of one or more among the user-related information and the social life-related information, providing second extended reality content for assessing one or more among the communication style and the communication skills of the user; on the basis of one or more among the user-related information and the social life-related information, providing third extended reality content for assessing one or more among the empathy style and the empathy ability of the user; and assessing an interpersonal relationship diagnosis of the user on the basis of one or more among the first to third extended reality content.

Description

Interpersonal relationship diagnosis method based on extended reality and computing device for performing the same

[0001] Embodiments of the present invention relate to extended reality-based interpersonal relationship diagnosis and psychotherapy technology.

[0002] Nowadays, modern people are under a lot of mental stress due to comparisons and competition with others, which can lead to various psychological problems such as depression and anxiety disorders. However, most people find it difficult or unwilling to discuss their issues face-to-face with a counselor, which hinders the effective implementation of psychological counseling. Therefore, there is a need for a method that allows psychological healing without face-to-face interaction with a counselor.

[0003] Furthermore, many office workers complain about communication issues and experience conflicts with colleagues, superiors, or subordinates. Therefore, self-awareness of one's communication skills and styles and ways to improve them are essential. Furthermore, in communal settings like school or work, self-awareness of one's empathy style and empathy abilities, along with strategies to improve them, are essential for the smooth formation, maintenance, and development of interpersonal relationships.

[0004] An embodiment of the present invention provides an extended reality-based interpersonal relationship diagnosis technique.

[0005] According to one embodiment of the present disclosure, a method for diagnosing interpersonal relationships based on extended reality is provided, which is performed in a computing device having one or more processors and a memory storing one or more programs executed by the one or more processors, the method comprising: receiving an interpersonal relationship diagnosis request from a user terminal, the interpersonal relationship diagnosis request including at least one of user-related information and social life-related information of the user; providing first extended reality content for psychological diagnosis of the user to the user terminal based on at least one of the user-related information and the social life-related information; providing second extended reality content for evaluating at least one of the user's communication type and communication ability based on at least one of the user-related information and the social life-related information to the user terminal; providing third extended reality content for evaluating at least one of the user's empathy type and empathy ability based on at least one of the user-related information and the social life-related information to the user terminal; and evaluating the interpersonal relationship diagnosis of the user based on at least one of the first extended reality content to the third extended reality content.

[0006] According to one embodiment of the present disclosure, a computing device comprises: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein the one or more programs include: a command for receiving an interpersonal relationship diagnosis request from a user terminal, the request including at least one of user-related information and social life-related information of the user; a command for providing, to the user terminal, first extended reality content for a psychological diagnosis of the user based on at least one of the user-related information and the social life-related information; a command for providing, to the user terminal, second extended reality content for evaluating at least one of the user's communication type and communication ability based on at least one of the user-related information and the social life-related information; a command for providing, to the user terminal, third extended reality content for evaluating at least one of the user's empathy type and empathy ability based on at least one of the user-related information and the social life-related information; and a command for evaluating the user's interpersonal relationship diagnosis based on at least one of the first extended reality content to the third extended reality content.

[0007] According to the disclosed embodiment, by allowing a user to converse with different types of user avatars or other avatars in an extended reality space, the user can comfort himself or express his feelings without having to meet a counselor face-to-face, thereby enabling self-healing through psychological counseling.

[0008] Furthermore, it can help each user become aware of their own communication style and style, and help them learn to balance consideration for others with their own emotions and thoughts in various situations encountered in the workplace and social life. Furthermore, rather than relying on simple questionnaires, the system allows users to participate as virtual humans (avatars) in augmented reality-based situational scenarios, allowing them to immerse themselves in the situation as if they were in real life, resulting in more accurate and reliable assessments.

[0009] Additionally, through interpersonal relationship diagnosis, you can identify your empathy type and empathy ability, and be matched with a psychological counselor to receive psychological counseling and psychological healing for any areas where you are lacking.

[0010] Furthermore, by enabling each user to recognize their own empathy type and capacity, the system can effectively express understanding and empathy for others in various social situations, such as work or school. Furthermore, rather than relying on simple questionnaires, the system allows users to participate as virtual humans (avatars) in augmented reality-based situational scenarios, allowing them to immerse themselves in the situation as if they were in real life, resulting in more accurate and reliable assessments.

[0011] FIG. 1 is a diagram illustrating an extended reality-based interpersonal relationship diagnosis system according to one embodiment of the present invention.

[0012] Figure 2 is a block diagram showing the configuration of a management server according to one embodiment of the present invention.

[0013] FIG. 3 is a diagram illustrating a process of creating a first type of user avatar using a user's photo in one embodiment of the present invention.

[0014] FIG. 4 is a diagram showing a third type of user avatar and a fourth type of user avatar in one embodiment of the present invention.

[0015] FIG. 5 is a screen showing a portion of a conversation between people around the user in the second extended reality content according to one embodiment of the present invention.

[0016] FIG. 6 is a diagram showing a problem submission screen for a situation scenario according to a first relationship in one embodiment of the present invention.

[0017] Figure 7 is a block diagram showing the configuration of a diagnostic evaluation module according to one embodiment of the present invention.

[0018] FIG. 8 is a diagram showing a state in which a first detailed empathy type to a third detailed empathy type are determined for a user in an embodiment of the present invention.

[0019] Figure 9 is a diagram showing an empathy type indicator according to one embodiment of the present invention.

[0020] Figure 10 is a diagram showing a state in which the first empathy ability level and the second empathy ability level are measured in one embodiment of the present invention.

[0021] FIG. 11 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.

[0022] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, devices, and / or systems described herein. However, these are merely examples and the present invention is not limited thereto.

[0023] In describing embodiments of the present invention, if a detailed description of a known technology related to the present invention is judged to unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of their functions in the present invention, and this may vary depending on the intention or custom of the user or operator. Therefore, the definitions should be made based on the contents throughout this specification. The terminology used in the detailed description is only for the purpose of describing embodiments of the present invention and should not be limited in any way. Unless clearly used otherwise, the singular form includes the plural form. In this description, expressions such as "comprises" or "having" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, and should not be construed to exclude the presence or possibility of one or more other features, numbers, steps, operations, elements, parts or combinations thereof other than those described.

[0024] Additionally, while terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms may be used to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component."

[0025] FIG. 1 is a diagram illustrating an extended reality-based interpersonal relationship diagnosis system according to one embodiment of the present invention.

[0026] Referring to FIG. 1, the interpersonal relationship diagnosis system (100) may include a user terminal (102), a feedback measurement device (104), a counselor terminal (106), and a management server (108). The user terminal (102), the feedback measurement device (104), and the counselor terminal (106) may be communicatively connected to the management server (108) via a communication network (150).

[0027] Here, the communication network (150) may include the Internet, one or more local area networks, wide area networks, a cellular network, a mobile network, other types of networks, or a combination of these networks.

[0028] In the disclosed embodiments, "extended reality" may refer to an environment comprising a computer-generated virtual space, such as virtual reality, augmented reality, mixed reality, and holograms. In this case, the virtual space may consist solely of virtually generated objects and backgrounds separate from the actual real space, or may include objects or backgrounds from the actual real world. Hereinafter, such a virtual space of extended reality may be referred to as an "extended reality space."

[0029] The user terminal (102) is a terminal of a user who wishes to receive interpersonal relationship diagnosis using augmented reality. The user can receive psychological healing through the interpersonal relationship diagnosis. In one embodiment, the user terminal (102) may be a head-mounted display (HMD) or smart glasses that the user can wear on his or her head, but is not limited thereto, and various types of devices capable of displaying the augmented reality space to the user may be used. The user terminal (102) may include an interface (e.g., a mouse, a joystick, a keyboard, etc.) for inputting user commands in the augmented reality space.

[0030] A user terminal (102) may access a management server (108) to receive extended reality content. Here, the extended reality content may be content for diagnosing a user's interpersonal relationships in an extended reality space. The extended reality content may include one or more situational scenarios. The extended reality content may include one or more of: first extended reality content for diagnosing the user's psychology; second extended reality content for evaluating one or more of the user's communication types and abilities; and third extended reality content for evaluating one or more of the user's empathy types and abilities.

[0031] A user terminal (102) may transmit a user voice registration request to the management server (108). The user voice registration request may include user-related information, such as the user's age, gender, etc. Additionally, the user voice registration request may include the user's voice data. Specifically, voice data obtained by recording the user's voice for a certain period of time may be included in the user voice registration request. At this time, the user may view and read example text provided by the management server (108).

[0032] After a user voice registration request is made, the user terminal (102) can receive a completion notification of user voice registration from the management server (108). The management server (108) can receive the user voice registration request, extract the user voice data from the user voice registration request, and then train a deep learning model using the user voice data as training data. When the training of the deep learning model is completed, the management server (108) can transmit a completion notification of user voice registration to the user terminal (102).

[0033] A user terminal (102) can receive extended reality content for interpersonal relationship diagnosis by sending a request for interpersonal relationship diagnosis to the management server (108). The request for interpersonal relationship diagnosis may include user-related information and social life-related information. User-related information may include the user's name, race, age, gender, occupation, photo, and life history information.

[0034] Here, the user's life history information is information related to the life the user has lived, and may include a history of the user's financial situation (e.g., poverty or wealth), a history of the user's educational background, a history of impressive events the user has experienced (e.g., school violence, surgical experiences, etc.), a history of the user's romantic relationships, a history of the user's family history, and a history of the user's physical changes (e.g., weight gain or loss, etc.).

[0035] Additionally, information related to the user's social life may include information about the community to which the user belongs and in which the user lives (e.g., school, workplace, etc.) and information about the user's surroundings within the community (e.g., information about the user's surroundings' gender, age, and relationship with the user, etc.).

[0036] That is, social life-related information may include information about people around the user who interact with the user in their social lives (e.g., community life such as work or school). For example, this may include information about the gender, age, and relationship with the user (work colleagues, superiors, subordinates, friends, etc.) of each person around the user.

[0037] The feedback measurement device (104) can measure the user's feedback while the interpersonal relationship diagnosis for the user is in progress through the extended reality content. Here, the user's feedback can include the user's gaze, facial expression, voice, gesture, and posture. The feedback measurement device (104) can transmit user feedback information including one or more of the user's gaze, facial expression, voice, gesture, and posture to the management server (108). The feedback measurement device (104) can be implemented by being separated into multiple devices to measure various user feedback. In this case, some functions or some configurations of the feedback measurement device (104) can also be implemented in the user terminal (102).

[0038] In one embodiment, the feedback measurement device (104) can capture a user while an interpersonal relationship diagnosis is being performed on the user, and track the user's gaze based on the captured image of the user. Since the image-based gaze tracking technology is a well-known technology, a detailed description thereof will be omitted. In addition, the feedback measurement device (104) can extract the user's facial expression (i.e., extract what facial expression the user is making) based on the captured image of the user. In addition, the feedback measurement device (104) can extract the user's voice from the captured image of the user. In addition, the feedback measurement device (104) can extract the user's gesture and posture (i.e., extract what gesture and posture the user is making) based on the captured image of the user.

[0039] The counselor terminal (106) is a counselor terminal that provides psychological counseling to a user who has applied for interpersonal relationship diagnosis. The counselor terminal (106) may be an HMD (Head Mounted Display) or smart glasses that the counselor can wear on his or her head, but is not limited thereto, and various types of devices that can display an extended reality space to the counselor may be used. Here, a corresponding counselor may be matched based on the user's interpersonal relationship diagnosis result. Each counselor may be a professional who has completed a preset extended reality-based psychological counseling training course. The user can undergo psychological healing through counseling with the matched counselor.

[0040] Each counselor can register as a counselor for the augmented reality-based service provided by the management server (108) through the counselor terminal (106). At this time, the counselor terminal (106) can transmit a counselor registration application including the counselor's career information and photo to the management server (108). Each counselor can apply for augmented reality-based psychological counseling education and receive augmented reality-based psychological counseling education.

[0041] The management server (108) may be a server computing device that provides psychological healing services through interpersonal relationship diagnosis based on extended reality. Upon receiving a request for interpersonal relationship diagnosis from a user terminal (102), the management server (108) may provide extended reality content for interpersonal relationship diagnosis to the user terminal (102). At this time, the management server (108) may also provide extended reality content (e.g., first extended reality content) to the counselor terminal (106) depending on the type of extended reality content.

[0042] FIG. 2 is a block diagram illustrating the configuration of a management server (108) according to one embodiment of the present invention. Referring to FIG. 2, the management server (108) may include a first content provision module (111), a second content provision module (113), a third content provision module (115), a reaction-related module (117), and a diagnostic evaluation module (119).

[0043] The first content provision module (111) can provide first extended reality content for psychological diagnosis of the user to the user terminal (102). To this end, the first content provision module (111) can obtain information related to the psychological state of the user. The information related to the psychological state of the user can include information about the user's emotions or psychological state. In one embodiment, the first content provision module (111) can obtain information related to the psychological state of the user from the user terminal (102) or a separate psychological state measurement device (not shown). For example, the user's emotions or psychological state can be obtained through a questionnaire on the user, or can be obtained by measuring the user's biosignals (e.g., brain waves, pulse, heart rate, body temperature, oxygen saturation, skin conductance, etc.).

[0044] The first content provision module (111) can determine a psychological counseling mode for the user based on one or more of the user's life history information and the acquired psychological state-related information included in the interpersonal relationship diagnosis application. That is, the first content provision module (111) can determine which psychological counseling mode to proceed with for the user based on one or more of the user's life history information and psychological state-related information.

[0045] Additionally, the first content provision module (111) may determine the order of progression of psychological counseling modes based on one or more of the user's life history information and psychological state-related information. For example, if it is determined that multiple psychological counseling modes will be performed for a user, the order of progression among the psychological counseling modes may also be determined based on one or more of the user's life history information and psychological state-related information.

[0046] Here, the psychological counseling mode is implemented in an extended reality space and is a mode for psychological counseling of the user, and may include a first psychological counseling mode, a second psychological counseling mode, a third psychological counseling mode, and a fourth psychological counseling mode.

[0047] In one embodiment, the first psychological counseling mode may be a psychological counseling mode in which the user engages in conversation with a past version of the user. The second psychological counseling mode may be a psychological counseling mode in which the user engages in conversation with a future version of the user. The third psychological counseling mode may be a psychological counseling mode in which the user engages in conversation with at least one of a low self-esteem user and a high self-esteem user. The fourth psychological counseling mode may be a psychological counseling mode in which the user engages in conversation with another person who is the subject of the user's grief or conflict.

[0048] The first content provision module (111) determines to proceed with at least one of the first psychological counseling mode to the fourth psychological counseling mode based on at least one of the user's life history information and psychological state-related information, and can determine the order of proceeding of the determined psychological counseling modes.

[0049] The first content provision module (111) can generate one or more user avatars based on the user's photo and life history information. The first content provision module (111) can generate user avatars or other avatars to be used in a predetermined psychological counseling mode. In this case, the first content provision module (111) can determine whether to generate a user avatar or other avatar depending on the type of psychological counseling mode (i.e., depending on the psychological counseling mode).

[0050] Specifically, if the predetermined psychological counseling mode is at least one of the first psychological counseling mode, the third psychological counseling mode, or the like, the first content provision module (111) may determine to generate a user avatar. If the predetermined psychological counseling mode is the fourth psychological counseling mode, the first content provision module (111) may determine to generate another person's avatar.

[0051] When creating a user avatar, the first content provision module (111) can create different types of user avatars depending on the type of psychological counseling mode. If the psychological counseling mode is the first psychological counseling mode, the first content provision module (111) can create a first type of user avatar based on the user's photo and life history information. The first type of user avatar may be an avatar that reflects the user's past appearance.

[0052] When the psychological counseling mode is the second psychological counseling mode, the first content provision module (111) can generate a second type of user avatar based on the user's photo and life history information. The second type of user avatar may be an avatar that reflects the user's future appearance. Here, the first type of user avatar or the second type of user avatar is generated by reflecting the user's life history information based on the user's photo, and may be generated using an image aging technique, an image modeling technique, or the like. FIG. 3 is a diagram illustrating a process of generating a first type of user avatar (an avatar reflecting the user's past appearance) using the user's photo in one embodiment of the present invention.

[0053] When the psychological counseling mode is the third psychological counseling mode, the first content provision module (111) can generate a third type of user avatar and a fourth type of user avatar based on the user's photo and life history information. The third type of user avatar and the fourth type of user avatar may be avatars representing different fragmented selves of the user. The third type of user avatar and the fourth type of user avatar may be avatars representing users with different facial expressions, confidence levels, and health conditions, respectively.

[0054] A third type of user avatar may be a user avatar with low self-esteem. For example, a third type of user avatar may be a user avatar with the appearance of a servant. A fourth type of user avatar may be a user avatar with high self-esteem. For example, a fourth type of user avatar may be a user avatar with the appearance of a superior. Figure 4 is a diagram illustrating a third type of user avatar and a fourth type of user avatar according to one embodiment of the present invention.

[0055] When the psychological counseling mode is the fourth psychological counseling mode, the first content provision module (111) can generate a third party avatar representing the other party who is the object of the user's grief or conflict. At this time, the first content provision module (111) can generate the third party avatar by receiving a photo of the target third party or text describing the target third party from the user terminal (102).

[0056] In one embodiment, the first content provision module (111) may generate a user avatar or another person's avatar based on deep learning technology. The first content provision module (111) may include a first deep learning model (111a). In one embodiment, the first content provision module (111) may input a user's photo, the user's life history information, and the type of psychological counseling mode into the first deep learning model (111a) to generate a user avatar of a type corresponding to the type of psychological counseling mode.

[0057] The first content provision module (111) can provide the first extended reality content to one or more of the user terminal (102) and the counselor terminal (106) according to the predetermined psychological counseling mode and the progression order between the psychological counseling modes.

[0058] In one embodiment, when the psychological counseling mode is the first psychological counseling mode, the first content provision module (111) can display a first type of user avatar in the extended reality space and allow the user to have a conversation with the first type of user avatar in the extended reality space.

[0059] Additionally, when the psychological counseling mode is the second psychological counseling mode, the first content provision module (111) can display a second type of user avatar in the extended reality space and allow the user to have a conversation with the second type of user avatar in the extended reality space.

[0060] When the psychological counseling mode is the third psychological counseling mode, the first content provision module (111) can display at least one of the third type of user avatar and the fourth type of user avatar in the extended reality space, and enable the user to conduct a conversation with at least one of the third type of user avatar and the fourth type of user avatar in the extended reality space.

[0061] When the psychological counseling mode is the fourth psychological counseling mode, the first content provision module (111) can display another person's avatar in the extended reality space and allow the user to engage in a conversation with the other person's avatar in the extended reality space. In this case, if the other person's avatar is the object of mourning, the user can provide an opportunity to say goodbye to family or friends who were unable to mourn due to the sudden death. Furthermore, if the other person's avatar is the object of conflict, the user can soothe their emotions by engaging in an indirect conversation with the object of the conflict.

[0062] The first content provision module (111) can enable each type of user avatar or another person's avatar to look at the user while sitting on a chair and conduct a conversation with the user.

[0063] Meanwhile, the first content provision module (111) can enable each type of user avatar or other avatar to answer the user's inquiry or ask the user a question. However, this is not limited thereto, and the first content provision module (111) can receive an answer to the user's inquiry or an inquiry to the user from the counselor terminal (106) and allow the received answer or inquiry to be spoken through each type of user avatar or other avatar. In this case, the counselor can perform a role-play by assuming the role of the user avatar or other avatar in the extended reality space. The counselor can remotely control the user avatar or other avatar in the extended reality space through the counselor terminal (106) to conduct a conversation with the user.

[0064] Additionally, the first content provision module (111) can obtain information related to the user's psychological state (i.e., information about the user's emotions or psychological state) while providing the first extended reality content. Furthermore, the first content provision module (111) can control one or more of the user's avatar's facial expressions and speech patterns or the avatars of other users in the extended reality space based on the information related to the user's psychological state during the user's consultation.

[0065] The first content provision module (111) can check the user's emotional or psychological state through information related to the user's psychological state, and control the user's avatar or another avatar (e.g., another avatar who is the object of mourning) in the extended reality space to make facial expressions and speech to comfort or sympathize with the user based on the user's emotional or psychological state. In addition, the first content provision module (111) can control another avatar (e.g., another avatar who is the object of conflict) in the extended reality space to make facial expressions and speech to ask for forgiveness from the user based on the user's emotional or psychological state.

[0066] The first content provision module (111) can terminate an ongoing psychological counseling mode based on information related to the user's psychological state. For example, if the user's emotional or psychological state exceeds a preset threshold, the first content provision module (111) can terminate the ongoing psychological counseling mode. Furthermore, the first content provision module (111) can change the order of progress between psychological counseling modes based on information related to the user's psychological state.

[0067] The second content provision module (113) may provide second extended reality content to the user terminal (102) for evaluating one or more of the user's communication types and abilities. The second content provision module (113) may generate second extended reality content for evaluating the user's communication abilities and communication types, etc., based on the user's social life-related information included in the interpersonal relationship diagnosis application.

[0068] Specifically, the second content provision module (113) can generate a first relationship-based situation scenario and a second relationship-based situation scenario based on the user's social life-related information. The first relationship-based situation scenario may be a scenario that enacts a situation in which people around the user's social life communicate in a horizontal relationship. The second relationship-based situation scenario may be a scenario that enacts a situation in which people around the user's social life communicate in a vertical relationship.

[0069] The second content provision module (113) can generate multiple situation scenarios based on the first relationship. For example, the second content provision module (113) can generate multiple situation scenarios based on the first relationship, such as a situation of dividing work among colleagues, a situation of selecting a menu at a restaurant, or a situation of planning a vacation schedule.

[0070] The second content provision module (113) can generate multiple situation scenarios based on the second relationship. For example, the second content provision module (113) can generate multiple situation scenarios based on the second relationship, such as a night shift situation, instructions for expedited work, or a situation where there are generational differences in opinions regarding company dinners.

[0071] The second content provision module (113) can provide second extended reality content according to the situation scenario according to the first relationship and the situation scenario according to the second relationship to the user terminal (102), respectively. The second extended reality content according to the situation scenario according to the first relationship and the situation scenario according to the second relationship may be sequentially provided according to the user's selection, but is not limited thereto.

[0072] The second augmented reality content may include virtual humans (i.e., avatars) representing the user's surroundings, generated based on information about the user's social life. Furthermore, the second augmented reality content may also include virtual humans representing the user. The virtual humans representing the user's surroundings may be displayed in the second augmented reality content, speaking and gesturing according to the relevant situational scenario.

[0073] The second content provision module (113) can submit questions to the user for communication-related evaluation in the second augmented reality content. The second content provision module (113) can submit questions to elicit the user's opinion in situations where the user and his / her surroundings appear as characters in a situation scenario based on the first or second relationship.

[0074] In one embodiment, the second content provision module (113) may submit questions in the form of multiple choice questions. That is, for each situation scenario, the user may be asked to select from multiple examples (i.e., multiple choice) how they would respond or express their opinion. For each situation scenario, the second content provision module (113) may submit questions in the user's voice.

[0075] That is, the second content provision module (113) can use the user's voice data included in the user voice registration application to submit a problem as the user's voice in each situation scenario. The second content provision module (113) can train the second deep learning model (113a) based on the user's voice data and the example text used to obtain the user's voice data. The second content provision module (113) can input the example text and cause the second deep learning model (113a) to output a voice corresponding to the example text. At this time, the second deep learning model (113a) can be trained so that the voice output by the second deep learning model (113a) is similar to the user's voice data by using the user's voice data as the correct answer value.

[0076] When the training of the second deep learning model (113a) is completed, the second content provision module (113) can input text corresponding to the problem in each situation scenario into the second deep learning model (113a), and have the second deep learning model (113a) output the text corresponding to the problem in the user's voice. At this time, the user can select his / her opinion while being more immersed in the situation by hearing several examples (i.e., options) of the problem in each situation scenario in his / her own voice.

[0077] In one embodiment, a situation scenario according to the first relationship is a situation in which a team has been working overtime for over a week on an important project, but is given a new task immediately after finishing the work. In this situation, it is assumed that the people around the user (A, B, C, and D) are having the following conversation. FIG. 5 is a screen showing a portion of the conversation between the people around the user in the second extended reality content according to one embodiment of the present invention.

[0078] A: Oh my, I really can't do this. My child has been alone in the classroom all day for the past week. It's so sad to see him left alone... You want him to do that for another week? No, no.

[0079] B: I also made a hospital appointment for tomorrow. I postponed it once last week because of working overtime, so I can't postpone it any longer. If I cancel the appointment, it will be pushed back by three months, which is a big problem.

[0080] C: Oh, my wife can't eat anything because of morning sickness... They say that the sadness you feel when you're pregnant lasts a lifetime.

[0081] D: I also have a gathering with my in-laws this weekend... What should I do? It's my father's birthday dinner, but I was busy with work last week, so I postponed it until this week.

[0082] C: Who doesn't have circumstances? Isn't that right, Jiyoung (user name)?

[0083] At this time, the second content provision module (113) can output multiple examples that the user can select from in the corresponding situation scenario through the user's voice. Fig. 6 is a diagram illustrating a question submission screen for a situation scenario according to the first relationship in one embodiment of the present invention.

[0084] Referring to Figure 6, example 1 is "That's right. So... I told you not to put up with it any longer. You're underestimating us because we're just sitting still.", example 2 is "Yes, it seems like a really difficult situation. But how about we find a way to share it somehow?", example 3 is "I don't think we'll find a solution if we talk about it right now. Let's go eat for now!", and example 4 is "Rather than talking about your situation, wouldn't it be more efficient to discuss how we can divide the work? Let's make a plan one by one." These four examples can be output as the user's voice, and the user can select one of the four responses.

[0085] The second content provision module (113) may provide an interface (in FIG. 6, an interface called "Direct Answer") that allows the user to directly answer (i.e., subjectively) if there is no example among the four examples for the user to select. The user may input his / her answer by voice through the interface.

[0086] The second content provision module (113) can ensure that each example submitted in each situation scenario falls into one of the preset communication types. In one embodiment, the communication types can be categorized into Leveling, Placing, Blaming, Computing, and Distracting.

[0087] Here, the congruent type is the most ideal type, a communication style that respects self, others, and the situation. It refers to a type where the content of communication matches internal feelings, and the type that is able to accurately and appropriately express their feelings in words and is psychologically stable. The appeasement type ignores themselves and respects others and the situation. They suppress their own internal feelings and thoughts, prioritize the feelings of others, and tend not to oppose others' opinions.

[0088] The criticizing type ignores others and respects themselves and the situation. They tend to find and criticize others' flaws to make themselves look strong, and they tend to control others' words and actions. The hyper-rational type ignores themselves and others and only respects the situation. They value principles and rules, strive to be objective in any situation, and are rational and cool-headed. The distracting type ignores themselves, others, and the situation altogether. They are characterized by inconsistent words and actions, displaying irrelevant behavior, and resorting to jokes or meaningless stories to avoid situations.

[0089] The second content provision module (113) can include four examples of communication types (excluding the consensual type) in the question submission for each situation scenario: persuasive, accusatory, hyper-rational, and distractive. In this case, the user can listen to the four examples of persuasive, accusatory, hyper-rational, and distractive types in their own voice and select an answer (response) that best reflects their thoughts or their usual behavior.

[0090] The third content provision module (115) may provide third extended reality content to the user terminal (102) for evaluating one or more of the user's empathy types and abilities. The third content provision module (115) may generate third extended reality content for diagnosing one or more of the user's empathy types and empathy abilities based on the user-related information included in the interpersonal relationship diagnosis application and the user's social life-related information.

[0091] Specifically, the third content provision module (115) can generate a first situation scenario, a second situation scenario, and a third situation scenario based on user-related information and information related to the user's social life. The first situation scenario may be a scenario that creates a situation to identify the user's cognitive empathy versus emotional empathy. Cognitive empathy may refer to an individual's ability to understand and recognize the emotions and thoughts of others. Affective empathy may refer to an individual's ability to vicariously feel another's emotional state.

[0092] The first situation scenario may include a 1-1 situation scenario that creates a positive situation to identify the user's cognitive empathy versus emotional empathy, and a 1-2 situation scenario that creates a negative situation to identify the user's cognitive empathy versus emotional empathy.

[0093] In one embodiment, the first scenario may be a scenario where the avatar of the user's friend A (male) confesses his feelings to a woman and is accepted. The second scenario may be a scenario where the avatar of the user's friend B (male) confesses his feelings to a woman and is rejected.

[0094] The second scenario could be a scenario that creates a situation to identify the user's positive versus negative empathy. Positive empathy can refer to an individual's ability to understand and recognize the positive emotions of others. Negative empathy can refer to an individual's ability to understand and recognize the negative emotions of others.

[0095] The second situation scenario may include a 2-1 situation scenario that creates a positive situation to identify the user's positive empathy versus negative empathy, and a 2-2 situation scenario that creates a negative situation to identify the user's positive empathy versus negative empathy.

[0096] In one embodiment, the 2-1 situation scenario may be a scenario in which the avatar of the user's friend C wins a badminton competition. The 2-2 situation scenario may be a scenario in which the avatar of the user's friend D is eliminated in the first round of a badminton competition.

[0097] A third scenario could be one that creates a situation to identify the user's majority empathy versus minority empathy. Majority empathy could refer to the ability to understand and recognize a situation that is positive for the majority but negative for the minority. Minority empathy could refer to the ability to understand and recognize a situation that is positive for the minority but negative for the majority.

[0098] The third situation scenario may include a 3-1 situation scenario that creates a majority positive situation and a minority negative situation to identify the majority empathy versus minority empathy of the user, and a 3-2 situation scenario that creates a majority negative situation and a minority positive situation to identify the majority empathy versus minority empathy of the user.

[0099] In one embodiment, the 3-1 situation scenario may be a scenario in which the avatar of E, one of the user's classmates, receives birthday wishes from his friends, while the avatar of F, one of the user's classmates, loses his cell phone. The 3-2 situation scenario may be a scenario in which the soccer team of the user's class loses, while the avatar of G, one of the user's classmates, wins a 100-meter dash.

[0100] The third content provision module (115) can provide third extended reality content according to the first to third situation scenarios to the user terminal (102). The third extended reality content according to the first to third situation scenarios can be sequentially provided according to the user's selection, but is not limited thereto.

[0101] Third-party augmented reality content may include virtual people (i.e., avatars) representing the user's surroundings and the background of the space where the user engages in social activities, generated based on information related to the user's social life. Third-party augmented reality content may also include virtual people representing the user. Virtual people representing surroundings may be displayed in the augmented reality content, speaking and gesturing according to the relevant situational scenario.

[0102] The third content provision module (115) can submit questions for empathy-related diagnostics to the user in the third augmented reality content. The third content provision module (115) can submit multiple questions to elicit the user's opinions on any one of the first to third situation scenarios provided to the user.

[0103] In one embodiment, the third content provision module (115) may present the question in the form of multiple choice questions. That is, the question may be presented so that the user can select from multiple examples (i.e., multiple choice) how they would respond or express their opinion in each situation scenario.

[0104] In one embodiment, the third content provision module (115) may submit questions such as “Please select the emotion you felt after listening to A’s story from the list”, “Please select the emotion you felt after listening to B’s story from the list”, “How much do you agree with A’s statement that he was excited and happy because he was confessed to?”, “How much do you agree with B’s statement that he felt depressed and sad as if he had confessed and been rejected?”, “Whose story did you want to hear more when you listened to A’s and B’s stories?”, “Whose emotion influenced you more between A and B?”

[0105] In addition, the third content provision module (115) can submit questions such as "Please select the emotion you felt after listening to C's story from the list", "How much do you agree with C's statement that he was excited and happy as if he won first place in a competition?", "Please select the emotion you felt after listening to D's story from the list", "How much do you agree with D's statement that he was depressed and sad as if he lost in the first round?", "You have a glass of water to give to your friend. Who would you rather give it to, C or D?", "When you listened to C and D's stories, whose story did you want to hear more?", "Whose emotion were you more influenced by, C or D?"

[0106] In addition, the third content provision module (115) can submit questions such as “Which did you experience more of, the feeling of celebrating a birthday than F’s feelings?”, “Which did you want to do more, helping F feel better or celebrating a birthday together?”, “Which did you feel more of, feeling sorry for F or feeling happy that there were students celebrating his birthday?” in the 3-1 situation scenario.

[0107] In addition, the third content provision module (115) can submit questions such as “Which did you experience more, G’s emotions or the soccer team’s emotions?”, “Which did you want to do more, helping the soccer team feel better or congratulating G?”, “Which did you feel more, feeling sorry for the soccer team or feeling good about G?” in the 3-2 situation scenario.

[0108] The reaction-related module (117) can obtain the user's reaction while conducting interpersonal relationship diagnosis based on extended reality content. Specifically, the reaction-related module (117) can receive user feedback information including one or more of the user's gaze, facial expression, voice, and gesture from the feedback measurement device (104) while the interpersonal relationship diagnosis is in progress. In one embodiment, the reaction-related module (117) can receive user feedback information from the feedback measurement device (104) while a question is submitted in extended reality content and the user answers the question, but is not limited thereto.

[0109] The reaction-related module (117) can extract one or more of the user's emotions and intentions based on user feedback information. The reaction-related module (117) can convert the user's voice from the user feedback information into text and interpret the meaning of the text through syntactic analysis of the converted text. The reaction-related module (117) can extract one or more of the user's emotions and intentions using the meaning of the user's voice, the tone and volume of the voice, the user's gaze, the user's facial expression, and the user's gestures.

[0110] The reaction-related module (117) may include a third deep learning model (117a) to identify the user's emotions and intentions. The third deep learning model (117a) may be composed of multiple models. In one embodiment, the third-first deep learning model may be trained to receive text about the user's voice and output the user's emotion or communication type. The third-second deep learning model may be trained to receive the user's facial expression and output the user's emotional state (e.g., positive, negative, neutral, etc.). The third-third deep learning model may be trained to receive the user's voice and output the tone and volume of the user's voice. The reaction-related module (117) may combine the respective output results of the second-first to second-third deep learning models to finally output what the user's emotion or intention is.

[0111] The reaction-related module (117) can enable the user's emotions or intentions to be expressed in the user avatar of the extended reality content. In other words, the reaction-related module (117) can enable the user avatar of the extended reality content to react according to the user's emotions or intentions. This allows the user and the user avatar in the extended reality space to interact.

[0112] The diagnostic evaluation module (119) can evaluate the user's interpersonal relationship diagnosis based on the user's answers to problems presented in the extended reality content provided to the user and user feedback information obtained during the interpersonal relationship diagnosis process. Fig. 7 is a block diagram illustrating the configuration of the diagnostic evaluation module (119) according to one embodiment of the present invention. Referring to Fig. 7, the diagnostic evaluation module (119) can include a first diagnostic evaluation unit (119-1), a second diagnostic evaluation unit (119-2), and a third diagnostic evaluation unit (119-3).

[0113] The first diagnostic evaluation unit (119-1) can calculate the user's psychological index based on one or more of the first extended reality content provided to the user, information related to the user's psychological state obtained while the first extended reality content is provided, and user feedback information.

[0114] The first diagnostic evaluation unit (119-1) can calculate a user's psychological index based on the psychological counseling mode of the first extended reality content, information related to the user's psychological state, and user feedback information. In one embodiment, the user's psychological index may be a numerical expression of whether the user has a positive or negative psychological state.

[0115] The second diagnostic evaluation unit (119-2) can evaluate one or more of the user's communication style and communication skills based on the user's answers to problems presented in the second extended reality content provided to the user and user feedback information obtained while the second extended reality content is provided.

[0116] Specifically, the second diagnostic evaluation unit (119-2) can check whether the user's answer to the question submitted in each of the multiple situation scenarios according to the first relationship and the multiple situation scenarios according to the second relationship is an objective answer or a subjective answer.

[0117] Here, if the user's answer to the problem is an objective answer, the second diagnostic evaluation unit (119-2) can count each communication type (persuasion type, accusatory type, hyperrational type, and distracting type) corresponding to the example selected by the user in the objective answer.

[0118] In addition, if the user's answer to the question is a subjective answer, the second diagnostic evaluation unit (119-2) can determine the user's communication type based on the user's voice for the subjective answer and user feedback information including the user's gaze, facial expression, and gestures during the subjective answer.

[0119] The second diagnostic evaluation unit (119-2) can convert the user's voice for a subjective answer into text and analyze the meaning of the converted text. At this time, the second diagnostic evaluation unit (119-2) can analyze the tone of the user's voice to confirm the exact meaning of the text. In addition, the second diagnostic evaluation unit (119-2) can analyze the user's gaze, facial expression, and gestures in a subjective answer situation. The second diagnostic evaluation unit (119-2) can determine the user's communication type based on the text of the user's voice for a subjective answer and the user's gaze, facial expression, and gestures in a subjective answer situation, and can count the determined communication types.

[0120] The second diagnostic evaluation unit (119-2) may include a fourth deep learning model (119-2a) to determine the user's communication type for a subjective answer. The fourth deep learning model (119-2a) may be trained to classify the user's communication type by inputting the user's voice text for a subjective answer, and the user's gaze, facial expression, and gesture in a subjective answer situation. The fourth deep learning model (119-2a) may express the user's communication type as a probability value for each communication type. At this time, the second diagnostic evaluation unit (119-2) may determine the user's communication type based on the communication type with the highest probability value output from the fourth deep learning model (119-2a), and count the determined communication types.

[0121] The second diagnostic evaluation unit (119-2) can determine the user's communication type based on the counted number of communication types according to the user's multiple-choice and subjective responses to questions submitted in the situation scenarios according to the first and second relationships. For example, the second diagnostic evaluation unit (119-2) can determine the communication type with the highest count as the user's communication type.

[0122] Additionally, the second diagnostic evaluation unit (119-2) may calculate a score or probability that the user belongs to each communication type based on the number of times the user's counted communication type occurs compared to the total number of submitted problems.

[0123] The second diagnostic evaluation unit (119-2) can evaluate the user's communication ability based on the user's answers to problems presented in the second extended reality content and user feedback information obtained during the communication evaluation process.

[0124] Specifically, the second diagnostic evaluation unit (119-2) can calculate a first evaluation score for the user's comprehension ability based on the user's responses and user feedback information. That is, the second diagnostic evaluation unit (119-2) can assign a first evaluation score by determining how well the user understands the conversation topic and submitted problem for each given situation scenario in the extended reality space.

[0125] Additionally, the second diagnostic evaluation unit (119-2) can calculate a second evaluation score for the user's concentration based on the user's responses and user feedback information. That is, the second diagnostic evaluation unit (119-2) can determine the degree to which the user is concentrating on each given situation scenario in the extended reality space and assign a second evaluation score.

[0126] Additionally, the second diagnostic evaluation unit (119-2) can calculate a third evaluation score for the user's communication skills based on the user's responses and feedback information. Specifically, the second diagnostic evaluation unit (119-2) can determine how well the user communicates their opinions on the presented issues in each given scenario in the extended reality space and assign a third evaluation score.

[0127] The second diagnostic evaluation unit (119-2) may evaluate the user's communication skills based on a first evaluation score for the user's comprehension ability, a second evaluation score for the user's concentration ability, and a third evaluation score for the user's communication skills. In one embodiment, the second diagnostic evaluation unit (119-2) may calculate the user's communication skills score by adding the first evaluation score, the second evaluation score, and the third evaluation score, but is not limited thereto.

[0128] The third diagnostic evaluation unit (119-3) can evaluate one or more of the user's empathy type and empathy ability based on the user's response to a problem presented in the third extended reality content provided to the user and the user feedback information obtained while the third extended reality content is provided.

[0129] Specifically, the third diagnostic evaluation unit (119-3) can measure the user's empathy type based on the user's responses to questions submitted in the third extended reality content. The third diagnostic evaluation unit (119-3) can measure the user's empathy type based on the user's responses to each question submitted in the first to third situation scenarios of the third extended reality content.

[0130] The third diagnostic evaluation unit (119-3) can determine the user's first to third detailed empathy types based on the user's answers to the questions submitted in the first to third situation scenarios. Figure 8 is a diagram illustrating a state in which the first to third detailed empathy types are determined for a user in an embodiment of the present invention.

[0131] In one embodiment, the third diagnostic evaluation unit (119-3) may determine the user's first detailed empathy type based on the answer to the question submitted in the first situation scenario. The third diagnostic evaluation unit (119-3) may determine the first detailed empathy type indicating which empathy type the user is closer to among cognitive empathy (C) and affective empathy (A) based on the answer to the question submitted in the first situation scenario. If the user is closer to cognitive empathy among cognitive empathy and affective empathy, the user's first detailed empathy type may be determined as cognitive empathy (C), and if the user is closer to affective empathy among cognitive empathy and affective empathy, the user's first detailed empathy type may be determined as affective empathy (A).

[0132] The third diagnostic evaluation unit (119-3) can determine the second detailed empathy type of the user based on the answer to the question submitted in the second situation scenario. The third diagnostic evaluation unit (119-3) can determine the second detailed empathy type indicating which empathy type the user is closer to among positive empathy (P) and negative empathy (N) based on the answer to the question submitted in the second situation scenario. If the user is closer to positive empathy among positive empathy and negative empathy, the user's second detailed empathy type may be determined as positive empathy (P), and if the user is closer to negative empathy among positive empathy and negative empathy, the user's second detailed empathy type may be determined as negative empathy (N).

[0133] The third diagnostic evaluation unit (119-3) can determine the user's third detailed empathy type based on the answer to the question submitted in the third situation scenario. The third diagnostic evaluation unit (119-3) can determine the third detailed empathy type indicating which empathy type the user is closer to among majority empathy (M) and minority empathy (m) based on the answer to the question submitted in the third situation scenario. If the user is closer to majority empathy among majority empathy and minority empathy, the user's third detailed empathy type can be determined as majority empathy (M), and if the user is closer to minority empathy among majority empathy and minority empathy, the user's third detailed empathy type can be determined as minority empathy (m).

[0134] The third diagnostic evaluation unit (119-3) can detect the user's empathy type indicator by combining the user's first detailed empathy type to the third detailed empathy type. Here, since there are three detailed empathy types, there can be a total of eight empathy type indicators. The empathy type measurement module (115) can detect one of the eight empathy type indicators of CPm, CPM, CNm, CNM, APm, APM, ANm, and ANM as the user's empathy type indicator by taking the first letters of the user's first detailed empathy type to the third detailed empathy type. For example, the empathy type indicator CPm represents the case where the first detailed empathy type is cognitive empathy (C), the second detailed empathy type is positive empathy (P), and the third detailed empathy type is minority empathy (m).

[0135] Figure 9 is a diagram illustrating empathy type indicators according to one embodiment of the present invention. Here, each empathy type indicator may be stored along with a supplementary description of the empathy type indicator. Furthermore, each empathy type indicator may be stored along with brief information about the type representing the empathy type indicator. Furthermore, each empathy type indicator may be stored along with a character representing the empathy type indicator.

[0136] The third diagnostic evaluation unit (119-3) can provide user empathy type information to the user terminal (102). Here, the empathy type information can include the user's empathy type indicator and related information (e.g., a side description, brief type information, and characters, etc.) for the empathy type indicator.

[0137] The third diagnostic evaluation unit (119-3) can classify user feedback information obtained while providing third augmented reality content into verbal feedback and nonverbal feedback. For example, verbal feedback may include the user's voice. Verbal feedback may include not only the user's verbal responses to submitted questions, but also the user's questions, opinions, and conversations with avatars during empathy-related diagnosis. Nonverbal feedback may include the user's gaze, facial expressions, gestures, and posture during empathy-related diagnosis.

[0138] The third diagnostic evaluation unit (119-3) can measure the user's first level of empathy based on the user's language feedback information. The third diagnostic evaluation unit (119-3) can convert the user's language feedback information into text and interpret the meaning of the text through syntactic analysis of the converted text. In addition, the third diagnostic evaluation unit (119-3) can analyze the tone and volume of the user's voice based on the user's language feedback information. The third diagnostic evaluation unit (119-3) can measure the user's first level of empathy through syntactic analysis of the text and analysis of the tone and volume of the user's voice.

[0139] In one embodiment, the third diagnostic evaluation unit (119-3) may calculate the user's concentration score, clarity score, help score, support score, and encouragement score, respectively, through text syntactic analysis and analysis of the user's voice tone and volume. The third diagnostic evaluation unit (119-3) may calculate the user's first empathy ability level by adding up the user's concentration score, clarity score, help score, support score, and encouragement score.

[0140] Here, the focus score could refer to the degree to which a person focuses on listening to the other person. The clarification score could refer to the score based on questions asked to understand the other person's thoughts and feelings. The help score could refer to the score based on offering solutions or relevant information to objectively view the other person's problematic situation. The support score could refer to the score based on agreeing with or affirming the other person's feelings. The encouragement score could refer to the score based on bolstering the other person's courage or motivation to alleviate their distress or anxiety.

[0141] The third diagnostic evaluation unit (119-3) can measure the user's second empathy ability level based on the user's nonverbal feedback information. The third diagnostic evaluation unit (119-3) can calculate a first empathy expression score based on the user's gaze, a second empathy expression score based on the user's facial expression, a third empathy expression score based on the user's gesture, and a fourth empathy expression score based on the user's posture. The third diagnostic evaluation unit (119-3) can calculate the user's second empathy ability level by adding up the first empathy expression score to the fourth empathy expression score.

[0142] The third diagnostic evaluation unit (119-3) can calculate a first empathy expression score based on the degree and duration of the user's gaze toward the other party. The third diagnostic evaluation unit (119-3) can calculate a second empathy expression score based on the user's facial expression when conversing with the other party or answering a question. The third diagnostic evaluation unit (119-3) can calculate a third empathy expression score and a fourth empathy expression score based on the user's gestures and posture when conversing with the other party or answering a question.

[0143] Figure 10 is a diagram illustrating a state in which a first empathy ability level and a second empathy ability level are measured in one embodiment of the present invention. Here, the first empathy ability level can be calculated based on the user's concentration score, clarification score, help score, support score, and encouragement score. In addition, the second empathy ability level can be calculated based on the user's first to fourth empathy expression scores. In this case, the user's concentration score, clarification score, help score, support score, and encouragement score and the first to fourth empathy expression scores can be respectively displayed in a graph.

[0144] The third diagnostic evaluation unit (119-3) can calculate the user's empathy ability index by synthesizing the user's first empathy ability level and the second empathy ability level. In one embodiment, the third diagnostic evaluation unit (119-3) can calculate the user's empathy ability index by assigning a first weight to the user's first empathy ability level, assigning a second weight to the user's second empathy ability level, and then adding them up. The third diagnostic evaluation unit (119-3) can provide information related to the user's empathy ability to the user terminal (102). The empathy ability-related information can include information such as the user's first empathy ability level, the second empathy ability level, and the empathy ability index.

[0145] According to the disclosed embodiment, by allowing a user to converse with different types of user avatars or other avatars in an extended reality space, the user can comfort himself or express his feelings without having to meet a counselor face-to-face, thereby enabling self-healing through psychological counseling.

[0146] Furthermore, it can help each user become aware of their own communication style and style, and help them learn to balance consideration for others with their own emotions and thoughts in various situations encountered in the workplace and social life. Furthermore, rather than relying on simple questionnaires, the system allows users to participate as virtual humans (avatars) in augmented reality-based situational scenarios, allowing them to immerse themselves in the situation as if they were in real life, resulting in more accurate and reliable assessments.

[0147] Furthermore, by enabling each user to recognize their own empathy type and capacity, the system can effectively express understanding and empathy for others in various social situations, such as work or school. Furthermore, rather than relying on simple questionnaires, the system allows users to participate as virtual humans (avatars) in augmented reality-based situational scenarios, allowing them to immerse themselves in the situation as if they were in real life, resulting in more accurate and reliable assessments.

[0148] As used herein, the term "module" may refer to a functional and structural combination of hardware for implementing the technical concepts of the present invention and software for operating the hardware. For example, the term "module" may refer to a logical unit of a given code and hardware resources for executing the given code, and does not necessarily refer to physically connected code or a single type of hardware.

[0149] FIG. 11 is a block diagram illustrating a computing environment (10) including a computing device suitable for use in exemplary embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

[0150] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a user terminal (102). Additionally, the computing device (12) may be a feedback measurement device (104). Additionally, the computing device (12) may be a counselor terminal (106). Additionally, the computing device (12) may be a management server (108).

[0151] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.

[0152] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and store desired information, or a suitable combination thereof.

[0153] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).

[0154] The computing device (12) may also include one or more input / output interfaces (22) that provide interfaces for one or more input / output devices (24) and one or more network communication interfaces (26). The input / output interfaces (22) and the network communication interfaces (26) are connected to the communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) via the input / output interfaces (22). Exemplary input / output devices (24) may include input devices such as pointing devices (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as display devices, printers, speakers and / or network cards. The exemplary input / output devices (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0155] While representative embodiments of the present invention have been described in detail above, those skilled in the art will appreciate that various modifications to the above-described embodiments are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims set forth below but also by equivalents thereof.

Claims

1. One or more processors, and A method performed on a computing device having a memory storing one or more programs executed by one or more processors, A step of receiving an interpersonal relationship diagnosis request including at least one of user-related information and social life-related information from a user terminal; A step of providing first extended reality content for psychological diagnosis of the user to the user terminal based on at least one of the user-related information and the social life-related information; A step of providing second extended reality content to the user terminal for evaluating one or more of the user's communication type and communication ability based on one or more of the user-related information and the social life-related information; A step of providing third extended reality content to the user terminal for evaluating one or more of the user's empathy type and empathy ability based on one or more of the user-related information and the social life-related information; and An extended reality-based interpersonal relationship diagnosis method, comprising a step of evaluating the user's interpersonal relationship diagnosis based on at least one of the first extended reality content and the third extended reality content.

2. In claim 1, The above user-related information includes the user's life history information, The above life history information is, An extended reality-based interpersonal relationship diagnosis method comprising one or more of the following: a history of the user's financial situation, a history of the user's education, a history of memorable events experienced by the user, a history of the user's romantic relationships, a history of the user's family history, and a history of the user's physical changes.

3. In claim 2, The above interpersonal relationship diagnosis method is, Further comprising a step of obtaining psychological state-related information about the user's emotional or psychological state while providing the first extended reality content; The step of providing the above first extended reality content is: An extended reality-based interpersonal relationship diagnosis method, comprising a step of determining to proceed with at least one of the first psychological counseling mode to the fourth psychological counseling mode based on at least one of the user's life history information and the psychological state-related information.

4. In claim 3, The first psychological counseling mode is a psychological counseling mode where users have conversations with users who appear in the past. The second psychological counseling mode is a psychological counseling mode where users have conversations with future users. The third psychological counseling mode is a psychological counseling mode in which users have conversations targeting at least one of the user appearances of low self-esteem and high self-esteem. The fourth psychological counseling mode is an extended reality-based interpersonal relationship diagnosis method that allows users to converse with others who are the subject of their grief or conflict.

5. In claim 4, In the step of determining the psychological counseling mode, if it is decided to proceed with multiple psychological counseling modes, a step of determining the order of proceeding among the multiple psychological counseling modes based on at least one of the user's life history information and the psychological state-related information is further included. The step of providing the above first extended reality content is: An extended reality-based interpersonal relationship diagnosis method, comprising a step of terminating the ongoing psychological counseling mode or changing the progress order of the psychological counseling mode based on information related to the psychological state of the user.

6. In claim 3, The steps to evaluate the above interpersonal relationship diagnosis are: An extended reality-based interpersonal relationship diagnosis method, comprising a step of calculating a user's psychological index based on a psychological counseling mode of the first extended reality content, information related to the user's psychological state, and user feedback information.

7. In claim 1, The above second extended reality content includes at least one situation scenario according to the first relationship and at least one situation scenario according to the second relationship, The situation scenario according to the above first relationship is a scenario that creates a situation in which people around the user's social life communicate in a horizontal relationship. The situation scenario according to the second relationship is an extended reality-based interpersonal relationship diagnosis method that creates a situation in which people around the user's social life communicate in a vertical relationship.

8. In claim 7, The step of providing the above second extended reality content is: Including a step of submitting a problem for communication-related evaluation in a situation scenario according to the first relationship and a situation scenario according to the second relationship, The steps to evaluate the above interpersonal relationship diagnosis are: An extended reality-based interpersonal relationship diagnosis method that evaluates one or more of a user's communication style and communication skills based on the user's response to a problem presented in the second extended reality content provided to the user and user feedback information obtained while the second extended reality content is provided.

9. In claim 8, The steps to evaluate the above interpersonal relationship diagnosis are: A step of checking whether the user's answer to the question submitted in each situation scenario of the second extended reality content is an objective answer or a subjective answer; If the user's answer is a multiple choice answer, a step of counting each communication type corresponding to an example selected by the user from the multiple choice answer; and An extended reality-based interpersonal relationship diagnosis method, comprising a step of determining a user's communication type based on the user's voice and user feedback information for the subjective answer, if the user's answer is a subjective answer.

10. In claim 9, The above user feedback information includes one or more of the user's gaze, facial expression, and gesture in a subjective response situation, The steps for determining the above communication type are: A step of inputting the text of the user's voice for the subjective answer, one or more of the user's gaze, facial expression, and gesture in the subjective answer situation into a pre-trained deep learning model to determine the user's communication type; and An extended reality-based interpersonal relationship diagnosis method, comprising a step of counting the determined communication types.

11. In claim 10, The steps to evaluate the above communication types are: An extended reality-based interpersonal relationship diagnosis method that evaluates a user's communication type based on the counted number of communication types according to the user's multiple-choice and subjective answers to questions submitted in each of the above situation scenarios.

12. In claim 8, The step of evaluating the user's communication skills is: A step of calculating a first evaluation score for the user's understanding ability based on the user's answer to the above problem and the user feedback information; A step of calculating a second evaluation score for the user's concentration based on the user's answer to the above problem and the user feedback information; A step of calculating a third evaluation score for the user's delivery ability based on the user's response to the above problem and the user feedback information; and An extended reality-based interpersonal relationship diagnosis method, comprising a step of calculating a communication ability score of the user based on the first evaluation score, the second evaluation score, and the third evaluation score.

13. In claim 1, The above third extended reality content is, An extended reality-based interpersonal relationship diagnosis method, comprising a first situation scenario that creates a situation to identify a user's cognitive empathy versus emotional empathy, a second situation scenario that creates a situation to identify a user's positive empathy versus negative empathy, and a third situation scenario that creates a situation to identify a user's majority empathy versus minority empathy.

14. In claim 13, The step of providing the above third extended reality content is: Including a step of submitting a problem for diagnosing the user's empathy in the first situation scenario, the second situation scenario, and the third situation scenario, An extended reality-based interpersonal relationship diagnosis method, wherein the step of evaluating the interpersonal relationship diagnosis includes a step of measuring the user's empathy type based on the user's response to the problem submitted in the third extended reality content.

15. In claim 14, The steps to measure the above empathy types are: A step of determining a first detailed empathy type indicating which type of empathy the user is closer to among cognitive empathy and emotional empathy based on the answer to the question submitted in the first situation scenario; A step of determining a second detailed empathy type indicating which type of empathy the user is closer to among positive empathy and negative empathy based on the answer to the question submitted in the second situation scenario; A step of determining a third detailed empathy type indicating which type of empathy the user is closer to among majority empathy and minority empathy based on the answer to the question submitted in the third situation scenario; and An extended reality-based interpersonal relationship diagnosis method, comprising a step of detecting a user's empathy type indicator by combining the first detailed empathy type, the second detailed empathy type, and the third detailed empathy type.

16. In claim 14, The steps to evaluate the above interpersonal relationship diagnosis are: A step of obtaining user feedback information while providing the third extended reality content; and An extended reality-based interpersonal relationship diagnosis method, comprising a step of measuring a user's empathy ability based on the above user feedback information.

17. In claim 16, The steps to measure the above empathy ability are: A step of dividing the above user feedback information into verbal feedback information and nonverbal feedback information; A step of measuring the user's first level of empathy ability based on the user's language feedback information; A step of measuring the user's second empathy ability level based on the user's nonverbal feedback information; and An extended reality-based interpersonal relationship diagnosis method, comprising a step of calculating a user's empathy ability index by synthesizing the user's first empathy ability level and second empathy ability level.

18. One or more processors; memory; and Contains one or more programs, The one or more programs are stored in the memory and configured to be executed by the one or more processors, One or more of the above programs, A command for receiving an interpersonal relationship diagnosis request including at least one of user-related information and social life-related information from a user terminal; A command for providing first extended reality content for psychological diagnosis of the user to the user terminal based on at least one of the user-related information and the social life-related information; A command for providing second extended reality content to the user terminal for evaluating one or more of the user's communication type and communication ability based on one or more of the user-related information and the social life-related information; A command for providing third extended reality content to the user terminal for evaluating one or more of the user's empathy type and empathy ability based on one or more of the user-related information and the social life-related information; and A computing device comprising a command for evaluating an interpersonal relationship diagnosis of the user based on one or more of the first extended reality content and the third extended reality content.

19. A computer program stored in a non-transitory computer readable storage medium, The computer program includes one or more instructions, which, when executed by a computing device having one or more processors, cause the computing device to: A step of receiving an interpersonal relationship diagnosis request including at least one of user-related information and social life-related information from a user terminal; A step of providing first extended reality content for psychological diagnosis of the user to the user terminal based on at least one of the user-related information and the social life-related information; A step of providing second extended reality content to the user terminal for evaluating one or more of the user's communication type and communication ability based on one or more of the user-related information and the social life-related information; A step of providing third extended reality content to the user terminal for evaluating one or more of the user's empathy type and empathy ability based on one or more of the user-related information and the social life-related information; and A computer program that performs a step of evaluating the interpersonal relationship diagnosis of the user based on at least one of the first extended reality content and the third extended reality content.

Images