Avatar service method reflecting user's cultural characteristics and computing device for performing the same

Abstract

A method for providing an avatar service that reflects the cultural characteristics of a user and a computing device for performing the same are disclosed. An avatar service method according to one disclosed embodiment is a method performed on a computing device having one or more processors and a memory for storing one or more programs executed by one or more processors, and comprises the steps of: building a face image database for face images that do not have issues related to portrait rights; receiving a service request for augmented reality content from a user terminal; extracting one face image from the face image database in accordance with the service request; and generating a user avatar based on the extracted face image.

Description

Technology Field

[0001] An embodiment of the present invention relates to avatar service technology that reflects the cultural characteristics of a user. Background Technology

[0003] Recently, with the advancement of artificial intelligence technology, technologies are continuously being developed to represent users as virtual humans (i.e., avatars) in virtual reality spaces and enable them to engage in activities through these avatars. However, since avatars were traditionally created by scanning users' photos, issues regarding portrait rights may arise. Prior art literature

[0005] Registered Patent Publication No. 10-1866407 (June 12, 2018) The problem to be solved

[0006] An embodiment of the present invention is intended to provide an avatar service method capable of generating an avatar that reflects the cultural characteristics of a user without portrait rights issues, and a computing device for performing the same. means of solving the problem

[0008] An avatar service method according to one disclosed embodiment is a method performed on a computing device having one or more processors and a memory storing one or more programs executed by said one or more processors, comprising: a step of building a face image database for face images that are free from portrait rights issues; a step of receiving a service request for augmented reality content from a user terminal; a step of extracting one face image from said face image database in accordance with said service request; and a step of generating a user avatar based on the extracted face image.

[0009] The above service application may include cultural characteristic information regarding one or more of the user's gender, race, age, personality, and occupation.

[0010] The step of constructing the above face image database includes the step of tagging and storing cultural characteristic information regarding the person of the corresponding face for each face image, and the step of extracting the face image can extract a face image that matches the user's cultural characteristic information from the above face image database.

[0011] Priorities are set for each of the cultural characteristic elements of gender, race, age, personality, and occupation, and the step of extracting the face image may include the step of sequentially extracting candidates for face images that match the user's cultural specific information according to the priority of the cultural characteristic elements from the face image database.

[0012] The above avatar service method may further include the step of providing augmented reality content including the generated user avatar to a user terminal in accordance with the service application; and the step of obtaining user feedback information while providing the augmented reality content.

[0013] The above service application includes information related to the user's social life, and the information related to the social life includes information about the community to which the user belongs and information about people around the user within the community, and the above augmented reality content can be generated based on the information related to the user's social life.

[0014] The above avatar service method may further include the step of dividing the user feedback information into linguistic feedback information and non-linguistic feedback information; the step of converting the user's language in the linguistic feedback information into text to perform segmentation analysis and analyzing the tone and volume of the user's voice; the step of analyzing one or more of the user's gaze, facial expression, gesture, and posture in the non-linguistic feedback information; and the step of controlling one or more of the user avatar's speech, gesture, and movement according to the analysis results of the linguistic feedback information and the analysis results of the non-linguistic feedback information.

[0015] The augmented reality content includes one or more situation scenarios, and the avatar service method may further include the step of submitting a problem to a user according to the situation scenario in the augmented reality content; and the step of evaluating one or more of the user's empathy type and empathy ability based on one or more of the user's response to the submitted problem and user feedback information.

[0016] A computing device according to one disclosed embodiment comprises one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include: a command for building a face image database for face images that are free from portrait rights issues; a command for receiving a service request for augmented reality content from a user terminal; a command for extracting one face image from the face image database in accordance with the service request; and a command for creating a user avatar based on the extracted face image. Effects of the invention

[0018] According to the disclosed embodiment, a user avatar reflecting the cultural characteristics of the user can be created without issues related to portrait rights, and various services can be provided to the user through augmented reality content including the user avatar. Brief explanation of the drawing

[0020] FIG. 1 is a drawing showing an avatar service system in an extended reality space according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of a management server according to an embodiment of the present invention. FIG. 3 is a diagram schematically illustrating the process of an avatar generation module according to an embodiment of the present invention generating an avatar corresponding to a user. FIG. 4 is a diagram showing the configuration of an interaction module according to an embodiment of the present invention. FIG. 5 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments. Specific details for implementing the invention

[0021] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, this is merely illustrative and the present invention is not limited thereto.

[0022] In describing the embodiments of the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such detailed descriptions may unnecessarily obscure the essence of the present invention. Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe the embodiments of the present invention and should not be limiting in any way. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "include" or "comprise" are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.

[0023] Additionally, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms may be used for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0025] FIG. 1 is a diagram showing an avatar service system in an extended reality space according to one embodiment of the present invention.

[0026] Referring to FIG. 1, the avatar service system (100) may include a user terminal (102), a feedback measuring device (104), and a management server (106). The user terminal (102) and the feedback measuring device (104) may each be connected to communicate with the management server (106) via a communication network (150).

[0027] Here, the communication network (150) may include the Internet, one or more local area networks, wide area networks, cellular networks, mobile networks, other types of networks, or a combination of these networks.

[0028] In the disclosed embodiments, extended reality may refer to an environment equipped with a computer-generated virtual space, such as virtual reality, augmented reality, mixed reality, and holograms. In this case, the virtual space may consist only of virtual objects and backgrounds created separately from the actual reality space, or it may include objects or backgrounds from the actual reality space. Hereinafter, such a virtual space of extended reality may be referred to as an "extended reality space."

[0029] The user terminal (102) is a terminal of a user who intends to use an augmented reality-based service through an avatar (virtual human) in an augmented reality space.

[0030] In one embodiment, the augmented reality-based service may be a service such as psychological counseling for a user, empathy type diagnosis, or communication flow diagnosis. In such a service, the user can interact with and answer questions, etc., while using psychological counseling content or diagnostic content through an avatar.

[0031] A user terminal (102) can transmit a service request to a management server (106). The service request may include information on the type of service the user intends to use. Additionally, the service request may include information on the user's cultural characteristics (e.g., age, race, gender, occupation, personality, etc.). Additionally, the service request may include the user's physical information (height, weight, body shape, etc.). The user terminal (102) can receive augmented reality content from the management server (106) in accordance with the service request.

[0032] The feedback measuring device (104) can measure user feedback while augmented reality content is provided. Here, user feedback may include the user's gaze, facial expressions, voice, gestures, and posture. The feedback measuring device (104) can transmit user feedback information, including one or more of the user's gaze, facial expressions, voice, gestures, and posture, to a management server (106). The feedback measuring device (104) may be implemented by being separated into multiple devices to measure various user feedbacks. At this time, some functions or some configurations of the feedback measuring device (104) may be implemented in a user terminal (102).

[0033] In one embodiment, the feedback measuring device (104) can photograph the user while augmented reality content is provided and track the user's gaze based on the captured video of the user. Since the video-based gaze tracking technology is a known technology, a detailed description thereof will be omitted. Additionally, the feedback measuring device (104) can extract the user's facial expression (i.e., extract what facial expression the user is making) based on the captured video of the user. Additionally, the feedback measuring device (104) can extract the user's voice from the captured video of the user. Additionally, the feedback measuring device (104) can extract the user's gesture and posture (i.e., extract what gesture and posture the user is taking) based on the captured video of the user.

[0034] The management server (106) can provide augmented reality content to the user terminal (102) in accordance with the service request of the user terminal (102). In one embodiment, the management server (106) can provide augmented reality content to diagnose the user's empathy type and empathy ability in accordance with the user's service request, but is not limited thereto, and it is obvious that it can also provide augmented reality content for various services that can be implemented in the augmented reality space (e.g., psychological counseling or communication diagnostic services, etc.). The augmented reality content may be content that includes one or more avatars in the augmented reality space. The management server (106) can create an avatar corresponding to the user and operate the created avatar by interacting in the augmented reality space according to user feedback information.

[0035] FIG. 2 is a block diagram showing the configuration of a management server (106) according to an embodiment of the present invention. Referring to FIG. 2, the management server (106) may include an avatar creation module (111), a content provision module (113), an interaction module (115), and a diagnostic evaluation module (117).

[0036] The avatar generation module (111) can generate an avatar corresponding to a user based on the user's cultural characteristic information included in the service application. FIG. 3 is a schematic diagram showing the process of the avatar generation module (111) generating an avatar corresponding to a user according to an embodiment of the present invention.

[0037] Referring to FIG. 3, the avatar generation module (111) can build a face image database by collecting face images. Specifically, the avatar generation module (111) can collect person images from the internet through an image library or image crawling (Fig. 3 (a)).

[0038] For example, the avatar creation module (111) can collect images of people from a commercial image library or dataset by paying a certain amount. Additionally, the avatar creation module (111) can collect images of people by crawling images from the internet using search terms. The images of people collected in this way may be images that do not have issues related to portrait rights.

[0039] Next, the avatar generation module (111) can generate a face image by preprocessing the collected person images (Fig. 3(b)). That is, the avatar generation module (111) can generate a face image by extracting the face portion from the collected person images and resizing the face portion to a preset size. At this time, the avatar generation module (111) can tag and store preset cultural characteristic information (e.g., age, race, gender, occupation, and personality, etc.) on each collected face image.

[0040] Next, the avatar generation module (111) can extract a face image that matches the user's cultural characteristic information from the face image database ((c) of FIG. 3). The avatar generation module (111) can extract a face image that satisfies all elements of the user's cultural characteristics (user's age, race, gender, occupation, and personality, etc.) from the face image database.

[0041] Here, priorities can be assigned to each element of cultural characteristics. For example, among the elements of cultural characteristics, gender may be given the first priority, race the second priority, age the third priority, personality (e.g., MBTI, etc.) the fourth priority, and occupation the fifth priority.

[0042] The avatar generation module (111) can sequentially extract candidates for face images that match the user's cultural specific information according to priority among each element of cultural characteristics in the face image database.

[0043] For example, the avatar generation module (111) can extract a face image that matches the user's gender, which is the first priority, from the face image database as a first candidate image. Next, the avatar generation module (111) can extract a face image that matches the user's race, which is the second priority, from among the first candidate images as a second candidate image. Next, the avatar generation module (111) can extract a face image that matches the user's age, which is the third priority, from among the second candidate images as a third candidate image. Next, the avatar generation module (111) can extract a face image that matches the user's personality, which is the fourth priority, from among the third candidate images as a fourth candidate image. Next, the avatar generation module (111) can extract a face image that matches the user's occupation, which is the fifth priority, from among the fourth candidate images as a fifth candidate image. If there are multiple fifth candidate images, the avatar generation module (111) can randomly extract one image from the multiple fifth candidate images and set it as the avatar base image for the user.

[0044] Here, if there is no face image that matches the element of the next priority cultural characteristic, the avatar generation module (111) may randomly extract one image from among the candidate images that match the priority and set it as the basic avatar image for the user. For example, if there is no face image in the database that matches the fourth priority, the user's personality, after extracting up to the third candidate image, the avatar generation module (111) may randomly extract one image from among the third candidate images and set it as the basic avatar image for the user. The avatar generation module (111) can generate a user avatar (an avatar corresponding to the user) using the basic avatar image and the user's body information.

[0045] The avatar generation module (111) can generate a face of a user avatar based on an avatar base image (i.e., a face image) (Fig. 3 (d)). In one embodiment, the avatar generation module (111) can generate a face of a user avatar by inputting the avatar base image into a pre-trained deep learning model (111a). The deep learning model (111a) may be a generative artificial intelligence model trained to generate a corresponding avatar face based on the input face image.

[0046] The avatar generation module (111) can generate the face of the user avatar by rendering a basic avatar image and applying 3D modeling techniques such as 3D geometry and textures. At this time, the face of the user avatar reflects the cultural characteristics of the user without any portrait rights issues. The avatar generation module (111) can generate parts of the user avatar excluding the face based on the user's body information.

[0047] Additionally, the avatar creation module (111) can create avatars for people around the user. The avatar creation module (111) can create avatars for people around the user based on information related to the user's social life. Information related to the user's social life may include information about the community to which the user belongs and in which they engage in social life (e.g., school, workplace, etc.) and information about people around the user within that community (e.g., information about the gender, age, and relationship with the user of each person around the user). Information related to the user's social life may be included in the service application. The avatar creation module (111) may create avatars for people around the user in the same or similar manner as creating the user's avatar, but is not limited thereto.

[0048] The content provision module (113) can provide augmented reality content including a user avatar to the user terminal (102) in accordance with the service request of the user terminal (102). In one embodiment, the augmented reality content may be for diagnosing the user's empathy type and empathy ability.

[0049] In one embodiment, the content provision module (113) may generate a first to third situation scenario, respectively, based on information related to the user's social life. The first situation scenario may be a scenario that creates a situation to identify the user's cognitive empathy versus affective empathy. Cognitive empathy may refer to an individual's ability to understand and recognize the emotions and thoughts of others. Affective empathy may refer to an individual's ability to vicariously feel the emotional state of another person.

[0050] The first situation scenario may include a 1-1 situation scenario that creates a positive situation to identify the user's cognitive empathy versus emotional empathy, and a 1-2 situation scenario that creates a negative situation to identify the user's cognitive empathy versus emotional empathy.

[0051] In one embodiment, the 1-1 situation scenario may be a scenario in which the avatar of user's friend A (male) confesses his feelings to a woman and is accepted. The 1-2 situation scenario may be a scenario in which the avatar of user's friend B (male) confesses his feelings to a woman but is rejected.

[0052] The second situation scenario may be a scenario designed to stage a situation for identifying the user's positive empathy versus negative empathy. Positive empathy may refer to an individual's ability to understand and recognize the positive emotions of others. Negative empathy may refer to an individual's ability to understand and recognize the negative emotions of others.

[0053] The second situation scenario may include a 2-1 situation scenario that creates a positive situation to identify the user's positive empathy versus negative empathy, and a 2-2 situation scenario that creates a negative situation to identify the user's positive empathy versus negative empathy.

[0054] In one embodiment, the 2-1 situation scenario may be a scenario in which the avatar of the user's friend C wins a badminton tournament. The 2-2 situation scenario may be a scenario in which the avatar of the user's friend D is eliminated in the first round of a badminton tournament.

[0055] The third situation scenario may be a scenario designed to stage a situation to identify the user's majority empathy versus minority empathy. Majority empathy can refer to the ability to understand and recognize situations that are positive for the majority but negative for the minority. Minority empathy can refer to the ability to understand and recognize situations that are positive for the minority but negative for the majority.

[0056] The third situation scenario may include a 3-1 situation scenario that creates a majority positive situation and a minority negative situation to identify the user's majority versus minority empathy, and a 3-2 situation scenario that creates a majority negative situation and a minority positive situation to identify the user's majority versus minority empathy.

[0057] In one embodiment, the 3-1 situation scenario may be a scenario in which the avatar of E, a classmate of the user, receives birthday wishes from friends, while the avatar of F, a friend of the user, loses a mobile phone. The 3-2 situation scenario may be a scenario in which the soccer team of the user's class loses, while the avatar of G, a friend of the user, wins a 100m race.

[0058] The content provision module (113) can provide augmented reality content according to the first to third situation scenarios to the user terminal (102), respectively. Augmented reality content according to the first to third situation scenarios may be provided sequentially according to the user's selection, but is not limited thereto. Augmented reality content may include a user avatar, avatars of people around the user generated based on information related to the user's social life, and a background of the space where the user conducts social life.

[0059] Additionally, the content provision module (113) may submit problems for an empathy-related diagnosis to the user in the augmented reality content. The content provision module (113) may submit multiple problems to hear the user's opinion on the situation scenario while any one of the first situation scenario to the third situation scenario is provided to the user.

[0060] In one embodiment, the content providing module (113) may submit questions in the form of multiple choices. That is, it may submit questions so that the user selects from several examples (i.e., multiple choices) how they would answer or give an opinion in each situation scenario.

[0061] In one embodiment, the content providing module (113) may submit questions in the first-1 situation scenario and the first-2 situation scenario, such as “Please select from the list the emotion you felt after hearing A’s story,” “Please select from the list the emotion you felt after hearing B’s story,” “How much do you agree with A’s statement that she felt excited and happy because her confession was accepted?”, “How much do you agree with B’s statement that she felt depressed and sad as if she had confessed and been rejected?”, “When you heard the stories of A and B, whose story did you want to hear more?”, and “Whose emotion between A and B had more influence?”

[0062] Additionally, the content provision module (113) can present questions in the 2-1 and 2-2 situation scenarios such as, “Please select from the list the emotion you felt after hearing C’s story,” “How much do you agree with C’s statement that she felt excited and happy as if she had won first place in a competition?”, “Please select from the list the emotion you felt after hearing D’s story,” “How much do you agree with D’s statement that she felt depressed and sad as if she had lost in the first round?”, “You have a glass of water to give to a friend. To whom would you want to give it, C or D?”, “When you heard the stories of C and D, whose story did you want to hear more?”, and “Whose emotion did you have more influence on you, C or D?”

[0063] In addition, the content provision module (113) can submit questions such as, "Which did you experience more: the emotion of celebrating the birthday or the emotion of F in the 3-1 situation scenario?", "Which did you want to do more: helping F feel better or celebrating the birthday together?", "Which did you feel more: the feeling of pity for F or the feeling of being happy because there are students celebrating the birthday?"

[0064] Additionally, the content provision module (113) can submit questions in the 3-2 situation scenario, such as "Which did you experience more, G's emotions or the soccer team's emotions?", "Which did you want to do more, helping the soccer team feel better or celebrating G?", and "Which did you feel more, the feeling that the soccer team is pitiful or that G is in a good mood?"

[0065] The interactive module (115) can obtain a user's reaction while providing augmented reality content to the user. While providing augmented reality content to the user, the interactive module (115) can receive user feedback information from a feedback measuring device (104), including one or more of the user's gaze, facial expressions, voice, and gestures. The interactive module (115) can control a user avatar in augmented reality content based on the user feedback information.

[0066] FIG. 4 is a diagram showing the configuration of an interactive module (115) according to an embodiment of the present invention. Referring to FIG. 4, the interactive module (115) may include a feedback classification unit (121), a linguistic feedback analysis unit (123), a non-linguistic feedback analysis unit (125), and an interactive unit (127).

[0067] The feedback classification unit (121) can classify user feedback information into verbal feedback information and non-verbal feedback information. For example, verbal feedback information may include the user's voice. Verbal feedback information may include not only the user's voice responses to problems submitted in augmented reality content, but also the user's questions, opinions, and conversations with avatars while performing empathy-related diagnoses. Non-verbal feedback information may include the user's gaze, facial expressions, gestures, and posture while augmented reality content is provided.

[0068] The language feedback analysis unit (123) can convert the user's language feedback information into text and interpret the meaning of the text through syntactic analysis of the converted text. In addition, the language feedback analysis unit (123) can analyze the tone and volume of the user's voice from the user's language feedback information.

[0069] The nonverbal feedback analysis unit (123) can analyze the user's nonverbal feedback information. That is, the nonverbal feedback analysis unit (123) can analyze where the user's gaze is directed, what facial expression the user is making, what gesture the user is making, what posture the user is in, etc.

[0070] The interactive unit (127) can control the user avatar in the augmented reality content based on the user's verbal feedback information and non-verbal feedback information. That is, the interactive unit (127) can control the user avatar in the augmented reality content to reflect the user's feedback based on the user's verbal feedback information and non-verbal feedback information.

[0071] For example, the interactive unit (127) can synchronize the mouth shape of the user avatar with the user so that the user avatar speaks text according to the user's language feedback information. At this time, the interactive unit (127) can make the user avatar speak in response to the tone and volume of the user's voice.

[0072] Additionally, the interaction unit (127) can control the gestures and movements of the user avatar according to the user's non-verbal feedback information. The interaction unit (127) can control the gaze, facial expression, gesture, and posture of the user avatar in response to the user's gaze, facial expression, gesture, and posture.

[0073] Additionally, the interaction unit (127) may extract the user's emotions based on the user's verbal feedback information and non-verbal feedback information. The interaction unit (127) may allow the user's emotions to be projected onto the user's avatar. The diagnostic evaluation module (117) may evaluate one or more of the user's empathy type and empathy ability based on the user's response to a problem submitted in the augmented reality content.

[0074] Specifically, the diagnostic evaluation module (117) can measure the user's empathy type based on the answers to each problem submitted in the first to third situation scenarios. The diagnostic evaluation module (117) can determine the user's first detailed empathy type to third detailed empathy type, respectively, based on the answers to each problem submitted in the first to third situation scenarios.

[0075] In one embodiment, the diagnostic evaluation module (117) can determine the user's first detailed empathy type based on the answer to the problem submitted in the first situation scenario. The diagnostic evaluation module (117) can determine the first detailed empathy type indicating which empathy type the user is closer to, between cognitive empathy (C) and affective empathy (A), based on the answer to the problem submitted in the first situation scenario. If the user is closer to cognitive empathy than affective empathy, the user's first detailed empathy type is determined as cognitive empathy (C), and if the user is closer to affective empathy than cognitive empathy than affective empathy, the user's first detailed empathy type is determined as affective empathy (A).

[0076] The diagnostic evaluation module (117) can determine the user's second detailed empathy type based on the answer to the problem submitted in the second situation scenario. The diagnostic evaluation module (117) can determine the second detailed empathy type indicating which empathy type the user is closer to, positive empathy (P) or negative empathy (N), based on the answer to the problem submitted in the second situation scenario. If the user is closer to positive empathy than negative empathy, the user's second detailed empathy type is determined as positive empathy (P), and if the user is closer to negative empathy than positive empathy, the user's second detailed empathy type is determined as negative empathy (N).

[0077] The diagnostic evaluation module (117) can determine the user's third detailed empathy type based on the answer to the problem submitted in the third situation scenario. The diagnostic evaluation module (117) can determine the third detailed empathy type indicating which empathy type the user is closer to, between Majority empathy (M) and Minority empathy (m), based on the answer to the problem submitted in the third situation scenario. If the user is closer to Majority empathy than Minority empathy, the user's third detailed empathy type is determined as Majority empathy (M), and if the user is closer to Minority empathy than Minority empathy, the user's third detailed empathy type is determined as Minority empathy (m).

[0078] The diagnostic evaluation module (117) can detect the user's empathy type indicator by combining the user's first to third detailed empathy types. Here, since there are three detailed empathy types, there may be a total of eight empathy type indicators. The diagnostic evaluation module (117) can detect one of eight empathy type indicators as the user's empathy type indicator by taking the first letters of the English letters of the user's first to third detailed empathy types: CPm, CPM, CNm, CNM, APm, APM, ANm, and ANM. For example, the empathy type indicator CPm represents the case where the first detailed empathy type is cognitive empathy (C), the second detailed empathy type is positive empathy (P), and the third detailed empathy type is minority empathy (m).

[0079] The diagnostic evaluation module (117) can provide empathy type information for the user to the user terminal (102). Here, the empathy type information may include the user's empathy type indicator and related information regarding the empathy type indicator (e.g., supplementary description, type brief information, and character, etc.).

[0080] Additionally, the diagnostic evaluation module (117) can measure the level of empathy of the user based on user feedback information (verbal feedback information and non-verbal feedback information) obtained while providing augmented reality content.

[0081] The diagnostic evaluation module (117) can measure the user's first level of empathy based on the user's language feedback information. The diagnostic evaluation module (117) can interpret the meaning of the text through syntactic analysis of the text in the user's language feedback information. Additionally, the diagnostic evaluation module (117) can analyze the tone and volume of the user's voice in the user's language feedback information. The diagnostic evaluation module (117) can measure the user's first level of empathy through syntactic analysis of the text and analysis of the tone and volume of the user's voice.

[0082] In one embodiment, the diagnostic evaluation module (117) can calculate the user's concentration score, clarification score, help score, support score, and encouragement score, respectively, through parsing the text and analyzing the tone and volume of the user's voice. The diagnostic evaluation module (117) can calculate the user's first empathy level by summing the user's concentration score, clarification score, help score, support score, and encouragement score.

[0083] Here, the focus score may refer to a score based on the degree of concentration in listening to the other person. The clarification score may refer to a score based on confirming questions to understand the other person's thoughts and feelings. The help score may refer to a score based on presenting solutions or relevant information to help the other person view the problem situation objectively. The support score may refer to a score based on agreeing with or affirming the other person's emotions. The encouragement score may refer to a score based on boosting courage or motivation to alleviate the other person's distress or worry.

[0084] The diagnostic evaluation module (117) can measure the user's second empathy level based on the user's non-verbal feedback information. The diagnostic evaluation module (117) can calculate a first empathy expression score based on the user's gaze, a second empathy expression score based on the user's facial expression, a third empathy expression score based on the user's gesture, and a fourth empathy expression score based on the user's posture. The diagnostic evaluation module (117) can calculate the user's second empathy level by summing the first to fourth empathy expression scores.

[0085] The diagnostic evaluation module (117) can calculate a first empathy expression score based on the degree and duration of the user's gaze toward the other party. The diagnostic evaluation module (117) can calculate a second empathy expression score based on the user's facial expression during conversation with the other party or when answering a question. The diagnostic evaluation module (117) can calculate a third empathy expression score and a fourth empathy expression score, respectively, based on the gesture and posture sent toward the other party during conversation with the other party or when answering a question.

[0086] The diagnostic evaluation module (117) can calculate the user's empathy index by combining the user's first empathy level and second empathy level. In one embodiment, the diagnostic evaluation module (117) can calculate the user's empathy index by assigning a first weight to the user's first empathy level, assigning a second weight to the user's second empathy level, and then summing them. The diagnostic evaluation module (117) can provide information related to the user's empathy to the user terminal (102). The information related to empathy may include information such as the user's first empathy level, second empathy level, and empathy index.

[0087] According to the disclosed embodiment, a user avatar reflecting the cultural characteristics of the user can be created without issues related to portrait rights, and various services can be provided to the user through augmented reality content including the user avatar.

[0088] In this specification, the term "module" may refer to a functional and structural combination of hardware for carrying out the technical concept of the present invention and software for driving said hardware. For example, the "module" may refer to a logical unit of a specific code and a hardware resource for executing said code, and does not necessarily refer to physically connected code or a single type of hardware.

[0090] FIG. 5 is a block diagram illustrating a computing environment (10) including a computing device suitable for use in exemplary embodiments. In the illustrated embodiments, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

[0091] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a user terminal (102). Additionally, the computing device (12) may be a feedback measurement device (104). Additionally, the computing device (12) may be a management server (106).

[0092] The computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) can cause the computing device (12) to operate according to the exemplary embodiment described above. For example, the processor (14) can execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause the computing device (12) to perform operations according to the exemplary embodiment when executed by the processor (14).

[0093] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by a processor (14). In one embodiment, the computer-readable storage medium (16) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other forms of storage media that are accessed by a computing device (12) and capable of storing desired information, or a suitable combination thereof.

[0094] The communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and the computer-readable storage medium (16).

[0095] The computing device (12) may also include one or more input / output interfaces (22) and one or more network communication interfaces (26) that provide interfaces for one or more input / output devices (24). The input / output interfaces (22) and network communication interfaces (26) are connected to a communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) through the input / output interfaces (22). An exemplary input / output device (24) may include an input device such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices and / or imaging devices, and / or an output device such as a display device, a printer, a speaker and / or a network card. An exemplary input / output device (24) may be included inside the computing device (12) as a component constituting the computing device (12), or it may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0097] Although representative embodiments of the present invention have been described in detail above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof. Explanation of the symbols

[0099] 10: Computing Environment 12: Computing device 14 : Processor 16: Computer-readable storage media 18: Communication bus 20 : Program 22 : Input / Output Interface 24 : Input / Output Devices 26: Network communication interface 100: Virtual Human Interaction System 102 : User terminal 104 : Feedback measuring device 106 : Management Server 111 : Avatar Creation Module 113 : Content Provision Module 115: Interaction Module 117: Diagnostic Evaluation Module 121 : Feedback Classification Unit 123 : Language Feedback Analysis Department 125 : Non-verbal Feedback Analysis Unit 127 : Interactive part

Claims

Claim 1 A method performed on a computing device having one or more processors and a memory for storing one or more programs executed by said one or more processors, comprising the steps of: constructing a face image database for face images that are free from issues related to portrait rights, wherein each face image is tagged with cultural characteristic information regarding the person of the corresponding face and stored; receiving a service request for augmented reality content from a user terminal; extracting one face image from the face image database in accordance with the service request; and generating a user avatar based on the extracted face image, wherein the service request includes cultural characteristic information regarding one or more of the user's gender, race, age, personality, and occupation, and each of the cultural characteristic elements of gender, race, age, personality, and occupation is assigned a priority, and the step of extracting the face image includes sequentially extracting candidates for face images that match the user's cultural characteristic information from the face image database according to the priority of the cultural characteristic elements. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 The avatar service method of claim 1, wherein the avatar service method further comprises: a step of providing augmented reality content including the generated user avatar to a user terminal in accordance with the service application; and a step of obtaining user feedback information while providing the augmented reality content. Claim 6 In claim 5, the service application includes information related to the user's social life, the information related to the social life includes information about the community to which the user belongs and information about people around the user within the community, and the augmented reality content is generated based on the information related to the user's social life, an avatar service method. Claim 7 The avatar service method of claim 6 further comprises: a step of dividing user feedback information into linguistic feedback information and non-linguistic feedback information; a step of converting the user's language in the linguistic feedback information into text to perform segmentation analysis and analyzing the tone and volume of the user's voice; a step of analyzing one or more of the user's gaze, facial expression, gesture, and posture in the non-linguistic feedback information; and a step of controlling one or more of the user avatar's speech, gesture, and movement according to the analysis result of the linguistic feedback information and the analysis result of the non-linguistic feedback information. Claim 8 The avatar service method according to claim 6, wherein the augmented reality content comprises one or more situation scenarios, and the avatar service method further comprises the steps of: submitting a problem to a user according to the situation scenario in the augmented reality content; and evaluating one or more of the user's empathy type and empathy ability based on one or more of the user's answer to the submitted problem and user feedback information. Claim 9 A computing device comprising one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and wherein the one or more programs include: a command to build a face image database for face images that are free from portrait rights issues, wherein each face image is tagged with and stored cultural characteristic information regarding the person of the corresponding face; a command to receive a service request for augmented reality content from a user terminal; a command to extract one face image from the face image database in accordance with the service request; and a command to create a user avatar based on the extracted face image, wherein the service request includes cultural characteristic information regarding one or more of the user's gender, race, age, personality, and occupation, wherein a priority is set for each of the cultural characteristic elements of gender, race, age, personality, and occupation, and the command to extract the face image includes a command to sequentially extract candidate face images that match the user's cultural characteristic information from the face image database according to the priority of the cultural characteristic elements. Claim 10 delete Claim 11 delete Claim 12 delete Claim 13 A computing device according to claim 9, wherein the one or more programs further comprise: a command for providing augmented reality content including the generated user avatar to a user terminal in accordance with the service application; and a command for obtaining user feedback information while providing the augmented reality content. Claim 14 A computing device according to claim 13, wherein the service application includes information related to the user's social life, the information related to the social life includes information about the community to which the user belongs and information about people around the user within the community, and the augmented reality content is generated based on the information related to the user's social life. Claim 15 A computing device according to claim 14, wherein the one or more programs further comprise: a command for distinguishing user feedback information into linguistic feedback information and non-linguistic feedback information; a command for converting the user's language in the linguistic feedback information into text to perform segmentation analysis and analyzing the tone and volume of the user's voice; a command for analyzing one or more of the user's gaze, facial expression, gesture, and posture in the non-linguistic feedback information; and a command for controlling one or more of the user avatar's speech, gesture, and movement according to the analysis result of the linguistic feedback information and the analysis result of the non-linguistic feedback information. Claim 16 A computing device according to claim 14, wherein the augmented reality content comprises one or more situation scenarios, and the one or more programs further comprise: a command for submitting a problem to a user according to the situation scenario in the augmented reality content; and a command for evaluating one or more of the user's empathy type and empathy ability based on one or more of the user's answer to the submitted problem and user feedback information. Claim 17 A computer program stored on a non-transitory computer-readable storage medium, comprising the steps of: constructing a database of face images that are free from issues related to portrait rights, wherein each face image is tagged with cultural characteristic information regarding the person of the corresponding face and stored; receiving a service request for augmented reality content from a user terminal; extracting one face image from the face image database in accordance with the service request; and generating a user avatar based on the extracted face image, wherein the service request includes cultural characteristic information regarding one or more of the user's gender, race, age, personality, and occupation, and each of the cultural characteristic elements of gender, race, age, personality, and occupation is assigned a priority, and the step of extracting the face image includes the step of sequentially extracting candidate face images that match the user's cultural characteristic information from the face image database according to the priority of the cultural characteristic elements.

Images