Personal Advice Providing Device and Personal Advice Providing System

Abstract

In a personal advice providing device, the expressions and / or voices of an avatar that provides advice are automatically optimized for each user to enhance the mental distance and emotional intimacy with the user. 【Solution means】The personal advice providing device 1 includes a photographing unit 10, an emotional state determination unit 41, a DNA information acquisition unit 20, an auxiliary information acquisition unit 30, an advice generation unit 42 that generates advice using DNA information and auxiliary information, an avatar image generation unit 43 that generates an avatar image capable of variably displaying expressions and / or voices, and an advice output unit 50 that outputs advice via the avatar image. In the emotional state determination unit 41, the emotional state score is determined by correcting the primary emotional state score obtained by the image analysis process of the expression image with the DNA information. In the avatar image generation unit 43, the expressions and / or voices of the avatar image are adjusted based on the emotional state score.

Description

Technical Field

[0001] The present invention relates to a "personal advice providing device" and a "personal advice providing system". More specifically, the present invention relates to a "personal advice providing device" and a "personal advice providing system" that provide advice in various fields contributing to improving the quality of life of individual users, and that personalize the advice for individual users.

Background Art

[0002] Since ancient times, humans have sought various types of advice from others (teachers, friends, fortune-tellers, consultants, etc.) in various fields as a means of problem-solving. And in modern times, various advice-providing systems that utilize conversation-type computer programs using artificial intelligence (AI) are increasingly being widely used as one of such means of problem-solving.

[0003] For example, in the medical and health fields, various devices have been developed that utilize user-specific DNA information as basic information to generate personalized health advice for users and provide the advice to the users (Patent Documents 1 and 2). The inventors of the present application have come to consider such devices that utilize DNA information as personal advice providing devices that provide advice in various fields contributing to improving the quality of life of individual users, not limited to the medical field.

[0004] And in such a highly versatile personal advice providing device, it is considered more desirable to provide personalized various types of advice in a state where the mental distance and emotional intimacy with the user are enhanced by providing the advice via an avatar optimized for each user. However, technical means for automatically optimizing the facial expression and / or voice (emotional expression) of the avatar that provides the advice, which is necessary to achieve this, has not yet existed.

Prior Art Documents

Patent Document

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] The present invention aims to automatically optimize the expression and / or voice (mode of pseudo-emotional expression) of an avatar that provides advice for each user in a highly versatile personal advice providing device that provides advice contributing to improving the quality of life of individual users, thereby enhancing the mental distance and emotional intimacy with the user.

Means for Solving the Problems

[0007] The inventors of the present invention conceived that the above problems can be solved by adjusting the expression and / or voice of the avatar image to the optimal expression and / or voice based on the corrected user emotional state score obtained by correcting the primary emotional state score of the user read from the user's expression with the user-specific DNA information when providing personalized advice to the user from the user-specific DNA information, and thus completed the present invention.

[0008] (1) A photographing unit, a DNA information acquisition unit that acquires DNA information unique to a user, an auxiliary information acquisition unit that acquires auxiliary information that is the user-unique biometric information and / or environmental information other than the DNA information, an emotional state determination unit that determines the emotional state of the user from the facial expression image of the user photographed by the photographing unit, an advice generation unit that generates advice for the user by arithmetic processing using the DNA information and the auxiliary information as input values, an avatar image generation unit that generates an avatar image capable of variably displaying expressions and / or voices corresponding to a plurality of emotions, and an advice output unit that outputs the advice via the avatar image. In the emotional state determination unit, an emotional state score is determined by correcting the primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information. In the avatar image generation unit, based on the emotional state score, the expression and / or voice of the avatar image is selected. A personal advice providing device.

[0009] According to the personal advice providing device of (1), in a highly versatile personal advice providing device that provides advice in various fields contributing to improving the quality of life of individual users by using DNA diagnosis results, the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby enhancing the mental distance and emotional intimacy with the user.

[0010] (2) In the avatar image generation unit, the expression and / or voice of the avatar image selected based on the emotional state score is adjusted according to the content of the advice, in the personal advice providing device according to (1).

[0011] According to the personal advice providing device of (2), in the personal advice providing device of (1), the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be optimized in consideration of the content of the advice. Thereby, the mental distance and emotional intimacy with the user can be further enhanced.

[0012] (3) In the emotional state determination unit, the amount of change in the emotional state score at the time points before and after the advice is output is extracted, and in the avatar image generation unit, the expression and / or voice of the avatar image are reselected based on the amount of change in the personal advice providing device according to (1) or (2).

[0013] According to the personal advice providing device of (3), in the personal advice providing device of (1) or (2), the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be optimized over time by fully considering the real-time changes in the emotional state of each user. Therefore, this enables a highly empathic communication that feels more similar to real human communication, and can greatly enhance the mental distance and emotional intimacy with the user to an extremely high level.

[0014] (4) A personal advice providing system comprising a photographing unit, a DNA information acquisition unit for acquiring the user-specific DNA information, an auxiliary information acquisition unit for acquiring auxiliary information which is the user-specific biological information and / or environmental information other than the DNA information, an emotional state determination unit for determining the emotional state of the user from the expression image of the user photographed by the photographing unit, an advice generation unit for generating advice for the user by arithmetic processing using the DNA information and the auxiliary information as input values, an avatar image generation unit for variably generating an avatar image capable of displaying expressions and / or voices corresponding to a plurality of emotions, and an advice output unit for outputting the advice via the avatar image. In the emotional state determination unit, the emotional state score is determined by correcting the primary emotional state score obtained by the image analysis process of the expression image with the DNA information, and in the avatar image generation unit, the expression and / or voice of the avatar image are selected based on the emotional state score.

[0015] According to the personal advice providing system of (4), in a highly versatile personal advice providing system that provides advice in various fields contributing to improving the quality of life of individual users by using DNA diagnosis results, the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, enhancing the mental distance and emotional intimacy with the user.

[0016] (5) In the avatar image generation unit, the personal advice providing system according to (4), wherein the expression and / or voice of the avatar image selected based on the emotional state score is adjusted according to the content of the advice.

[0017] According to the personal advice providing system of (5), in the personal advice providing system of (4), the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be optimized in consideration of the content of the advice. Thereby, the mental distance and emotional intimacy with the user can be further enhanced.

[0018] (6) In the emotional state determination unit, the amount of change in the emotional state score at points in time before and after the time when the advice is output is extracted, and in the avatar image generation unit, the expression and / or voice of the avatar image is reselected based on the amount of change, in the personal advice providing system according to (4) or (5).

[0019] According to the personal advice providing system of (6), in the personal advice providing system of (4) or (5), the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be optimized over time, fully considering the real-time changes in the emotional state of each user. Therefore, by this, it is possible to conduct highly empathetic communication that feels more approximated to communication with a real person, and thus the mental distance and emotional intimacy with the user can be enhanced to an extremely high level.

[0020] (7) In the personal advice providing system according to (4) or (5), in the emotional state determination unit, a process of determining the emotional state score by correcting the primary emotional state score obtained by the image analysis process of the facial expression image with the DNA information, and in the avatar image generation unit, a process of selecting the facial expression and / or voice of the avatar image based on the emotional state score. A program for a personal advice providing system that executes the above processes.

[0021] According to the program for the personal advice providing system of (7), in a highly versatile personal advice providing system that provides advice in various fields contributing to improving the quality of life of individual users by using DNA diagnosis results, the facial expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby enhancing the mental distance and emotional intimacy with the user.

[0022] (8) A personal advice providing system comprising: a photographing unit; a DNA information acquisition unit for acquiring DNA information unique to a user; an auxiliary information acquisition unit for acquiring auxiliary information which is the user-unique biological information and / or environmental information other than the DNA information; an emotional state determination unit for determining the emotional state of the user from a facial expression image of the user photographed by the photographing unit; an advice generation unit for generating advice for the user by arithmetic processing using the DNA information and the auxiliary information as input values; a robot control unit for generating an emotion designation signal for designating an output mode of facial expression and / or voice for a robot capable of variably outputting facial expressions and / or voices corresponding to a plurality of emotions; and an advice output unit composed of the robot. In the emotional state determination unit, the emotional state score is determined by correcting the primary emotional state score obtained by the image analysis process of the facial expression image with the DNA information. In the robot control unit, the emotion designation signal is selected based on the emotional state score.

[0023] According to the personal advice providing system of (8), in a highly versatile personal advice providing device that provides advice in various fields contributing to improving the quality of life of individual users using DNA diagnosis results, the expression and / or voice (mode of pseudo-emotional expression) of the robot providing the advice can be automatically optimized for each user, enhancing the mental distance and emotional intimacy with the user.

[0024] (9) In the robot control unit, the personal advice providing system according to (8), wherein the emotion designation signal selected based on the emotion state score is adjusted according to the content of the advice.

[0025] (9) According to the personal advice providing system of (9), in the personal advice providing system of (8), the expression and / or voice (mode of pseudo-emotional expression) of the robot providing the advice can be optimized in consideration of the content of the advice. Thereby, the mental distance and emotional intimacy with the user can be further enhanced.

[0026] (10) In the emotion state determination unit, the amount of change in the emotion state score at the time points before and after the time when the advice is output is extracted, and in the robot control unit, the emotion designation signal is reselected based on the amount of change, in the personal advice system according to (8) or (9).

[0027] (10) According to the personal advice providing system of (10), in the personal advice providing system of (4) or (5), the expression and / or voice (mode of pseudo-emotional expression) of the robot providing the advice can be optimized over time, fully considering the real-time changes in the emotion state of each user. Therefore, by this, it is possible to conduct highly empathetic communication that feels more similar to communication with a real human being, and thus the mental distance and emotional intimacy with the user can be enhanced to an extremely high level.

[0028] In the personal advice providing system according to (11), (8) or (9), in the emotional state determination unit, a process of determining the emotional state score by correcting the primary emotional state score obtained by the image analysis process of the facial expression image with the DNA information, and in the robot control unit, a process of selecting the emotional designation signal based on the emotional state score. A program for a personal advice providing system that causes the system to execute these processes.

[0029] According to the program for the personal advice providing system of (11), in a highly versatile personal advice providing system that provides advice in various fields contributing to improving the quality of life of individual users by using DNA diagnosis results, the facial expression and / or voice (mode of pseudo-emotional expression) of the robot providing the advice can be automatically optimized for each user to increase the mental distance and emotional intimacy with the user. The facial expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user to increase the mental distance and emotional intimacy with the user.

[0030] (12) A facial photography step in which a photography unit photographs a facial image of a user; a DNA information acquisition step in which a DNA information acquisition unit acquires the user-specific DNA information; an auxiliary information acquisition step in which an auxiliary information acquisition unit acquires auxiliary information which is the user-specific biological information and / or environmental information other than the DNA information; an emotional state determination step in which an emotional state determination unit determines the emotional state of the user from the facial image; an advice generation step in which an advice generation unit generates advice for the user by arithmetic processing using the DNA information and the auxiliary information as input values; an avatar image generation step in which an avatar image generation unit generates an avatar image capable of variably displaying expressions and / or voices corresponding to a plurality of emotions; and an advice output step in which an advice output unit outputs the advice via the avatar image. In the emotional state determination step, the emotional state score is determined by correcting the primary emotional state score obtained by image analysis processing of the facial image with the DNA information. In the avatar image generation step, the expression and / or voice of the avatar image are selected based on the emotional state score. A personal advice providing method.

[0031] According to the personal advice providing method of (12), when providing highly versatile personalized advice provided by an information processing device or an information processing system, which contributes to improving the quality of life of individual users by using DNA diagnosis results in various fields, the expressions and / or voices (modes of pseudo-emotional expressions) of the avatar providing the advice can be automatically optimized for each user, thereby enhancing the mental distance and emotional intimacy with the user.

[0032] (13) The personal advice providing method according to (12), wherein in the avatar image generation step, the expressions and / or voices of the avatar image selected based on the emotional state score are adjusted according to the content of the advice.

[0033] According to the personal advice providing method of (13), in the personal advice providing method of (12), the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be optimized in consideration of the content of the advice. Thereby, the mental distance from the user and the intimacy in the emotional aspect can be further enhanced.

[0034] (14) In the emotional state determination step, the amount of change in the emotional state score at the time points before and after the time when the advice is output is extracted, and in the avatar image generation step, the expression and / or voice of the avatar image are reselected based on the amount of change, the personal advice providing method according to (12) or (13).

[0035] (14) According to the personal advice providing method of (14), in the personal advice providing method of (12) or (13), the avatar providing the advice can fully consider the real-time change in the emotional state of the user, and optimize the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice for each user over time. Therefore, thereby, it is possible to perform highly empathetic communication that feels more approximate to communication with a real person, so that the mental distance from the user and the intimacy in the emotional aspect can be enhanced to an extremely high level.

Effect of the Invention

[0036] According to the present invention, in a highly versatile personal advice providing device that provides advice in various fields contributing to improving the quality of life of individual users by using DNA diagnosis results, the expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user to enhance the mental distance from the user and the intimacy in the emotional aspect.

Brief Description of the Drawings

[0037]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0038] <Personal Advice Providing Apparatus / Personal Advice Providing System> Hereinafter, the best mode for carrying out the present invention will be described with appropriate reference to the drawings. The "personal advice providing apparatus" and "personal advice providing system" of the present invention are advice in various fields that contribute to improving the quality of life of individual users, and are information processing apparatuses or information processing systems that provide personalized advice for individual users.

[0039] The range of target fields for which advice can be provided is not particularly limited, but specific examples of target fields to which the "personal advice providing apparatus" and "personal advice providing system" of the present invention can be preferably applied include the respective fields shown in Table 1 below.

[0040]

Table 1

[0041] As an example, when providing advice on career and future path, conventionally, in addition to the age and gender of the user which are biological information, only the user-specific acquired environmental information such as school age and hobbies has been exclusively used as the input value to the variables in the function for advice generation. However, in the present application, the temperament and constitution of the user that can be derived from DNA information are also actively used as the above-mentioned input values for generating advice. As a result, even when the user himself / herself is not aware of his / her genetic constitution and temperament, these can be taken into account to generate personalized advice for the user from a perspective different from the conventional one.

[0042] [Overall Configuration] An example of a preferred embodiment of the "Personal Advice Providing Device" of the present invention, the basic configuration of the Personal Advice Providing Device 1 is as shown in the block diagram of FIG. 1. The Personal Advice Providing Device 1 includes a photographing unit 10, a DNA information acquisition unit 20, an auxiliary information acquisition unit 30, an arithmetic processing unit 40, and an advice output unit 50. And the arithmetic processing unit 40 includes an emotional state determination unit 41, an advice generation unit 42, and an avatar image generation unit 43.

[0043] The Personal Advice Providing Device 1 is an information processing device in which the photographing unit 10, the DNA information acquisition unit 20, the auxiliary information acquisition unit 30, the arithmetic processing unit 40, and the advice output unit 50 are integrated.

[0044] Further, the present invention can be implemented as a distributed information processing system, that is, a "Personal Advice Providing System", which can exhibit the same operational effects as the Personal Advice Providing Device 1 by dispersing and arranging all or part of each constituent element of the Personal Advice Providing Device 1 shown in FIG. 1 at any appropriate location separated from other constituent elements. When the present invention is configured as such a distributed system, the above-mentioned respective units may be communicably connected by various communication means such as wired connection using a dedicated communication cable, wired LAN, wireless LAN, short-range wireless communication, or mobile phone line.

[0045] As another embodiment of the present invention, in the above-described "personal advice providing system", the advice output unit 50 is configured by a robot capable of variably outputting expressions and / or voices corresponding to a plurality of emotions, and the arithmetic processing unit 40 includes, instead of the avatar image generation unit 43, a robot control unit that generates an emotion designation signal for designating the output mode of the expression and / or voice of this robot. It is also possible to adopt an embodiment in which the configuration is partially changed to be provided.

[0046] [Shooting unit] The shooting unit 10 can be configured by various cameras capable of shooting an image of the expression of the user of the personal advice providing device 1 (referred to as "expression image" in this specification). The camera constituting the shooting unit 10 only needs to have a function of outputting the captured "expression image" of the user as digital image data so that it can be used for arithmetic processing related to image analysis in the emotion state determination unit 41. Specifically, a digital camera having various existing moving image shooting functions can be used without particular limitation as the camera constituting the shooting unit 10.

[0047] The shooting unit 10 is arranged at a position where it can continuously shoot the "expression image" of the user when the user uses the personal advice providing device 1. For this purpose, for example, as shown in FIGS. 3 and 4, in the personal advice providing device 1, the shooting unit 10 is preferably configured to be joined or incorporated into the advice output unit 50 as a small camera. Further, when the advice output unit 50 is configured by a laptop personal computer or various portable information terminals, a built-in small camera built into these PCs or the like so as to be able to shoot in front of the information display screen can also be configured to function as the shooting unit 10.

[0048] [DNA information acquisition unit] The DNA information acquisition unit 20 can be configured by an information processing apparatus having a function of acquiring user-specific DNA information as data used for generating advice for the user and for appropriately correcting and determining the user's emotional state score. More specifically, the DNA information acquisition unit 20 acquires and temporarily stores "user-specific DNA information", and it suffices to be an information processing apparatus having a function of supplying this information to the arithmetic processing unit 40 that constitutes the personal advice providing apparatus 1 or the personal advice supply system, and it is not limited to a specific information processing apparatus.

[0049] The personal advice providing apparatus 1 preferably has a configuration further including a DNA information processing unit. The advice generation unit 42 and the avatar image generation unit 43 are preferably configured by a machine learning type information processing apparatus having a neural network or other artificial intelligence (AI), but the DNA information processing unit refers to an information processing apparatus having a function of analyzing the user-specific DNA information input to the DNA information acquisition unit 20 and processing it into a data format and content that can be used for predetermined calculations in the above information processing apparatus and the like. Here, the analysis of DNA information specifically includes, for example, SNP analysis, CNV (Copy Number Variation) analysis, microsatellite analysis, epigenetic DNA analysis, or gene expression level analysis, etc.

[0050] [Auxiliary information acquisition unit] The auxiliary information acquisition unit 30 can be configured by an information processing apparatus having a function of acquiring auxiliary information as data used for generating advice for the user in combination with the above DNA information and for determining the user's emotional state score in combination with the above DNA information. Here, the auxiliary information is information related to the user other than the above DNA information, and refers to user-specific biological information and / or environmental information, etc.

[0051] The user-specific biological information includes, for example, height, weight, electrocardiogram waveform, autonomic nerve activity level, respiratory frequency, walking distance, sleep duration, respiratory rate, blood oxygen saturation, blood glucose level, blood pressure, heart rate, etc. These biological information are data that can be measured by various biological sensors (measuring instruments). The auxiliary information acquisition unit 30 is preferably configured to include the above various biological sensors for acquiring these biological information. As an example, among the above biological information, for the electrocardiogram waveform, autonomic nerve activity level, and respiratory frequency, they can be measured by using an electrocardiograph as a biological sensor. Also, for the walking distance, sleep duration, and respiratory rate, they can be measured by an acceleration sensor. Furthermore, for the blood oxygen saturation, blood glucose level, blood pressure, heart rate, etc., they can be measured by an optical sensor.

[0052] Also, as other examples of the user-specific biological information, date of birth (age), gender, disease history of the user and immediate family members, etc. can be cited. Regarding these information, they are data that can be made available when the user subjectively inputs the information directly, or by having a recording medium or the like on which personal information is recorded read by a digital data reading device such as various card readers. The auxiliary information acquisition unit 30 is preferably configured to include the above various information input devices for acquiring these biological information, specifically, the above digital data reading device, or a microphone for collecting the user's voice, a touch panel type information input device capable of inputting character information, etc.

[0053] On the other hand, the user-specific environmental information includes, for example, diet, social status, income, family composition, geographical environment of the place of residence, etc. These information are also data that can be made available when the user subjectively inputs the information directly, or by having a recording medium or the like on which personal information is recorded read by a reading device. The auxiliary information acquisition unit 30 is preferably configured to include the above various information input devices for acquiring these environmental information.

[0054] [Arithmetic processing unit] The arithmetic processing unit 40 includes, as essential components, an emotional state determination unit 41 that determines the emotional state of the user, an advice generation unit 42 that generates advice for the user, and an avatar image generation unit 43 that generates an avatar image. As described above, it is also possible to adopt a configuration in which a robot control unit is provided instead of the avatar image generation unit 43. Details of each sub-configuration will be described later.

[0055] The arithmetic processing unit 40 can be configured as a dedicated information processing device specialized for specific operations according to the implementation of the present invention. Alternatively, the arithmetic processing unit 40 can also be configured using a personal computer, a tablet terminal, various portable information processing terminals, and the like. Further, the arithmetic processing unit 40 can be configured as an information processing device that can operate as an emotional state determination unit 41, an advice generation unit 42 that generates advice for the user, and an avatar image generation unit 43 that generates an avatar image on the cloud as a server and shared by a number of clients.

[0056] In any of the above configurations, the arithmetic processing unit 40 can be configured by various known information processing devices such as a computer including hardware such as a CPU, a memory, and a communication unit. And the arithmetic processing unit 40 having the above configuration can specifically execute various operations of the personal advice providing device 1 by the "program for personal advice providing system" of the present invention.

[0057] (Emotional State Determination Unit) The emotional state determination unit 41 can be configured by an information processing device having a function of determining the emotional state of the user (for example, a score value composed of the distinction of each emotional state such as joy, anger, surprise, sadness, etc. and the intensity of each distinguished emotional state) using the facial expression image of the user captured by the imaging unit 10.

[0058] Here, the determination of the user's emotional state by the emotional state determination unit 41 in the personal advice providing apparatus 1 first involves the first-stage process of analyzing the user's facial expression image through image analysis. Subsequently, the second-stage process of determining the emotional state score is performed by correcting the emotional state score (primary emotional state score) obtained through the same process using the user-specific DNA information and other auxiliary information related to the user.

[0059] Therefore, the emotional state determination unit 41 needs to be composed of various information processing apparatuses capable of performing the above two-stage processes. For this purpose, it is assumed that the information processing apparatus particularly includes image analysis means for executing the first-stage process of analyzing the facial expression image to obtain the primary emotional state score. The image analysis means for determining the emotional state of a person by analyzing the image of the person's face and obtaining the emotional state score (primary emotional state score) can be configured using a machine learning type information processing apparatus having a neural network (particularly, a "deep learning type image recognition apparatus"), or various other artificial intelligences (AI). It is also possible to use various artificial intelligence systems on the cloud as such artificial intelligence (AI), etc. Specific examples of image recognition technology using deep learning are publicly disclosed below. "Deep Learning and Image Recognition, Operations Research" (http: / / www.orsj.o.jp / archive2 / or60-4 / or60_4_198.pdf)

[0060] Furthermore, regarding the technology of reading the emotional state of a person from the person's image using the image recognition technology of the above-described machine learning type information processing apparatus having a neural network, etc., as disclosed in, for example, "Japanese Patent Application Laid-Open No. 2012-59107", "Japanese Patent Application Laid-Open No. 2022-114789", etc., a number of specific technical means are already known. Also in the personal advice providing apparatus 1 of the present invention, the emotional state determination unit 41 can be configured using these known various technical means.

[0061] Also, regarding an information processing apparatus that performs a second-stage process of determining an emotional state score by correcting the "primary emotional state score" obtained in the above-described first-stage process using the user-specific DNA information and other auxiliary information related to the user, it is preferable to configure it using a machine learning-type information processing apparatus having a neural network or various other artificial intelligences (AIs). Thereby, in determining the emotional state score, by contributing the past learning results, the accuracy of the determination of the emotional state can be further improved, and the compatibility of the finally output emotional state score with the user can be improved to a higher level.

[0062] (Advice Generation Unit) The advice generation unit 42 can be configured by an information processing apparatus having a function of generating advice for the user (advice personalized for each user) output from the advice output unit 50 through arithmetic processing using the user-specific DNA information and other auxiliary information related to the user as input values.

[0063] Here, the generation of advice for the user by the advice generation unit 42 in the personal advice providing apparatus 1 first involves performing a first-stage arithmetic process of obtaining advice (primary advice) through the above-described arithmetic processing using the user-specific DNA information or the like as input values. However, thereafter, it is also possible to adopt a procedure of generating corrected advice corrected according to the user's emotional state by correcting the advice (primary advice) obtained by the same process with the emotional state score obtained by the emotional state determination unit 41. As a specific method of correcting the advice, for example, when the user is in a state of feeling strong anger or deep sadness, the degree of the request made to the user in the advice for the user is lowered, and conversely, when the user is in a state of feeling a great feeling of joy, the above request degree is raised. Such correction modes can be considered.

[0064] The advice generation unit 42 is preferably configured by various information processing apparatuses capable of performing the above-described two-stage processing. Such an information processing apparatus can be configured using a machine learning-type information processing apparatus having a neural network, or various other artificial intelligences (AIs).

[0065] (Avatar Image Generation Unit) The avatar image generation unit 43 can be configured by an information processing apparatus having a function of generating an avatar image capable of variably displaying expressions and / or voices corresponding to a plurality of emotions, and a function of selecting the expression and / or voice of the generated avatar image based on the emotional state (emotional state score) determined by the emotional state determination unit 41. In the selection of the expression and / or voice of the avatar image, the content of the advice generated by the advice generation unit 42 may also be taken into account, and the expression and / or voice of the avatar image may be selected based on the combination of the emotional state (emotional state score) and the content of the advice.

[0066] Note that the generation of the avatar image can be performed by known technical means (for example, the avatar image generation method described in Japanese Patent Application Laid-Open No. 2016-71721) for generating various avatar images through arithmetic processing of various modeling materials and parameters related to age, gender, and other morphological characteristics. Also, for the selection of the above-described expression and / or voice of the generated avatar, similarly, through arithmetic processing of modeling materials and parameters for each emotional state, the expression and / or voice of the above avatar image can be changed to an expression and / or voice representing a predetermined emotional state.

[0067] The object of the present invention is to automatically optimize the facial expressions and / or voices (modes of pseudo-emotional expressions) of an avatar that provides advice for each user, and to enhance the mental distance and emotional intimacy with the user. As one approach to achieving this object, as the basic character setting of the avatar, it may be an avatar that imitates the appearance and voice of the user as if it were the user's alter ego, or alternatively, an avatar having attributes that are presumed to be compatible with the user (such as the opposite sex or the same sex of the same age group, an older male, a female, etc., which may be appropriately selected for each field of advice). Thereby, the user can receive personalized advice through communication in a state where the emotional intimacy with the avatar is enhanced, and as a result, the effect that the user is more likely to accept the advice more positively on a psychological level occurs.

[0068] (Robot control unit) In the personal advice providing system of the present invention, the advice output unit 50 can also be constituted by a robot equipped with artificial intelligence and having a conversation function. In this case, as this robot, a robot capable of variably outputting facial expressions and / or voices corresponding to a plurality of emotions is used. And when the personal advice providing system has such a configuration, the arithmetic processing unit 40 may be configured to be provided with a "robot control unit" instead of the avatar image generation unit 43. The "robot control unit" can be constituted by various information processing devices having a function of generating an emotion designation signal for designating an output mode of facial expressions and / or voices for the above-mentioned robot.

[0069] [Advice output unit] The advice output unit 50 can be constituted by various information output means having a function of outputting the advice for the user (advice personalized for each user) generated by the advice generation unit 42 through the avatar generated by the avatar image generation unit 43.

[0070] Specifically, the advice output unit 50 can be configured by a desktop personal computer, a notebook personal computer, a PDA (Personal Digital Assistants), a smartphone, a tablet personal computer, or other portable information processing devices as the information terminal on the client (user) side. Alternatively, the advice output unit 50 can be configured by a device capable of displaying characters, videos, and still images, such as a plasma display, an LED display, a VFD display, or various other projectors, and various speakers capable of outputting sound.

[0071] Furthermore, the advice output unit 50 can also be configured by a conversational robot equipped with artificial intelligence (AI), etc., and the robot is designed to be able to variably change its expression and / or voice and the generated voice.

[0072] <Operation of the Personal Advice Providing Device (System) and Method for Providing Personal Advice> Preferably, the personal advice providing method of the present invention (hereinafter, also simply referred to as the "personal advice providing method") that can be executed by the operation of the personal advice providing device 1 (or the personal advice system) is an information processing process in which the "facial expression photographing step" executed by the photographing unit 10, the "DNA information acquisition step" executed by the DNA information acquisition unit 20, the "auxiliary information acquisition step" executed by the auxiliary information acquisition unit 30, the "emotional state determination step" executed by the emotional state determination unit 41, the "advice generation step" executed by the advice generation unit 42, the "avatar image generation step" executed by the avatar image generation unit 43, and the "advice output step" executed by the advice output unit 50 are sequentially and automatically executed as procedures controlled by a program of the present invention (a program for the personal advice providing system), etc.

[0073] In the method for providing personal advice according to the present invention, among the above steps, at least the "emotional state determination step" is preferably performed using a machine learning type information processing device having a neural network (particularly, a "deep learning type image recognition device"), or various other artificial intelligences (AIs). Also, for the "advice generation step" and the "avatar image generation step", it is preferable to perform them using a machine learning type information processing device having a neural network, or various other artificial intelligences (AIs).

[0074] [Facial expression photographing step] In the "facial expression photographing step", an image (facial expression image) of the user's facial expression is photographed by the photographing unit 10. The "facial expression image" photographed in the "facial expression photographing step" is output as digital format image data and input to the arithmetic processing unit 40 (emotional state determination unit 41), and is used for image analysis processing in the "emotional state determination step".

[0075] [DNA information acquisition step] In the "DNA information acquisition step", the DNA information acquisition unit 20 acquires the user-specific DNA information. This DNA information can be primarily acquired by each individual user, for example, by using various acquisition methods already in practical use in medical institutions and the like, or a dedicated DNA acquisition kit. The unique DNA information of each user thus acquired is input to the DNA information acquisition unit 20 by the user or a system administrator entrusted with the input operation by the user.

[0076] [Auxiliary information acquisition step] In the "auxiliary information acquisition step", the auxiliary information acquisition unit 30 acquires the above-mentioned auxiliary information, that is, the user-specific biological information and / or environmental information, etc., which are information related to the user other than the DNA information. These auxiliary information are acquired by various biological sensors, various information input devices, etc., as described above.

[0077] The acquisition of biometric information in the "auxiliary information acquisition step" may be performed by the user arbitrarily operating a biometric sensor, but it is more preferable that the acquisition of some or all of the information is obtained by measurements that are periodically and automatically performed by a program.

[0078] [Emotional state determination step] In the "emotional state determination step", the emotional state of the user is determined by the emotional state determination unit 41 from the "facial expression image" captured in the "facial expression capture step".

[0079] The determination of the user's emotional state in the "emotional state determination step" is carried out in a process where a "primary emotional state score" is obtained through image analysis processing of the "facial expression image", and this "primary emotional state score" is further corrected by the "user-specific DNA information" that has been previously obtained in the preceding "DNA information acquisition step". Through these two steps of processing, the "emotional state score" is determined. Here, in this specification, the "emotional state score" refers to a value composed of the type of emotional state classified by each type such as "joy" or "sadness", and the intensity of the emotion of that type (see Table 2).

[0080] Regarding the first-stage image analysis processing of the "facial expression image" in the "emotional state determination step", as described above, it can be performed using a machine learning-based information processing device having a neural network (specifically, a "deep learning-based image recognition device"), or various other artificial intelligences (AI). Through this image analysis processing, specifically, as an example, as shown in Table 2 below, an emotional state score (primary emotional state score), which is a combination of the type of the user's emotion and its intensity, is detected. For example, when the extracted facial expression is recognized as an expression of "joy", the emotional intensity is further determined according to the degree of coincidence of feature points, etc., with the facial expression showing the strongest "joy" emotion, and the "primary emotional state score" is calculated.

[0081]

Table 2

[0082] Then, as an example, the process of correcting the above-mentioned "primary emotional state score" with the user-specific DNA information is performed by correcting the "primary emotional state score" with the score related to the user's temperament that can be obtained from the DNA information, as shown in Table 3 below. Even if human expressions are the same image as an image, the actual intensity of emotions differs depending on the temperament of each person. By correcting the "primary emotional state score" as the result of image analysis with the temperament of each person, a corrected emotional state score with higher suitability for each user can be calculated.

[0083]

Table 3

[0084] [Advice generation step] In the "advice generation step", the advice generation unit 42 generates personalized advice for each user by using the user-specific DNA information obtained in the "DNA acquisition step" and the user-specific "auxiliary information" obtained in the auxiliary information acquisition step as input values.

[0085] The generation of advice in the "advice generation step" can also be performed in such a manner that the advice (primary advice) obtained from the DNA information and the auxiliary information is corrected with the "emotional state score" of the user himself / herself obtained through a two-step process in the "emotional state determination step", and the corrected advice is output as the corrected advice corrected according to the user's emotional state. The specific method of correcting the advice in this case is as described above. For example, as also shown in Tables 4 and 5 below as an example, when the user is in a state of feeling strong anger or deep sadness, the degree of the request required of the user in the advice for the user is lowered, and conversely, when the user is in a state of feeling great joy, the degree of the above request is raised. Such a correction mode can be considered.

[0086] [Table 4]

[0087] [Table 5]

[0088] [Avatar Image Generation Step] In the "Avatar Image Generation Step", an avatar image capable of variably displaying expressions corresponding to a plurality of emotions is generated by the avatar image generation unit 43. In generating the avatar image, based on the emotion state score obtained in the "Emotion State Determination Step", an appropriate expression and / or voice of the avatar image is selected. In selecting the expression and / or voice of the avatar image, the content of the advice generated by the advice generation unit 42 may also be taken into account, and the expression and / or voice of the avatar image may be selected based on the combination of the emotion state (emotion state score) and the content of the advice.

[0089] Also, in the "Avatar Image Generation Step", it is preferable to extract the amount of change in the emotion state score at the time points before and after the time when the advice is output, and based on the amount of change, adjust the expression and / or voice of the avatar image by reselecting the expression and / or voice of the avatar image.

[0090] FIG. 3 and FIG. 4 are examples of avatar images capable of variably displaying expressions corresponding to a plurality of emotions generated in the "Avatar Image Generation Step". The avatar in FIG. 3 is assumed to be in a case where the emotion state score of the user is an average value or in a state of joy, and the avatar in FIG. 4 is assumed to be in a case where the emotion state score of the user is in a state of sadness or anger. By approximating the expression and / or voice of the avatar, that is, the mode of pseudo-emotional expression, to the emotion state of the user, an effect of enhancing the mental distance and emotional intimacy with the user is produced.

[0091] [Advice Output Step] In the "Advice Output Step", personalized advice for each user generated in the "Advice Generation Step" via the avatar image generated in the "Avatar Image Generation Step" is output by the advice output unit 50 to the user.

Explanation of Signs

[0092] 1 Personalized Advice Provision System 10 Photographing Unit 20 DNA Information Acquisition Unit 30 Auxiliary Information Acquisition Unit 40 Arithmetic Processing Unit 41 Emotional State Judgment Unit 42 Advice Generation Unit 43 Avatar Image Generation Unit 50 Advice Output Unit

Claims

1. A personal advice providing device that uses a DNA diagnostic result of a user to provide advice that contributes to improving the quality of life of the user, The advice comprises personalized career or path advice for the user; The photography department and A DNA information acquisition unit that acquires DNA information unique to a user; an auxiliary information acquisition unit that acquires auxiliary information, which is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines an emotional state of the user from a facial expression image of the user captured by the imaging unit; an advice generating unit that generates the advice by a calculation process using the DNA information and the auxiliary information as input values, and when generating advice on a course or career personalized for the user, generates the advice by a calculation process that further uses the temperament and constitution of the user that can be derived from the DNA information as input values; an avatar image generator that generates an avatar image capable of variably displaying facial expressions and / or voices corresponding to a plurality of emotions; an advice output unit that outputs the advice via the avatar image, the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information acquired by the DNA information acquisition unit; the avatar image generation unit selects a facial expression and / or a voice of the avatar image based on the emotional state score. A personal advice providing device.

2. the avatar image generation unit adjusts a facial expression and / or a voice of the avatar image selected based on the emotional state score in accordance with the content of the advice.

2. The apparatus for providing personal advice according to claim 1.

3. The emotional state determination unit extracts an amount of change in the emotional state score between a time point before and a time point when the advice is output, the avatar image generation unit reselects a facial expression and / or a voice of the avatar image based on the amount of change.

3. The personal advice providing device according to claim 1 or 2.

4. A personal advice providing system that uses a DNA diagnostic result of a user to provide advice that contributes to improving the quality of life of the user, The advice comprises personalized career or path advice for the user; The photography department and A DNA information acquisition unit that acquires DNA information unique to a user; an auxiliary information acquisition unit that acquires auxiliary information, which is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines an emotional state of the user from a facial expression image of the user captured by the imaging unit; an advice generating unit that generates the advice by a calculation process using the DNA information and the auxiliary information as input values, and when generating advice on a course or career personalized for the user, generates the advice by a calculation process that further uses the temperament and constitution of the user that can be derived from the DNA information as input values; an avatar image generator that generates an avatar image capable of variably displaying facial expressions and / or voices corresponding to a plurality of emotions; an advice output unit that outputs the advice via the avatar image, the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information acquired by the DNA information acquisition unit; the avatar image generation unit selects a facial expression and / or a voice of the avatar image based on the emotional state score. Personal advice delivery system.

5. the avatar image generation unit adjusts a facial expression and / or a voice of the avatar image selected based on the emotional state score in accordance with the content of the advice.

5. The system for providing personal advice according to claim 4.

6. The emotional state determination unit extracts an amount of change in the emotional state score between a time point before and a time point when the advice is output, the avatar image generation unit reselects a facial expression and / or a voice of the avatar image based on the amount of change.

6. A system for providing personal advice according to claim 4 or 5.

7. A personal advice providing system that uses a DNA diagnostic result of a user to provide advice that contributes to improving the quality of life of the user, The advice comprises personalized career or path advice for the user; The photography department and A DNA information acquisition unit that acquires DNA information unique to a user; an auxiliary information acquisition unit that acquires auxiliary information, which is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines an emotional state of the user from a facial expression image of the user captured by the imaging unit; an advice generating unit that generates the advice by a calculation process using the DNA information and the auxiliary information as input values, and when generating advice on a course or career personalized for the user, generates the advice by a calculation process that further uses the temperament and constitution of the user that can be derived from the DNA information as input values; a robot control unit that generates an emotion designation signal that designates an output mode of an emotion and / or a voice for a robot capable of variably outputting an emotion and / or a voice corresponding to a plurality of emotions; an advice output unit comprising the robot; the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information acquired by the DNA information acquisition unit; the robot control unit selects the emotion designation signal based on the emotion state score. Personal advice delivery system.

8. the robot control unit adjusts the emotion designation signal selected based on the emotional state score in accordance with the content of the advice. The system for providing personal advice according to claim 7.

9. The emotional state determination unit extracts an amount of change in the emotional state score between a time point before and a time point when the advice is output, the robot control unit reselects the emotion designation signal based on the amount of change.

9. A system for providing personal advice according to claim 7 or 8.

10. A method for providing personal advice that utilizes a DNA diagnostic result of a user to provide advice that contributes to improving the quality of life of the user, comprising: The advice comprises personalized career or path advice for the user; A facial expression capturing step in which an image capturing unit captures an image of a user's facial expression; a DNA information acquisition step in which a DNA information acquisition unit acquires DNA information specific to the user; an auxiliary information acquisition step of acquiring auxiliary information, which is biometric information and / or environmental information specific to the user other than the DNA information, by an auxiliary information acquisition unit; an emotional state determination step in which an emotional state determination unit determines an emotional state of the user from the facial expression image; an advice generating step in which an advice generating unit generates the advice by a calculation process using the DNA information and the auxiliary information as input values, and when generating advice on a course or career personalized for the user, the advice is generated by a calculation process using, as input values, the temperament and constitution of the user that can be derived from the DNA information; an avatar image generating step in which an avatar image generating unit generates an avatar image capable of variably displaying facial expressions and / or voices corresponding to a plurality of emotions; an advice output step in which an advice output unit outputs the advice via the avatar image; In the emotional state determination step, a primary emotional state score obtained by image analysis processing of the facial expression image is corrected by the DNA information acquired by the DNA information acquisition unit to determine the emotional state score, In the avatar image generating step, a facial expression and / or a voice of the avatar image is selected based on the emotional state score. How to provide personalized advice.

11. In the avatar image generating step, a facial expression and / or a voice of the avatar image selected based on the emotional state score is adjusted in accordance with the content of the advice. A method for providing personal advice according to claim 10.

12. In the emotional state determination step, a change amount of the emotional state score between a time point before and a time point when the advice is output is extracted, In the avatar image generating step, a facial expression and / or a voice of the avatar image is reselected based on the amount of change.

12. A method for providing personal advice according to claim 10 or 11.

13. 12. A method for providing personal advice comprising the steps of: A program for providing personalized advice.

Images