Personal advice providing apparatus and personal advice providing system

Abstract

To provide a personal advice providing apparatus configured to automatically optimize, for each user, expression and / or voice of an avatar that provides advice, thereby reducing psychological distance and enhancing emotional closeness with the user.SOLUTION: A personal advice providing apparatus 1 includes an imaging unit 10, an emotional state determination unit 41, a DNA information acquisition unit 20, an auxiliary information acquisition unit 30, an advice generation unit 42 which generates advice using DNA information and auxiliary information; an avatar image generation unit 43 which generates an avatar image which can variably display expression and / or voice; and an advice output unit 50 which outputs the advice via the avatar image. The emotional state determination unit 41 determines an emotional state score by correcting a primary emotional state score to be obtained by performing image analysis processing on an expression image, with the DNA information. The avatar image generation unit 43 adjusts the expression and / or voice of the avatar image based on the emotional state score.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a "personal advice providing device" and a "personal advice providing system." More specifically, the present invention relates to a "personal advice providing device" and a "personal advice providing system" that provide personalized advice to individual users across a variety of fields that contribute to improving the quality of life of each individual user. [Background technology]

[0002] Since ancient times, humans have sought advice from others (teachers, friends, fortune tellers, consultants, etc.) in various fields as a means of problem-solving. Nowadays, various advice-providing systems that utilize conversational computer programs using artificial intelligence (AI) are becoming widely used as one of the means of problem-solving.

[0003] For example, in the medical and health fields, various devices have been developed that utilize a user's unique DNA information as basic information to generate personalized health advice for the user and provide that advice to the user (Patent Documents 1 and 2).The inventors of this application came up with the idea of ​​using such a device that utilizes DNA information as a personal advice providing device that is not limited to the medical field but provides advice across various fields that contribute to improving the quality of life of each individual user.

[0004] We thought that it would be more desirable for such a versatile personal advice providing device to provide various personalized advice through an avatar optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.However, there was no technical means yet available to automatically optimize the facial expressions and / or voice (emotional expressions) of the avatar providing the advice for each user, which would be necessary to achieve this. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-96183 [Patent Document 2] Patent No. 6793267 Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention aims to provide a highly versatile personal advice providing device that uses DNA diagnostic results to provide advice across a variety of fields that contributes to improving the quality of life of individual users, by automatically optimizing the facial expressions and / or voice (modes of pseudo-emotional expression) of the avatar providing the advice for each user, thereby increasing the mental distance and emotional intimacy with the user. [Means for solving the problem]

[0007] The inventors have conceived the idea that the above problem can be solved by adjusting the facial expression and / or voice of an avatar image to an optimal facial expression and / or voice based on a corrected emotional state score of a user, which is obtained by correcting the user's primary emotional state score read from the user's facial expression with the user's unique DNA information, when providing the user with personalized advice based on the user's unique DNA information, and have thus completed the present invention.

[0008] (1) A personal advice providing device comprising: an imaging unit; a DNA information acquisition unit that acquires DNA information specific to a user; an auxiliary information acquisition unit that acquires auxiliary information, which is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines the emotional state of the user from an expression image of the user captured by the imaging unit; an advice generation unit that generates advice for the user by performing an arithmetic process using the DNA information and the auxiliary information as input values; an avatar image generation unit that generates an avatar image that can variably display expressions and / or voices corresponding to a plurality of emotions; and an advice output unit that outputs the advice via the avatar image, wherein the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the expression image with the DNA information, and the avatar image generation unit selects the expression and / or voice of the avatar image based on the emotional state score.

[0009] According to the personal advice providing device (1), in a highly versatile personal advice providing device that uses DNA diagnostic results to provide advice across a variety of fields that contributes to improving the quality of life of each individual user, the facial expressions and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.

[0010] (2) The personal advice providing device according to (1), wherein the avatar image generating unit adjusts the facial expression and / or voice of the avatar image selected based on the emotional state score in accordance with the content of the advice.

[0011] According to the personal advice providing device of (2), the facial expression and / or voice (manner of pseudo-emotional expression) of the avatar providing advice can be optimized taking into consideration the content of the advice, thereby further increasing the mental distance and emotional intimacy with the user.

[0012] (3) The personal advice providing device described in (1) or (2), wherein the emotional state determination unit extracts the amount of change in the emotional state score between the time before and the time when the advice is output, and the avatar image generation unit reselects the facial expression and / or voice of the avatar image based on the amount of change.

[0013] According to the personal advice providing device of (3), in the personal advice providing device of (1) or (2), the facial expression and / or voice (manner of pseudo-emotional expression) of the avatar providing advice can be optimized over time for each user, taking into full consideration real-time changes in the emotional state of the user. This allows for highly empathetic communication that feels more similar to communication with a real person, thereby increasing the mental distance and emotional intimacy with the user to an extremely high level.

[0014] (4) A personal advice providing system comprising: an imaging unit; a DNA information acquisition unit that acquires DNA information specific to a user; an auxiliary information acquisition unit that acquires auxiliary information, which is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines the emotional state of the user from an expression image of the user captured by the imaging unit; an advice generation unit that generates advice for the user by performing arithmetic processing using the DNA information and the auxiliary information as input values; an avatar image generation unit that generates an avatar image that can variably display expressions and / or voices corresponding to a plurality of emotions; and an advice output unit that outputs the advice via the avatar image, wherein the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the expression image with the DNA information, and the avatar image generation unit selects the expression and / or voice of the avatar image based on the emotional state score.

[0015] According to (4), the personal advice providing system is a highly versatile personal advice providing system that uses DNA diagnostic results to provide advice across a variety of fields that contributes to improving the quality of life of individual users.The facial expressions and / or voice (modes of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.

[0016] (5) The personal advice providing system according to (4), wherein the avatar image generating unit adjusts the facial expression and / or voice of the avatar image selected based on the emotional state score according to the content of the advice.

[0017] According to the personal advice providing system of (5), the facial expression and / or voice (the manner of pseudo-emotional expression) of the avatar providing advice can be optimized taking into consideration the content of the advice, thereby further increasing the mental distance and emotional intimacy with the user.

[0018] (6) A personal advice providing system as described in (4) or (5), wherein the emotional state determination unit extracts the amount of change in the emotional state score between the time before and the time when the advice is output, and the avatar image generation unit reselects the facial expression and / or voice of the avatar image based on the amount of change.

[0019] According to the personal advice providing system of (6), in the personal advice providing system of (4) or (5), the facial expression and / or voice (manner of pseudo-emotional expression) of the avatar providing advice can be optimized over time for each user, taking into full consideration real-time changes in the emotional state of the user. This allows for highly empathetic communication that feels more similar to communication with a real person, thereby increasing the mental distance and emotional intimacy with the user to an extremely high level.

[0020] (7) A program for a personal advice providing system described in (4) or (5), which causes the emotional state determination unit to execute a process of determining the emotional state score by correcting the primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information, and a process of selecting the facial expression and / or voice of the avatar image based on the emotional state score, in the avatar image generation unit.

[0021] (7) According to the program for the personal advice providing system, in a highly versatile personal advice providing system that uses DNA diagnostic results to provide advice across a variety of fields that contribute to improving the quality of life of each individual user, the facial expressions and / or voice (modes of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.

[0022] (8) A personal advice providing system comprising: an imaging unit; a DNA information acquisition unit that acquires DNA information specific to a user; an auxiliary information acquisition unit that acquires auxiliary information that is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines the emotional state of the user from an expression image of the user captured by the imaging unit; an advice generation unit that generates advice for the user by arithmetic processing using the DNA information and the auxiliary information as input values; a robot control unit that generates an emotion designation signal that specifies the output manner of an expression and / or voice to a robot that can variably output expressions and / or voices corresponding to a plurality of emotions; and an advice output unit consisting of the robot, wherein the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the expression image with the DNA information, and the robot control unit selects the emotion designation signal based on the emotional state score.

[0023] According to the personal advice providing system (8), in a highly versatile personal advice providing device that uses DNA diagnostic results to provide advice across a variety of fields that contribute to improving the quality of life of each individual user, the facial expressions and / or voice (modes of pseudo-emotional expression) of the robot providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.

[0024] (9) The personal advice providing system according to (8), wherein the robot control unit adjusts the emotion designation signal selected based on the emotional state score in accordance with the content of the advice.

[0025] According to the personal advice providing system of (9), the facial expressions and / or voice (manner of pseudo-emotional expression) of the robot providing advice can be optimized taking into consideration the content of the advice, thereby further increasing the mental distance and emotional intimacy with the user.

[0026] (10) A personal advice system as described in (8) or (9), wherein the emotional state determination unit extracts the amount of change in the emotional state score between the time before and the time when the advice is output, and the robot control unit reselects the emotion designation signal based on the amount of change.

[0027] According to the personal advice providing system of (10), in the personal advice providing system of (4) or (5), the facial expressions and / or voice (manner of pseudo-emotional expression) of the robot providing advice can be optimized over time for each user, taking into full consideration real-time changes in the emotional state of the user. This allows for highly empathetic communication that feels more similar to communication with a real person, thereby increasing the mental distance and emotional intimacy with the user to an extremely high level.

[0028] (11) A program for a personal advice providing system, which causes the emotional state determination unit to execute a process of determining the emotional state score by correcting the primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information, in the personal advice providing system described in (8) or (9), and a process of selecting the emotion designation signal based on the emotional state score, in the robot control unit.

[0029] According to the program for a personal advice providing system of (11), in a highly versatile personal advice providing system that uses DNA diagnostic results to provide advice across a variety of fields that contribute to improving the quality of life of each individual user, the facial expression and / or voice (mode of pseudo-emotional expression) of the robot providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.The facial expression and / or voice (mode of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.

[0030] (12) A method for providing personal advice, comprising: an expression capturing step in which an image capturing unit captures an expression image of a user; a DNA information acquiring step in which a DNA information acquiring unit acquires DNA information specific to the user; an auxiliary information acquiring step in which an auxiliary information acquiring unit acquires auxiliary information which is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determining step in which an emotional state determining unit determines the emotional state of the user from the expression image; an advice generating step in which an advice generating unit generates advice for the user by arithmetic processing using the DNA information and the auxiliary information as input values; an avatar image generating step in which an avatar image generating unit generates an avatar image capable of variably displaying expressions and / or voices corresponding to a plurality of emotions; and an advice output step in which an advice output unit outputs the advice via the avatar image, wherein in the emotional state determining step, the emotional state score is determined by correcting a primary emotional state score obtained by image analysis processing of the expression image with the DNA information, and in the avatar image generating step, the expression and / or voice of the avatar image is selected based on the emotional state score.

[0031] According to the personal advice providing method (12), when an information processing device or information processing system provides versatile personalized advice that uses DNA diagnostic results to provide advice across a variety of fields that contributes to improving the quality of life of each individual user, the facial expression and / or voice (aspects of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user.

[0032] (13) The method for providing personal advice according to (12), wherein in the avatar image generating step, the facial expression and / or voice of the avatar image selected based on the emotional state score is adjusted according to the content of the advice.

[0033] According to the method for providing personal advice (13), the facial expression and / or voice (the manner of pseudo-emotional expression) of the avatar providing advice can be optimized in consideration of the content of the advice, thereby further increasing the mental distance and emotional intimacy with the user.

[0034] (14) A method for providing personal advice according to (12) or (13), wherein in the emotional state determination step, the amount of change in the emotional state score between the time before and the time when the advice is output is extracted, and in the avatar image generation step, the facial expression and / or voice of the avatar image is reselected based on the amount of change.

[0035] According to the personal advice providing method of (14), in the personal advice providing method of (12) or (13), the avatar providing the advice can optimize the facial expression and / or voice (the manner of pseudo-emotional expression) of the avatar providing the advice over time for each user, fully considering real-time changes in the user's emotional state. This allows for highly empathetic communication that feels more similar to communication with a real person, thereby increasing the mental distance and emotional intimacy with the user to an extremely high level. [Effects of the Invention]

[0036] According to the present invention, in a highly versatile personal advice providing device that uses DNA diagnostic results to provide advice across a variety of fields that contributes to improving the quality of life of individual users, the facial expressions and / or voice (modes of pseudo-emotional expression) of the avatar providing the advice can be automatically optimized for each user, thereby increasing the mental distance and emotional intimacy with the user. [Brief explanation of the drawings]

[0037] [Figure 1]1 is a block diagram showing the components of a personal advice providing device according to the present invention; [Figure 2] 1 is a conceptual diagram schematically illustrating the basic design concept, configuration, and operation of a personal advice providing device of the present invention. [Figure 3] 1 is a front view showing an example of a specific embodiment and implementation (aspect of pseudo-emotional expression of an avatar) of a personal advice providing device of the present invention.

[0023] FIG. [Figure 4] 1 is a front view showing an example of a specific embodiment and implementation (aspect of pseudo-emotional expression of an avatar) of a personal advice providing device of the present invention.

[0023] FIG. DETAILED DESCRIPTION OF THE INVENTION

[0038] <Personal advice providing device / system> The best mode for carrying out the present invention will be described below with reference to the accompanying drawings. The "personal advice providing device" and "personal advice providing system" of the present invention are information processing devices or information processing systems that provide personalized advice to individual users across a variety of fields that contribute to improving the quality of life of each user.

[0039] The range of fields to which advice can be provided is not particularly limited, but specific examples of fields to which the ``personal advice providing device'' and ``personal advice providing system'' of the present invention can be suitably applied include the fields shown in Table 1 below.

[0040] [Table 1]

[0041] For example, when providing advice on future career paths, in the past, biometric information such as the user's age and gender, as well as acquired environmental information specific to the user, such as school age and hobbies and preferences, were used exclusively as input values ​​for variables in functions for generating advice. However, in this application, the user's temperament and constitution, which can be derived from DNA information, are also actively used as input values ​​for generating advice. This makes it possible to take into account the user's genetic constitution and temperament, even if the user himself is not aware of them, and generate personalized advice for the user from a perspective different from conventional ones.

[0042] [Overall configuration] The basic configuration of a personal advice providing device 1, which is one example of a preferred embodiment of the "personal advice providing device" of the present invention, is as shown in the block diagram of Figure 1. The personal advice providing device 1 includes an imaging unit 10, a DNA information acquisition unit 20, an auxiliary information acquisition unit 30, a calculation processing unit 40, and an advice output unit 50. The calculation processing unit 40 includes an emotional state determination unit 41, an advice generation unit 42, and an avatar image generation unit 43.

[0043] The personal advice providing device 1 is an information processing device in which an imaging unit 10, a DNA information acquiring unit 20, an auxiliary information acquiring unit 30, an arithmetic processing unit 40, and an advice output unit 50 are integrated.

[0044] Furthermore, the present invention can also be implemented as a distributed information processing system, i.e., a "personal advice providing system," that can achieve the same effects as the personal advice providing device 1 by distributing all or part of the components of the personal advice providing device 1 shown in Fig. 1 in any suitable location away from the other components. When the present invention is configured as a distributed system in this way, the above-mentioned components may be connected to each other so as to be able to communicate with each other by various communication means, such as a wired connection using a dedicated communication cable, a wired LAN, a wireless LAN, short-range wireless communication, or a mobile phone line.

[0045] As another embodiment of the present invention, the above-mentioned "personal advice providing system" may be partially modified so that advice output unit 50 is configured as a robot capable of variably outputting facial expressions and / or voices corresponding to a plurality of emotions, and processing unit 40 is provided with, instead of avatar image generation unit 43, a robot control unit that generates an emotion designation signal that designates the output manner of the robot's facial expressions and / or voices.

[0046] [Photography Department] The photographing unit 10 can be configured with various cameras capable of capturing images of the facial expressions of the user of the personal advice providing device 1 (referred to as "facial expression images" in this specification). The camera constituting the photographing unit 10 may be any camera that has the function of outputting the captured "facial expression images" of the user as digital image data so that the captured "facial expression images" can be used for calculation processing related to image analysis in the emotional state determination unit 41. Specifically, various existing digital cameras with video capture functions can be used as the camera constituting the photographing unit 10 without any particular restrictions.

[0047] The photographing unit 10 is placed in a position where it can continuously photograph "facial expression images" of the user when the user uses the personal advice providing device 1. For this reason, for example, as shown in Figures 3 and 4, in the personal advice providing device 1, it is preferable that the photographing unit 10 is configured as a small camera that is connected to or built into the advice output unit 50. Furthermore, when the advice output unit 50 is configured by a laptop personal computer or various types of portable information terminal, it is also possible to configure such a PC or the like to have a built-in small camera that is built in so as to be able to photograph the area in front of the information display screen function as the photographing unit 10.

[0048] [DNA Information Acquisition Department] The DNA information acquisition unit 20 can be configured by an information processing device having a function to acquire user-specific DNA information as data used to generate advice for the user and to appropriately correct and determine the user's emotional state score. More specifically, the DNA information acquisition unit 20 is not limited to a specific information processing device as long as it has a function to acquire and temporarily store "user-specific DNA information" and supply this information to the calculation processing unit 40 that constitutes the personal advice providing device 1 or the personal advice supply system.

[0049] The personal advice providing device 1 preferably further includes a DNA information processing unit. The advice generation unit 42 and the avatar image generation unit 43 are preferably configured by a machine learning information processing device having a neural network or other artificial intelligence (AI), and the DNA information processing unit refers to an information processing device that has the function of analyzing the user's unique DNA information input to the DNA information acquisition unit 20 and processing it into a data format and content that can be used for predetermined calculations in the above-mentioned information processing device, etc. Here, DNA information analysis specifically refers to, for example, SNP analysis, CNV (Copy Number Variation) analysis, microsatellite analysis, epiDNA analysis, or gene expression level analysis.

[0050] [Auxiliary information acquisition department] The auxiliary information acquisition unit 30 can be configured by an information processing device having a function of acquiring auxiliary information as data to be used in combination with the above DNA information to generate advice for the user and in combination with the above DNA information to determine the user's emotional state score. Here, auxiliary information refers to information related to the user other than the above DNA information, such as biometric information and / or environmental information specific to the user.

[0051] Examples of the user-specific biometric information include height, weight, electrocardiogram waveform, autonomic nervous activity, respiratory frequency, walking distance, amount of sleep, respiratory rate, blood oxygen saturation, blood glucose level, blood pressure, and heart rate. These pieces of biometric information are data that can be measured using various biosensors (measuring instruments). The auxiliary information acquisition unit 30 is preferably configured to include the various biosensors described above for acquiring these pieces of biometric information. As an example, of the above pieces of biometric information, the electrocardiogram waveform, autonomic nervous activity, and respiratory frequency can be measured using an electrocardiograph as a biosensor. Furthermore, the walking distance, amount of sleep, and respiratory rate can be measured using an acceleration sensor. Furthermore, the blood oxygen saturation, blood glucose level, blood pressure, and heart rate can be measured using an optical sensor.

[0052] Other examples of biometric information specific to a user include date of birth (age), gender, and the disease history of the user and their close relatives. This information can be made available by the user either by directly inputting the information or by having a digital data reading device such as a card reader read a recording medium on which personal information is recorded. The auxiliary information acquisition unit 30 is preferably configured to include the various information input devices described above for acquiring this biometric information, specifically, the digital data reading device, a microphone that collects the user's voice, a touch panel type information input device that can input text information, etc.

[0053] On the other hand, environmental information specific to a user includes, for example, dietary habits, social status, income, family structure, and geographical environment of residence. This information is also data that can be used by the user either by directly inputting the information or by having a reading device read a recording medium or the like on which personal information is recorded. The auxiliary information acquisition unit 30 is preferably configured to include the above-mentioned various information input devices for acquiring this environmental information.

[0054] [Calculation processing unit] Processing unit 40 essentially includes an emotional state determination unit 41 that determines the emotional state of the user, an advice generation unit 42 that generates advice for the user, and an avatar image generation unit 43 that generates an avatar image. As described above, a robot control unit may be provided instead of avatar image generation unit 43. Details of each component will be described later.

[0055] Processing unit 40 may be configured as a dedicated information processing device specialized for a specific operation according to the present invention. Alternatively, processing unit 40 may be configured using a personal computer, a tablet terminal, various types of portable information processing terminals, etc. Processing unit 40 may also be configured as a cloud server that can operate as an information processing device that functions as emotional state determination unit 41, advice generation unit 42 that generates advice for a user, and avatar image generation unit 43 that generates an avatar image, and this server may be shared by multiple clients.

[0056] In any of the above configurations, the arithmetic processing unit 40 can be configured by various known information processing devices such as a computer equipped with hardware such as a CPU, memory, a communication unit, etc. The arithmetic processing unit 40 having the above configuration can specifically execute various operations of the personal advice providing device 1 using the "program for the personal advice providing system" of the present invention.

[0057] (Emotional state determination section) The emotional state determination unit 41 can be configured as an information processing device that has the function of determining the emotional state of the user (for example, a score value consisting of a distinction between each emotional state such as joy, anger, surprise, sadness, etc. and the intensity of each distinguished emotional state) using the facial expression image of the user captured by the capture unit 10.

[0058] Here, the emotional state determination unit 41 in the personal advice providing device 1 determines the emotional state of the user by first performing a first-stage process of image analysis of the user's facial expression image, and then performing a second-stage process of determining the emotional state score by correcting the emotional state score (primary emotional state score) obtained by the first stage process using DNA information specific to the user and other auxiliary information related to the user.

[0059] Therefore, the emotional state determination unit 41 needs to be configured with various information processing devices capable of performing the above-mentioned two-stage processing. For this purpose, the information processing device is configured to include, in particular, image analysis means for performing the first-stage processing of analyzing facial expression images to obtain a primary emotional state score. The image analysis means for analyzing an image of a person's face to determine the emotional state of the person and obtain an emotional state score (primary emotional state score) can be configured using a machine learning information processing device having a neural network (particularly, a "deep learning image recognition device") or various other types of artificial intelligence (AI). Various artificial intelligence systems on the cloud can also be used as such artificial intelligence (AI). Specific examples of image recognition technology using deep learning are disclosed below. "Deep Learning and Image Recognition, Operations Research" (http: / / www.orsj.o.jp / archive2 / or60-4 / or60_4_198.pdf)

[0060] Furthermore, with regard to the technology for reading the emotional state of a person from an image of the person using image recognition technology based on machine learning information processing devices having the above-mentioned neural networks, many specific technical means are already publicly known, as disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-59107 and Japanese Patent Application Laid-Open No. 2022-114789. In the personal advice providing device 1 of the present invention, the emotional state determination unit 41 can also be configured using these various publicly known technical means.

[0061] Furthermore, the information processing device that performs the second stage of processing, which determines the emotional state score by correcting the "primary emotional state score" obtained in the first stage of processing using the user's unique DNA information and other auxiliary information related to the user, is preferably configured using a machine learning information processing device with a neural network or other types of artificial intelligence (AI). This allows past learning results to contribute to the determination of the emotional state score, further improving the accuracy of the emotional state judgment and further improving the compatibility of the final output emotional state score with the user.

[0062] (Advice generation unit) The advice generation unit 42 can be configured as an information processing device that has the function of generating advice for users (advice personalized for each user) to be output from the advice output unit 50 by performing calculations using the user's unique DNA information and other auxiliary information related to the user as input values.

[0063] Here, the generation of advice for a user by the advice generation unit 42 in the personal advice providing device 1 first involves a first stage of calculation processing in which advice (primary advice) is obtained by the above-mentioned calculation processing using the user's unique DNA information, etc., as input values, and then the advice (primary advice) obtained by this processing can be corrected with the emotional state score obtained by the emotional state determination unit 41 to generate corrected advice that has been corrected in accordance with the user's emotional state. Specific ways of correcting advice include, for example, lowering the level of demands made on the user in the advice to that user when the user is feeling strong anger or deep sadness, and conversely, raising the level of the demands when the user is feeling great joy.

[0064] The advice generation unit 42 is preferably configured with various information processing devices capable of performing the above-mentioned two-stage processing. Such information processing devices can be configured using machine learning information processing devices with neural networks or various other artificial intelligence (AI).

[0065] (Avatar image generation section) Avatar image generation unit 43 can be configured by an information processing device having a function to generate an avatar image capable of variably displaying facial expressions and / or voices corresponding to a plurality of emotions, and a function to select the facial expression and / or voice of the generated avatar image based on the emotional state (emotional state score) determined by emotional state determination unit 41. When selecting the facial expression and / or voice of the avatar image, the content of the advice generated by advice generation unit 42 may also be taken into consideration, and the facial expression and / or voice of the avatar image may be selected based on a combination of the emotional state (emotional state score) and the content of the advice.

[0066] The generation of avatar images can be performed by known technical means (e.g., the avatar image generation method described in JP 2016-71721 A) that generates various avatar images by arithmetic processing of various modeling materials and parameters related to age, gender, and other morphological characteristics. Similarly, the selection of the facial expression and / or voice of the generated avatar can be performed by arithmetic processing of modeling materials and parameters for each emotional state, thereby changing the facial expression and / or voice of the avatar image to one that expresses a predetermined emotional state.

[0067] The object of the present invention is to automatically optimize the facial expression and / or voice (mode of pseudo-emotional expression) of an avatar providing advice for each user, thereby increasing the psychological distance and emotional intimacy with the user. One approach to achieving this object is to set the basic character of the avatar to an avatar that imitates the user's appearance and voice, as if it were an alter ego of the user, or to set an avatar with attributes that are presumed to be compatible with the user (such as a person of the same age, opposite or same sex, an older man, a woman, etc., which can be selected as appropriate for each field of advice). This allows the user to receive personalized advice through communication with the avatar in a state of increased emotional intimacy, which has the effect of making the user more likely to accept the advice positively from a psychological perspective.

[0068] (Robot control unit) In the personal advice providing system of the present invention, the advice output unit 50 can also be configured as a robot equipped with artificial intelligence and having a conversation function. In this case, the robot can be a robot that can variably output facial expressions and / or voices corresponding to a plurality of emotions. When the personal advice providing system is configured in this way, the arithmetic processing unit 40 can be configured to include a "robot control unit" instead of the avatar image generation unit 43. The "robot control unit" can be configured as any of various information processing devices that have a function of generating an emotion designation signal that designates the output mode of facial expressions and / or voice for the robot.

[0069] [Advice output section] The advice output unit 50 can be configured by various information output means that have the function of outputting advice for users (advice personalized for each user) generated by the advice generation unit 42 through an avatar generated by the avatar image generation unit 43.

[0070] Specifically, the advice output unit 50 can be configured as a client (user) side information terminal using a portable information processing device such as a desktop personal computer, a notebook personal computer, a PDA (Personal Digital Assistant), a smartphone, a tablet personal computer, etc. Alternatively, the advice output unit 50 can be configured using a plasma display, an LED display, a VFD display, various other projectors, or other devices capable of displaying text, video, and still images, and various speakers capable of outputting audio.

[0071] The advice output unit 50 can also be configured by a conversational robot equipped with artificial intelligence (AI) and designed to be able to variably change its facial expressions and / or voice or the voice it produces.

[0072] <Operation of the personal advice providing device (system) and method of providing personal advice> Preferably, the personal advice providing method of the present invention (hereinafter also simply referred to as the "personal advice providing method"), which can be executed by the operation of the personal advice providing device 1 (or personal advice system), is an information processing process in which an "facial expression capturing step" executed by the imaging unit 10, a "DNA information acquisition step" executed by the DNA information acquisition unit 20, an "auxiliary information acquisition step" executed by the auxiliary information acquisition unit 30, an "emotional state determination step" executed by the emotional state determination unit 41, an "advice generation step" executed by the advice generation unit 42, an "avatar image generation step" executed by the avatar image generation unit 43, and an "advice output step" executed by the advice output unit 50 are automatically executed in sequence as a procedure controlled by the program of the present invention (a program for the personal advice providing system) or the like.

[0073] In the personal advice providing method of the present invention, of the above steps, it is preferable that at least the "emotional state determination step" be performed using a machine learning type information processing device having a neural network (particularly a "deep learning type image recognition device") or any other type of artificial intelligence (AI), and it is also preferable that the "advice generation step" and the "avatar image generation step" be performed using a machine learning type information processing device having a neural network or any other type of artificial intelligence (AI).

[0074] [Facial Expression Shooting Steps] In the "facial expression capturing step", an image of the user's facial expression (facial expression image) is captured by the capturing unit 10. The "facial expression image" captured in the "facial expression capturing step" is output as image data in a digital format and input to the calculation processing unit 40 (emotional state determination unit 41), and is used for image analysis processing in the "emotional state determination step".

[0075] [DNA information acquisition step] In the "DNA information acquisition step," the DNA information acquisition unit 20 acquires DNA information specific to the user. This DNA information can be primarily acquired by each individual user, for example, by using various acquisition methods already in practical use in medical institutions or dedicated DNA acquisition kits. The unique DNA information of each user acquired in this way is input into the DNA information acquisition unit 20 by the user or a system administrator delegated by the user to perform the input operation.

[0076] [Auxiliary information acquisition step] In the "auxiliary information acquisition step," the auxiliary information acquisition unit 30 acquires the above-mentioned auxiliary information, i.e., information related to the user other than DNA information, such as biometric information and / or environmental information specific to the user. As described above, this auxiliary information is acquired by various biosensors, various information input devices, etc.

[0077] The acquisition of biometric information in the "auxiliary information acquisition step" may be performed by the user by operating the biometric sensor at will, but it is more preferable that some or all of the information be acquired through measurements that are performed periodically and automatically by a program.

[0078] [Emotional state determination step] In the "emotional state determination step", the emotional state determination unit 41 determines the emotional state of the user from the "facial expression image" captured in the "facial expression capturing step".

[0079] The determination of the user's emotional state in the "emotional state determination step" is carried out through a two-stage process: obtaining a "primary emotional state score" through image analysis of the "facial expression image," and then correcting this "primary emotional state score" with "user-specific DNA information" previously acquired in the preceding "DNA information acquisition step." Here, in this specification, the "emotional state score" refers to a value that is a combination of the type of emotional state classified into categories such as "joy" and "sadness," and the intensity of that type of emotion (see Table 2).

[0080] As described above, the image analysis process of the "facial expression image" in the first stage of the "emotional state determination step" can be performed using a machine learning information processing device having a neural network (particularly a "deep learning image recognition device") or various other types of artificial intelligence (AI). Specifically, this image analysis process detects an emotional state score (primary emotional state score) that is a combination of the type of emotion of the user and its intensity, as shown in Table 2 below, for example. For example, if the extracted facial expression is recognized as an expression of "joy," the emotional intensity is further determined according to the degree of match of feature points, etc. with the facial expression showing the strongest emotion of "joy," and the "primary emotional state score" is calculated.

[0081] [Table 2]

[0082] The process of further correcting the above-mentioned "primary emotional state score" with the user's unique DNA information is carried out by correcting the "primary emotional state score" with a score related to the user's temperament that can be obtained from DNA information, as shown in Table 3 below, for example. Even if the images of human facial expressions are the same, the actual intensity of the emotion varies depending on each person's temperament. Therefore, by correcting the "primary emotional state score" as the result of image analysis with each person's temperament, it is possible to calculate a corrected emotional state score that is more suitable for each user.

[0083] [Table 3]

[0084] [Advice generation step] In the "advice generation step," the advice generation unit 42 generates personalized advice for each individual user using the user-specific DNA information acquired in the "DNA acquisition step" and the user-specific "auxiliary information" acquired in the auxiliary information acquisition step as input values.

[0085] The generation of advice in the "advice generation step" can also be performed in a manner in which advice obtained from DNA information and auxiliary information (primary advice) is corrected with the user's own "emotional state score" obtained through two-stage processing in the "emotional state determination step," and this corrected advice is output as corrected advice corrected according to the user's emotional state. Specific methods for correcting advice in this case are as described above, but as an example, as shown in Tables 4 and 5 below, when the user is feeling strong anger or deep sadness, the level of demands made on the user in the advice to that user can be lowered, and conversely, when the user is feeling great joy, the level of the demands can be raised.

[0086] [Table 4]

[0087] [Table 5]

[0088] [Avatar image generation step] In the "avatar image generation step", avatar image generation unit 43 generates an avatar image capable of variably displaying facial expressions corresponding to a plurality of emotions. In generating the avatar image, an appropriate facial expression and / or voice of the avatar image is selected based on the emotional state score obtained in the "emotional state determination step". In selecting the facial expression and / or voice of the avatar image, the content of the advice generated by advice generation unit 42 may also be taken into consideration, and the facial expression and / or voice of the avatar image may be selected based on a combination of the emotional state (emotional state score) and the content of the advice.

[0089] Furthermore, in the "avatar image generation step", it is preferable to extract the amount of change in the emotional state score between the time before and the time when the advice is output, and adjust the facial expression and / or voice by reselecting the facial expression and / or voice of the avatar image based on the amount of change.

[0090] 3 and 4 show examples of avatar images that can variably display facial expressions corresponding to a plurality of emotions generated in the "avatar image generation step." The avatar in FIG. 3 is assumed to represent a case where the user's emotional state score is average or in a state of joy, while the avatar in FIG. 4 is assumed to represent a case where the user's emotional state score is in a state of sadness or anger. By approximating the avatar's facial expression and / or voice, i.e., the manner of pseudo-emotional expression, to the user's emotional state, an effect of increasing the mental distance and emotional intimacy with the user is achieved.

[0091] [Advice Output Step] In the "advice output step", advice personalized for each user, generated in the "advice generation step" via the avatar image generated in the "avatar image generation step", is output to the user by the advice output unit 50. [Explanation of symbols]

[0092] 1. Personal advice system 10. Photography Department 20 DNA information acquisition department 30 Auxiliary information acquisition section 40 Processing unit 41 Emotional state determination unit 42 Advice Generation Unit 43 Avatar image generation unit 50 Advice output section

Claims

1. The photography department and a DNA information acquisition unit that acquires DNA information unique to a user; an auxiliary information acquisition unit that acquires auxiliary information that is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines an emotional state of the user from the facial expression image of the user captured by the image capture unit; an advice generation unit that generates advice for the user by performing a calculation process using the DNA information and the auxiliary information as input values; an avatar image generation unit that generates an avatar image capable of variably displaying facial expressions and / or voices corresponding to a plurality of emotions; an advice output unit that outputs the advice via the avatar image, the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information; the avatar image generation unit selects a facial expression and / or a voice of the avatar image based on the emotional state score. Personal advice delivery device.

2. the avatar image generation unit adjusts a facial expression and / or a voice of the avatar image selected based on the emotional state score in accordance with the content of the advice. The personal advice providing device according to claim 1 .

3. the emotional state determination unit extracts a change in the emotional state score between a time point before and a time point when the advice is output; the avatar image generation unit reselects a facial expression and / or a voice of the avatar image based on the amount of change; 3. The personal advice providing device according to claim 1 or 2.

4. The photography department and a DNA information acquisition unit that acquires DNA information unique to a user; an auxiliary information acquisition unit that acquires auxiliary information that is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines an emotional state of the user from the facial expression image of the user captured by the image capture unit; an advice generation unit that generates advice for the user by performing a calculation process using the DNA information and the auxiliary information as input values; an avatar image generation unit that generates an avatar image capable of variably displaying facial expressions and / or voices corresponding to a plurality of emotions; an advice output unit that outputs the advice via the avatar image, the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information; the avatar image generation unit selects a facial expression and / or a voice of the avatar image based on the emotional state score. Personal advice delivery system.

5. the avatar image generation unit adjusts a facial expression and / or a voice of the avatar image selected based on the emotional state score in accordance with the content of the advice. The system for providing personal advice according to claim 4.

6. the emotional state determination unit extracts a change in the emotional state score between a time point before and a time point when the advice is output; the avatar image generation unit reselects a facial expression and / or a voice of the avatar image based on the amount of change; 6. The personal advice providing system according to claim 4 or 5.

7. 6. The personal advice providing system according to claim 4, a process in which the emotional state determination unit determines the emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information; selecting a facial expression and / or a voice of the avatar based on the emotional state score in the avatar image generation unit; A program for a personal advice providing system.

8. The photography department and a DNA information acquisition unit that acquires DNA information unique to a user; an auxiliary information acquisition unit that acquires auxiliary information that is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination unit that determines an emotional state of the user from the facial expression image of the user captured by the image capture unit; an advice generation unit that generates advice for the user by performing a calculation process using the DNA information and the auxiliary information as input values; a robot control unit that generates an emotion designation signal that designates an expression and / or voice output mode for a robot that is capable of variably outputting expressions and / or voices corresponding to a plurality of emotions; an advice output unit comprising the robot, the emotional state determination unit determines an emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information; the robot control unit selects the emotion designation signal based on the emotional state score. Personal advice delivery system.

9. the robot control unit adjusts the emotion designation signal selected based on the emotional state score in accordance with the content of the advice. The system for providing personal advice according to claim 8.

10. the emotional state determination unit extracts a change in the emotional state score between a time point before and a time point when the advice is output; the robot control unit reselects the emotion designation signal based on the amount of change.

10. The system for providing personal advice according to claim 8 or 9.

11. 10. The personal advice providing system according to claim 8, a process in which the emotional state determination unit determines the emotional state score by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information; a process of selecting the emotion designation signal based on the emotional state score in the robot control unit; A program for a personal advice providing system.

12. a facial expression capturing step in which a capturing unit captures a facial expression image of a user; a DNA information acquisition step in which a DNA information acquisition unit acquires the user's unique DNA information; an auxiliary information acquisition step in which an auxiliary information acquisition unit acquires auxiliary information which is biometric information and / or environmental information specific to the user other than the DNA information; an emotional state determination step in which an emotional state determination unit determines the emotional state of the user from the facial expression image; an advice generating step in which an advice generating unit generates advice for the user by performing a calculation process using the DNA information and the auxiliary information as input values; an avatar image generating step in which an avatar image generating unit generates an avatar image capable of variably displaying facial expressions and / or voices corresponding to a plurality of emotions; an advice output step in which an advice output unit outputs the advice via the avatar image; In the emotional state determination step, the emotional state score is determined by correcting a primary emotional state score obtained by image analysis processing of the facial expression image with the DNA information, In the avatar image generating step, a facial expression and / or a voice of the avatar image is selected based on the emotional state score. How to provide personal advice.

13. In the avatar image generating step, a facial expression and / or a voice of the avatar image selected based on the emotional state score is adjusted in accordance with the content of the advice. The method for providing personal advice according to claim 12.

14. In the emotional state determination step, a change amount of the emotional state score between a time point before and a time point when the advice is output is extracted, In the avatar image generating step, a facial expression and / or a voice of the avatar image is reselected based on the amount of change.

14. The method for providing personal advice according to claim 12 or 13.

Images