Recommendation device, recommendation method, and recommendation program

Abstract

To improve well-being of an inhabitant through the execution of his or her behavior while guiding his or her behavior to an activity or an action recommended by a request source.SOLUTION: A recommendation server 10 estimates an emotion of a user and the sense of value of the user on the basis of voice data of the user, character data, an image obtained by capturing the face of the user, and / or biological information of the user. The recommendation server 10 estimates an action which matches the emotion of the user and the sense of value of the user and is expected that the user becomes an excellent state, on the basis of the emotion of the user and the sense of value of the user. The recommendation server 10 transmits recommendation information including a behavior corresponding to the action in an action list recommended by a request source, and an emotion sharing content to a user terminal. The recommendation server 10 causes a device used by the user who performs the behavior, to output information based on sensory data of the user having experienced the behavior or a person different from the user.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a recommendation device, a recommendation method, and a recommendation program. [Background technology]

[0002] Various initiatives are being undertaken under the guidance of government authorities to improve the well-being of people (a good physical, mental and social state). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Cabinet Office, "Well-being Initiatives", [Retrieved August 28, 2023], Internet<URL:https: / / www5.cao.go.jp / keizai2 / wellbeing / index.html> Summary of the Invention [Problem to be solved by the invention]

[0004] However, while consumers want to live a comfortable life, they do not know their current emotions or what actions will make them feel comfortable. In response to this, for example, each ministry and agency hopes that consumers will choose the activities and actions that they recommend as experiences that will contribute to improving their well-being.

[0005] For this reason, it is hoped that the well-being of consumers will be improved by guiding the behavior of each consumer toward the activities and behaviors recommended by each ministry and agency, while at the same time changing each consumer's current emotions in a constantly comfortable direction in response to the implementation of these activities and behaviors.

[0006] The present invention has been made in consideration of the above, and aims to provide a recommendation device, a recommendation method, and a recommendation program that guide a consumer's actions toward activities or behaviors recommended by a requester, and improve the consumer's well-being by carrying out such actions. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the recommendation device of the present invention is characterized by having an estimation unit that estimates the user's emotions and values ​​based on the user's voice data, text data input by the user, an image of the user's face, and / or biometric information of the user; an estimation unit that estimates an action that is in line with the user's emotions and values ​​and is expected to put the user in a good state based on the user's emotions and values ​​estimated by the estimation unit; a recommendation unit that transmits recommendation information to a user terminal used by the user, the recommendation information including an action corresponding to the action from a list of actions recommended by a requester and sensory sharing content that is expected to increase the motivation to perform the action when the user performs the action; and a sharing control unit that outputs information based on sensory data of the user who has experienced the action or a person different from the user from a device used by the user who performs the action. Effect of the Invention

[0008] According to the present invention, the behavior of a consumer can be guided to an activity or behavior recommended by a requester, and by carrying out this behavior, the consumer's well-being can be improved. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a recommendation system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of the user terminal shown in FIG. [Diagram 3]FIG. 3 is a block diagram showing an example of the configuration of the recommendation server shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of the emotion / value master. [Diagram 5] FIG. 5 is a diagram showing an example of a list of actions to be referred to by the emotion / value fluctuation point estimation unit. [Figure 6] FIG. 6 is a diagram showing an example of a list of actions recommended by a request source. [Figure 7] FIG. 7 is a diagram showing an example of a list of actions recommended by a request source. [Figure 8] FIG. 8 is a diagram showing an example of a list of actions recommended by a request source. [Figure 9] FIG. 9 is a diagram showing the data structure of data stored in the sense sharing database (DB). [Figure 10] FIG. 10 is a diagram showing an example of the menu screen of the bulletin board. [Figure 11] FIG. 11 is a sequence diagram illustrating a processing procedure of the recommendation process according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating the improvement of a user's well-being by the recommendation system shown in FIG. [Figure 13] FIG. 13 is a diagram illustrating an example of a computer that realizes each device of the recommendation system by executing a program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are indicated by the same reference numerals.

[0011] [Embodiment Mode] In the embodiment, among the activities and behaviors recommended by the requester, the system recommends actions that are in line with the emotions and values ​​of the user (consumer) and are expected to put the user in a good state. In addition, in the embodiment, the system has the user share sensory data of people who have experienced the recommended behavior, and guides the user to the activities and behaviors recommended by the requester, thereby improving the user's well-being (being in a good state physically, mentally, and socially) by carrying out the behavior.

[0012] [Recommendation system] Next, a recommendation system according to an embodiment will be described. Fig. 1 is a diagram showing an example of the configuration of the recommendation system according to an embodiment.

[0013] As shown in FIG. 1, a recommendation system 100 according to an embodiment includes a recommendation server 10 (recommendation device), a user terminal 30 used by a user who is to receive recommended activities or behaviors (hereinafter referred to as behaviors), a wearable device 40 worn by the user, and shared output devices 50-1 and 50-2 worn by the user.

[0014] The requesting server 20 to which the recommendation server 10 is connected transmits a list of recommended actions and requests the recommendation server 10 to guide the user to the recommended actions. Requesters having the requesting server 20 are, for example, various government ministries and agencies (Ministry of the Environment, Ministry of Health, Labor and Welfare, Ministry of Education, Culture, Sports, Science and Technology, Ministry of Agriculture, Forestry and Fisheries, Ministry of Land, Infrastructure, Transport and Tourism, etc.), as well as organizations such as local governments, local communities, businesses, medical institutions, schools, or individuals.

[0015] Based on the list of actions transmitted from the requesting server 20, the recommendation server 10 recommends to the user terminal 30 actions that are in line with the user's emotions and values ​​and are expected to put the user in a good state, from the list of actions recommended by the requesting server, based on the list of actions transmitted from the requesting server 20. In addition, the recommendation server 10 guides the user to the activities and actions recommended by the requesting server by having the user share sensory data of a person who has experienced the recommended action (the user himself / herself or another person different from the user), thereby improving the user's well-being by carrying out the action.

[0016] The user terminal 30 is, for example, a smart device such as a smartphone or a tablet, and has a position detection function, etc. The user terminal 30 controls the wearable device 40 and the sharing output devices 50-1 and 50-2. The user terminal 30 communicates with the recommendation server 10 via a network.

[0017] The wearable device 40 has various sensors, and transmits sensor information of the various sensors by wireless communication to the user terminal 30. The wearable device 40 has, for example, a vital sensor and an acceleration sensor, and measures data such as the heart rate, heart rate variability, blood pressure, cardiac potential, body temperature, respiratory rate, and the like, as well as the movement, number of steps, and the like, of the user wearing the wearable device 40. For example, the wearable device 40 may be a wristwatch type or a ring type.

[0018] The sharing output devices 50-1 and 50-2 are devices for allowing users wearing the sharing output devices 50-1 and 50-2 to share sensory data of people who have experienced an action recommended by the recommendation server 10. In FIG. 1, as an example, tactile sharing in which muscles are electrically stimulated to contract, and visual and auditory sharing by projecting an image using VR (Virtual Reality) or the like and outputting sound from a speaker are described. The sharing output devices 50-1 and 50-2 perform processing for allowing users to share sensory data such as tactile, visual, and auditory data under the control of the user terminal 30.

[0019] The shared output device 50-1 (electrical stimulation device) performs electrical stimulation on muscles, contracting the muscles and providing a tactile sensation to the wearer. For example, the shared output device 50-1 also serves as a muscle displacement sensing device that acquires muscle displacement data of the user who wears it. The shared output device 50-2 is, for example, AR glasses that can display virtual content.

[0020] The shared output device 50-2 is, for example, a pair of glasses-type VR glasses that are worn on the user's head. The shared output device 50-2 has an imaging function, an audio input / output function, a communication function, and a display function, and captures an image of a real space and displays the captured image of the real space on a display corresponding to a lens portion. Note that the imaging, audio input / output, and image display performed by the shared output device 50-2 can also be performed by the user terminal 30. Hereinafter, the shared output devices 50-1 and 50-2 are collectively referred to as the shared output device 50.

[0021] [User device] Next, a description will be given of the user terminal 30. Fig. 2 is a block diagram showing an example of the configuration of the user terminal 30 shown in Fig. 1. As shown in Fig. 2, the user terminal 30 has a communication unit 31, an input / output unit 32, an imaging unit 33, a storage unit 34, and a control unit 35.

[0022] The communication unit 31 controls communication related to various information. For example, the communication unit 31 controls communication between the wearable device 40 and the sharing output device 50, and communication between the communication unit 31 and the recommendation server 10.

[0023] The input / output unit 32 outputs various information and receives input from the user. The input / output unit 32 includes, for example, a liquid crystal screen, a touch panel, a speaker, a microphone, and the like. The input / output unit 32 outputs the recommendation information transmitted from the recommendation server 10 by image and / or sound. The recommendation information includes the recommended action and sensory sharing content that is expected to increase the motivation to perform the action when the user performs the action. The sensory sharing content is tactile, visual, and auditory content that is shared by the user via the sharing output device 50 and the user terminal 30, and is electrical stimulation to muscles, images, sounds, and the like.

[0024] The imaging unit 33 is configured by an imaging module, and captures an image according to a user's operation or under the control of a recommended application control unit 351 (described later). The imaging unit 33 captures an image of the face of the user during conversation according to the control of the recommended application control unit 351.

[0025] The storage unit 34 stores data and programs necessary for various processes by the control unit 35. For example, the storage unit 34 is a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 34 stores position data 341 detected by a position detection unit 352 (described later) and biological information 342 transmitted from the wearable device 40.

[0026] The control unit 35 has an internal memory for storing programs that define various processing procedures and required data, and executes various processes using these. Here, the control unit 35 is an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 35 has a recommendation application control unit 351, a position detection unit 352, a biometric information acquisition unit 353, and a sensory data output control unit 354.

[0027] The recommendation application control unit 351 is an application for activity recommendation installed in the user terminal 30. Based on communication with the recommendation server 10, the recommendation application control unit 351 controls dialogue processing performed for activity recommendation, output of recommendation information by the user terminal 30, various processes such as sensory data output and sharing control, and output processing of an activity execution history by a user. As described later, the dialogue processing is performed by transmitting and receiving data such as voice, video, and text to and from the recommendation server 10 via an avatar, a synthesized person image, a chat screen, and the like. In addition, the recommendation application control unit 351 transmits position data of the user terminal 30 and biometric information detected by the wearable device 40 to the recommendation server 10.

[0028] The position detection unit 352 detects the current position of the user terminal 30 using a positioning system such as a Global Positioning System (GPS), and outputs position data indicating the detected position.

[0029] The biological information acquisition unit 353 collects, from the wearable device 40, biological information detected by the wearable device 40.

[0030] The sensory data output control unit 354 controls the sharing output device 50 or the input / output unit 32 to execute a sharing process for sensory data such as touch, vision, and hearing. This allows a user wearing the sharing output device 50 to share sensory data such as touch, vision, and hearing of a person who has experienced an action recommended by the recommendation server 10.

[0031] [Recommendation Server] Fig. 3 is a block diagram showing an example of the configuration of the recommendation server 10 shown in Fig. 1. The recommendation server 10 is a server device that is realized by, for example, loading a predetermined program into a computer including a ROM (Read Only Memory), a RAM, a CPU, etc., and having the CPU execute the predetermined program. The recommendation server 10 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network, etc.

[0032] The recommendation server 10 has a communication AI (Artificial Intelligence) unit 11 (communication unit), an emotion / value estimation unit 12 (estimation unit), an emotion / value change point estimation unit 13 (estimation unit), a recommendation unit 14, a sense sharing unit 15 (sharing control unit), a self-awareness information generation unit 16, a posting environment providing unit 17, and a chat environment providing unit 18.

[0033] The communication AI unit 11, for example, uses a trained dialogue model (machine learning model) to communicate with the user via the user terminal 30. The communication AI unit 11 communicates with the user via an avatar, a synthesized human image, and a chat screen. The dialogue is carried out by repeatedly outputting voice, gesture video, and text from the user terminal 30, and receiving the user's voice, gesture video, and text in response to the output via the user terminal 30. When the user expresses his or her intention using sign language, the user's intention is acquired by analyzing a video capturing the user's hands, fingers, face, etc.

[0034] The communication AI unit 11 acquires data transmitted from the user terminal 30, such as voice data uttered by the user, character data input by the user to the user terminal 30, and / or an image of the user's face (hereinafter referred to as a facial image). The communication AI unit 11 continues to have a dialogue with the user to follow up on the user before, during, and / or after the recommended action is performed.

[0035] The communication AI unit 11 receives voice data uttered by the user, character data input by the user to the user terminal 30, the user's action result log, and the user's face image as input. The communication AI unit 11 generates conversation content. For example, the conversation content may be content that naturally encourages empathy. In addition, the conversation content may be content that follows up before, during, and / or after the recommended action is performed. In addition, the conversation content may be content that indicates the user's emotions and / or values, or content that indicates an update of the definition of the user's emotions and values ​​based on the results of performing the recommended action.

[0036] The emotion and value estimation unit 12 estimates the emotion and value of the user based on the user's voice data, character data input by the user, the user's face image, and / or the user's biometric information acquired by the communication AI unit 11. For example, the emotion and value estimation unit 12 estimates the emotion and value of the user before, during, and / or after the recommended action based on the user's voice data, character data input by the user, the user's face image, and / or the user's biometric information acquired by the dialogue between the communication AI unit 11 and the user. The emotion and value estimation unit 12 classifies the type of the user's facial expression from the user's face image and biometric information, and estimates the user's emotion based on the type of facial expression. The emotion and value estimation unit 12 compares the type of emotion with the dialogue portion to estimate the user's value.

[0037] The emotion and value estimation unit 12 may estimate the emotion and value of the user using the emotion and value master. In this case, the emotion and value estimation unit 12 receives the user's voice data and / or character data input by the user and the emotion and value master as input, and outputs an estimation result of the user's current emotion and a classification result of the current value. FIG. 4 is a diagram showing an example of the emotion and value master. The emotion and value master is created using the classification shown in FIG. 4 (Reference 1). Reference 1: Junji Watanabe, "Our Well-Being Card", [Searched on September 12, 2023], Internet<URL:https: / / socialwellbeing.ilab.ntt.co.jp / tool_measure_wellbeingcard.html>

[0038] For example, as an emotion master, voice waveform patterns and spoken words for classifying human emotions into positive or negative and their intensity are registered in advance. Then, feature information (voice waveform patterns, spoken words, etc.) for estimating joy, anger, sadness, and happiness from the voice waveform patterns and spoken words is registered in advance in association with the emotion master. The emotion / value estimation unit 12 converts the user's voice data and / or character data input by the user into features and compares them with the registered feature information to estimate whether the user's emotion is positive or negative and its intensity.

[0039] Then, words associated with the value categories are registered in advance as a value master, as shown in the list in Fig. 4. The emotion and value estimation unit 12 breaks down the user's voice data and / or the character data input by the user into words based on the value categories shown in Fig. 4, and when a word associated with a value category appears among the words, it classifies the value category to which this word is associated.

[0040] For example, "I (Pattern A)" is a value classification based on factors related to one's own personal factors, "WE (Pattern B)" is a factor related to relationships with specific others, "SOCIETY (Pattern C)" is a factor related to relationships with a broader society including an unspecified number of others, and "UNIVERSE (Pattern D)" is a factor with a perspective that includes everything that transcends them. The emotion / value classification unit 12 classifies the user's current values ​​based on which of these value classifications the words included in the user's voice data and / or the text data input by the user more often fall into. The emotion / value classification unit 12 may also use biometric information transmitted from the user terminal 30 to classify the user's emotions and values.

[0041] The emotion / value fluctuation point estimation unit 13 estimates behavior based on the user's emotion and user's values ​​estimated by the emotion / value estimation unit 12. The estimated behavior is behavior that is in line with the user's emotion and user's values ​​and is expected to put the user in a good state. The emotion / value fluctuation point estimation unit 13 estimates behavior that fits the user's emotion and values.

[0042] Fig. 5 is a diagram showing an example of a list of actions referred to by the emotion / value fluctuation point estimation unit 13. In Fig. 5, recommended actions are associated with emotion and value patterns. In this way, each action in the list of actions is tagged according to the type that contributes to improving the Well-Being of each user.

[0043] The emotion / value fluctuation point estimation unit 13 refers to the behavior list in Fig. 5 and extracts behaviors that match the user's emotion and values. If the user's emotion is "positive" and the values ​​are "Pattern C", the emotion / value fluctuation point estimation unit 13 estimates the following behaviors: "plant removal", "activity in mountainous areas", and "health check".

[0044] Alternatively, the emotion / value fluctuation point estimation unit 13 estimates behavior using a trained machine learning model that outputs behavior that is expected to bring the user into a good physical, mental, and social state when the user's emotions and values ​​are input. For example, the trained machine learning model estimates "behavior of moving the body because well-being is likely to improve by moving the body" as behavior.

[0045] The recommendation unit 14 creates recommendation information including an action corresponding to the action in the list of actions recommended by the requester, and a sense-sharing content that is expected to improve the motivation of the user to perform the action when the user performs the action. Then, the recommendation unit 14 transmits the recommendation information to the user terminal 30 used by the user and causes it to be output. The action recommended by the recommendation unit 14 is an action recommended by the requester, and is a user action that contributes to improving well-being while following the user's emotions and values. In addition, the recommendation unit 14 may recommend multiple actions so that the user can flexibly select.

[0046] 6 to 8 are diagrams showing examples of a list of actions recommended by a requester. For example, as shown in FIG. 6, when the requester is a ministry or agency K, the actions include, for example, eradicating invasive plants from rivers and reducing household waste. As shown in FIG. 7, when the requester is a ministry or agency L, the actions include, for example, walking, stretching, yoga, etc. As shown in FIG. 8, when the requester is a ministry or agency N, the actions include, for example, attending a qualification course. Each action list is transmitted from the requester's server 20 and stored in the action DB 141.

[0047] The recommendation unit 14 estimates the action to be recommended, the sense sharing content, the timing to perform the action to be recommended, and the sense sharing timing, taking into consideration the user's location information, the user's schedule, weather information, traffic information, and the like.

[0048] For example, when the user's actions estimated by the emotion / value fluctuation point estimation unit 13 are "food removal," "activity in mountainous areas," and "health check," and the user is within walking distance of a river on a sunny day, the recommendation unit 14 estimates "removal of invasive plants from the river" as the recommended action. If a shared output device 50 can be rented on the route to the river, the recommendation unit 14 estimates touch, vision, and hearing as the sensory shared content. The sensory shared content is output by the shared output device 50, and when the sensory shared content is visual or auditory, it can also be output from the user terminal 30.

[0049] The recommendation unit 14 supports the user in forming a habit of the action by transmitting recommendation information recommending continuous performance of the recommended action to the user terminal 30 at a predetermined timing.

[0050] Furthermore, when the user performs an action a predetermined number of times and / or for a predetermined period of time, and / or when the user's emotions and values ​​change, the recommendation unit 14 transmits recommendation information including other actions different from the action and sense-sharing content corresponding to the other actions to the user terminal 30. When the user gets tired of an action, the recommendation unit 14 recommends another action to continue improving well-being.

[0051] The sense sharing unit 15 outputs information based on sensory data (sensor data) of the user who experienced the action or a person different from the user from a device used by the user via the user terminal 30. The sense sharing unit 15 uses sensory data when the user himself performs the action, or sensory data that has the same attributes as the user and corresponds to the action.

[0052] The sensory sharing unit 15 allows the user to share the sensation (e.g., touch) felt when performing the action by executing electrical stimulation to the user's muscles through the sharing output device 50-1 worn by the user. And / or, the sensory sharing unit 15 allows the user to share the sensation (e.g., hearing, vision) felt when performing the action by outputting voice, text, and / or image through the device used by the user (the sharing output device 50-2, the user terminal 30).

[0053] The sense sharing unit 15 receives as input a list of sense sharing contents and sensory data of the user or a person other than the user at the time of performing the action. The sense sharing unit 15, for example, measures sensor data (sensor data) of a large number of people who performed each action in advance and stores the data in the sense sharing DB 151 in association with the action and attributes.

[0054] Fig. 9 is a diagram showing the data structure of data stored in the sense sharing DB 151. As shown in Fig. 9, the sense sharing DB 151 stores, in association with each other, the identification information (name) of each action, the attributes (age, sex, etc.) of the person who performed the action, time-series sensory data acquired when performing the action, and identification information of content suitable for sensory data sharing. The sensory data includes muscle displacement data acquired via a muscle displacement sensing device worn by the person who performed the action, image data of a scene seen by the person who performed the action, audio data (including audio spoken by other people or an avatar) heard by the person who performed the action, or text data thereof, etc.

[0055] The sensory sharing unit 15 executes sensory sharing content prepared in advance according to the user's instructions. The sensory sharing unit 15 generates and outputs sensory highlights of the user when performing the action based on sensory data when performing the action. The sensory highlights are external stimuli for approaching a state of feeling comfortable through the experience. The external stimuli are electrical stimulation to muscles, sounds, images, etc. The output device of the sensory data is, for example, a device capable of outputting content selected by the user from the shared contents recommended by the recommendation unit 14.

[0056] The timing of the sense sharing is, for example, a timing selected by the user from among the timings recommended by the recommendation unit 14. For example, the sense sharing unit 15 allows the user before performing the action to share the senses of other people who have experienced the action. This allows the user to virtually experience the action that he or she is about to perform. Therefore, the user can confirm to what extent the user is expected to be in a good condition, and can positively consider performing the action.

[0057] In addition, the sensation sharing unit 15 allows the user who has performed the action to share his / her own sensations after the action. This allows the user to reflect on how much the action has improved his / her condition, and motivates him / her to continue the action.

[0058] In addition, the sense sharing unit 15 allows the user performing the action to share the senses of other people who have experienced the action. For example, the sense sharing unit 15 allows the user to perform the action while sharing the senses of other people who have experienced the action, so that the action can be performed smoothly.

[0059] The estimation results of the user's emotions and values ​​by the emotion / value estimation unit 12 are accumulated before, during, and after the user performs an action as the dialogue between users continues.

[0060] The self-recognition information generating unit 16 generates time-series data indicating changes in the user's emotions and values ​​based on the accumulated user's emotions and values ​​before, during, and / or after the user performs an action. For example, the self-recognition information generating unit 16 generates time-series data when the user's values ​​change, when a predetermined period of time has elapsed since the start of the action, or when the action has been performed a predetermined number of times.

[0061] The self-awareness information generating unit 16 transmits time-series data indicating the changes in the user's emotions and values ​​to the user terminal 30 and causes it to be output. The self-awareness information generating unit 16 generates, as time-series data, summary information indicating the changes in the user's emotions and values, clearly indicating the timing of the action taken, and also generates a short movie indicating the changes in the emotions and values.

[0062] The user can check time series data showing the fluctuations of the user's emotions and values ​​while also referring to the event participation history and the action history for performing the action. This allows the user to understand how their emotions and values ​​have changed in light of the action they performed. The user can then realize that their daily well-being has improved as a result of the action.

[0063] The posting environment providing unit 17 provides a posting environment that connects each request source and a user on the network. For example, when a request source posts a problem or a request, the posting environment providing unit 17 posts the problem or the request in a bulletin board format.

[0064] FIG. 10 is a diagram showing an example of a menu screen of a bulletin board. In the menu screen M1 of FIG. 10, for example, when a user of a circle E posts a problem or a request, the problem or request is displayed in a list. A user can search for each post in the menu screen M1 and refer to the details of each post. When there is an action that the user himself / herself wants to take, he / she can apply by selecting the corresponding post field in the menu screen M1. The posted problem or request is resolved by the user's application or action.

[0065] The posting environment providing unit 17 displays a message board for posting problems and requests on the terminal device used by the requester or user, and also displays the results of applications from users. In addition, when the implementation date and time of the problem or request approaches, the posting environment providing unit 17 outputs a reminder to the user's terminal device.

[0066] The chat environment providing unit 18 provides a chat environment so that a chat can be conducted between the requester and the user. When character data and standard phrases for conversation are input from the user terminal 30 or the requester's terminal, the chat environment providing unit 18 displays them in the chat environment.

[0067] [Recommendation processing] Next, a description will be given of the recommendation process of the recommendation system 100 according to the embodiment. Fig. 11 is a sequence diagram showing the procedure of the recommendation process according to the embodiment.

[0068] A recommendation app is started in the user terminal 30 (step S1), and the user terminal 30 logs in to the recommendation server 10 (step S2). Then, a dialogue between the user and the recommendation server 10 is executed via communication between the user terminal 30 and the recommendation server 10 (step S3).

[0069] In step S3, the recommendation server 10 acquires voice data uttered by the user, character data input by the user to the user terminal 30, and / or a face image of the user (step S6). The user terminal 30 may transmit biometric information acquired by the wearable device 40 during the conversation to the recommendation server 10 (steps S4 and S5). The user terminal 30 may transmit position data of the user terminal 30 to the recommendation server 10.

[0070] The recommendation server 10 estimates the user's feelings and values ​​based on the voice data uttered by the user, the character data input by the user to the user terminal 30, the user's face image, and / or the user's biometric information during the conversation (step S7). The recommendation server 10 may also estimate the user's feelings and values ​​using the biometric information transmitted from the user terminal 30.

[0071] During login to the recommendation server 10, the recommendation server 10 and the user terminal 30 continue to execute the processes of steps S3 to S7 while continuing the dialogue of action follow-up with the user. As a result, the recommendation server 10 accumulates data on the user's emotions and values.

[0072] The recommendation server 10 predicts the user's behavior based on the user's feelings and values ​​estimated in step S7 (step S8).

[0073] The recommendation server 10 creates recommendation information including an action corresponding to the action from the list of actions recommended by the requester and a sense-sharing content that is expected to increase the motivation to perform the action when the user performs the action (step S9).

[0074] The recommendation server 10 transmits recommendation information to the user terminal 30 (step S10), and recommends an action and a sense sharing content via the user terminal 30 (step S11).

[0075] When the recommended action and sense sharing content are selected by the user operating the user terminal 30 (step S12), the selection result of step S12 is transmitted to the recommendation server through the operation of the user terminal 30 (step S13).

[0076] For example, the recommendation server 10 performs a process of having the user share sensory data at the time of performing an action before performing the action in order to improve the user's motivation for the action. In this case, the recommendation server 10 refers to sensory data of other people who have experienced the same action from the sensory data stored in the sensory sharing DB 151 (step S14) and generates a sensory highlight (step S15). The recommendation server 10 transmits sensory sharing control information to the user terminal 30 (step S16) and causes the user terminal 30 to execute a sensory data output process by the sharing output device 50 (step S17).

[0077] When the user terminal 30 receives an input for starting to perform an action by a user operation (step S18), it controls the sharing output device 50, acquires sensory data such as muscle displacement data of the user wearing the sharing output device 50, and transmits it to the recommendation server 10 (steps S19, S20). The recommendation server 10 associates the user's sensory data transmitted from the user terminal 30 with the user's identification information and the identification information of the action performed by the user, and stores them in the sensory sharing DB 151.

[0078] For example, the recommendation server 10 performs a process of having the user share sensory data at the time of performing an action in order to reflect on the user's action. In this case, the recommendation server 10 generates a sensory highlight based on the sensory data of the user who experienced the action (step S21). Then, the recommendation server 10 transmits sensory sharing control information to the user terminal 30 (step S22), and causes the user terminal 30 to execute output processing of the sensory data by the sharing output device 50 (step S23).

[0079] At a predetermined timing, the recommendation server 10 acquires the user's emotions and values ​​accumulated before, during, and / or after the user performs an action (step S24). The recommendation server 10 generates time-series data showing the fluctuations of the user's emotions and values ​​based on the data acquired in step S24 (step S25). The recommendation server 10 then transmits the time-series data showing the fluctuations of the user's emotions and values ​​to the user terminal 30 (step S26) and causes it to be output (step S27).

[0080] [Effects of the embodiment] FIG. 12 is a diagram illustrating the improvement of a user's well-being by the recommendation system 100 shown in FIG.

[0081] For example, consider a case where the user is vaguely dissatisfied with their daily life ((A) in FIG. 12). The user terminal 30 acquires the user's voice data, text data, and / or face image of the user from the initial greeting through a dialogue between the user and the communication AI unit 11, and transmits them to the recommendation server 10 ((1) in FIG. 12). The emotion / value estimation unit 12 estimates the user's emotions and values ​​based on the user's voice data, character data input by the user, and / or the user's face image. The communication AI unit 11 continues the dialogue with the user by having natural conversations and movements using an avatar or the like, and accumulates the estimation results of the user's emotions and values.

[0082] The recommendation server 10 then guides the user to the activity or behavior recommended by the request source, thereby making the user's performance of the behavior a habit (FIG. 12(B)).

[0083] Specifically, the recommendation server 10 acquires the user's voice data, text data, face image, and / or biometric information of the daily greeting from the dialogue between the communication AI unit 11 and the user, and continues to estimate the user's emotions and values ​​((2) in FIG. 12). The recommendation unit 14 recommends environmental conservation activities and sensory sharing content to the user from a list of actions recommended by the requester. The environmental conservation activities correspond to actions estimated based on the user's emotions and values.

[0084] In this case, the recommendation system 100 allows the user to share sensory data of other people during environmental conservation activities via the sharing output device 50 ((3) in FIG. 12). This increases the user's motivation to engage in environmental conservation activities, enabling them to take a proactive approach to environmental conservation activities.

[0085] After the environmental conservation activity is carried out, the recommendation server 10 continues to acquire the user's voice data, text data, and / or facial images through natural conversations and actions that follow the action via the communication AI unit 11, and to estimate the user's emotions and values ​​((4) in Figure 12).

[0086] Furthermore, the recommendation server 10 allows the user to share his / her own sensory data during the environmental conservation activity via the sharing output device 50 ((5) in FIG. 12). This allows the user to reflect on how much his / her condition has improved as a result of the environmental conservation activity, and motivates him / her to continue the environmental conservation activity in the future.

[0087] The recommendation server 10 makes the user's behavior (environmental conservation activities) a habit through the loop of (2) to (5) in Fig. 12. In other words, the recommendation system 100 can improve the user's daily well-being by guiding the user's behavior to the activities and behaviors recommended by the requester and carrying out these actions (Fig. 12 (C)). In this way, the recommendation system 100 can make the user feel that their life has naturally become more comfortable without placing a large burden on the user.

[0088] Therefore, according to the recommendation system 100, the user's actions can be guided to activities or behaviors recommended by the requester, and the user's well-being can be improved by performing these actions.

[0089] Furthermore, if the user becomes bored with environmental conservation activities, the recommendation unit 14 recommends other actions, allowing the user to explore other social behaviors that will make the user feel even more comfortable ((6) in FIG. 12). In this case, the recommendation unit 14 encourages the user to continue the environmental conservation activities while also recommending actions related to other themes ((7) in FIG. 12). Therefore, according to the recommendation system 100, it is possible to guide the user to other activities or actions recommended by the requester so that the user's well-being will continue to improve.

[0090] For example, when the requester is the Ministry of the Environment, the recommendation system 100 recommends actions that contribute to improving well-being in line with the user's emotions and values ​​from a list of environmental conservation activities (such as reducing household waste and preserving local rivers and mountains) recommended by the Ministry of the Environment. When reducing household waste becomes a habit, it is assumed that multiple actions will be recommended, such as recommending mountain preservation activities.

[0091] Furthermore, when the requester is the Ministry of Health, Labor and Welfare, the recommendation system 100 recommends actions that contribute to improving well-being in line with the user's emotions and values ​​from a list of preventive medical actions (such as improving lifestyle habits and regular health checkups) recommended by the Ministry of Health, Labor and Welfare. It is assumed that multiple actions will be recommended, such as recommending exercising once a week when a moderate diet has become a habit.

[0092] Furthermore, when the requester is the Ministry of Education, Culture, Sports, Science and Technology, the recommendation system 100 recommends actions that contribute to improving well-being in line with the user's emotions and values ​​from a list of actions for reskilling (such as learning support and job-hunting / career change support) recommended by the Ministry of Education, Culture, Sports, Science and Technology. It is assumed that multiple actions will be recommended, such as encouraging the acquisition of another skill after the acquisition of one skill.

[0093] [System configuration of the embodiment] Each component of each device of the recommendation system 100 is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of each device of the recommendation system 100 is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0094] Furthermore, all or any part of each process performed in each device of the recommendation system 100 may be realized by a CPU, a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU. Furthermore, each process performed in each device of the recommendation system 100 may be realized as hardware using wired logic.

[0095] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the process procedures, control procedures, specific names, and information including various data and parameters described above and shown in the drawings can be changed as appropriate unless otherwise specified.

[0096] [program] 13 is a diagram showing an example of a computer in which each device of the recommendation system 100 is realized by executing a program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. Each of these units is connected by a bus 1080.

[0097] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0098] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of each device of the recommendation system 100 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing the same process as the functional configuration of each device of the recommendation system 100 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0099] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary, and executes the program.

[0100] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0101] Although the embodiment of the invention made by the present inventor has been described above, the present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to the present embodiment. In other words, other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiment are all included in the scope of the present invention. [Explanation of symbols]

[0102] 10 Recommendation Server 11 Communication AI Department 12 Emotion and Value Estimation Department 13 Emotion and Value Change Point Prediction Section 14 Recommendation Department 15 Sensory Sharing 16 Self-recognition information generation unit 17 Posting Environment Provision Department 18 Chat Environment Provision Department 20 Requesting Server 30 User terminal 31 Communications Department 32 Input / output section 33 Imaging unit 34 Storage section 35 Control section 40 Wearable Devices 50,50-1,50-2 Shared output device 100 Recommendation System 141 Behavioral Database (DB) 151 DB for sensory sharing

Claims

1. an estimation unit that estimates the user's emotions and values ​​based on voice data of a user, character data input by the user, an image of the user's face, and / or biometric information of the user; an estimation unit that estimates an action that is in line with the user's emotions and the user's values ​​and is expected to put the user in a good state, based on the user's emotions and the user's values ​​estimated by the estimation unit; a recommendation unit that transmits recommendation information to a user terminal used by the user, the recommendation information including an action corresponding to the action in a list of actions recommended by a request source and a sense sharing content that is expected to improve the motivation to perform the action when the user performs the action; A shared control unit that outputs information based on sensory data of the user who has experienced the action or a person other than the user from a device used by the user who performs the action; A recommendation device comprising:

2. The recommendation device of claim 1, characterized in that the sharing control unit allows the user to share the sensations felt when performing the action by performing electrical stimulation on the user's muscles using an electrical stimulation device worn by the user, and / or by outputting audio, text and / or images using a device used by the user.

3. A communication unit that uses a dialogue model to engage in dialogue with the user to follow up on the user before, during, and / or after the action is taken, The recommendation device described in claim 1, characterized in that the estimation unit estimates the user's emotions and the user's values ​​before, during, and / or after performing the action based on the user's voice data and / or character data input by the user obtained through a dialogue between the communication unit and the user, an image of the user's face during the dialogue, and / or biometric information of the user during the dialogue.

4. The recommendation device according to claim 3, further comprising a generation unit that generates time series data indicating changes in the user's emotions and the user's values ​​based on the user's emotions and the user's values ​​before, during, and / or after the user performs the action, and transmits the data to the user terminal.

5. The recommendation device according to claim 4 , wherein the generation unit generates, as the time-series data, a short movie showing changes in the user's emotions and values.

6. The recommendation device according to claim 1 , wherein the recommendation unit transmits, at a predetermined timing, recommendation information recommending continuous performance of the action to the user terminal.

7. The recommendation device described in claim 1, characterized in that the recommendation unit sends recommendation information to the user terminal when the user performs the action a predetermined number of times and / or for a predetermined period of time, and / or when the user's emotions and values ​​change, the recommendation information including other actions different from the action and sensory sharing content corresponding to the other actions.

8. A recommendation method executed by a recommendation device, A step of estimating the emotions and values ​​of the user based on voice data of a user, character data input by the user, an image of the face of the user, and / or biological information of the user; A step of predicting an action that is in line with the user's emotions and the user's values ​​and is expected to put the user in a good state based on the user's emotions and the user's values; A step of transmitting recommendation information including an action corresponding to the action from a list of actions recommended by a request source and a sense sharing content that is expected to increase the motivation to perform the action when the user performs the action to a user terminal used by the user; A step of outputting information based on sensory data of the user who has experienced the action or a person different from the user from a device used by the user who performs the action; A recommendation method comprising:

9. A step of estimating the emotions and values ​​of the user based on voice data of a user, character data input by the user, an image of the face of the user, and / or biological information of the user; A step of predicting an action that is in line with the user's emotions and the user's values ​​and is expected to put the user in a good state based on the user's emotions and the user's values; transmitting, to a user terminal used by the user, recommendation information including an action corresponding to the action in a list of actions recommended by a request source and a sense sharing content that is expected to increase the motivation to perform the action when the user performs the action; outputting information based on sensory data of the user who has experienced the action or a person different from the user from a device used by the user who performs the action; A recommendation program for having a computer execute the above.

Images