Control system and program

The control system addresses the limitations of existing emotion identification systems by integrating user characteristics and emotional responses to generate personalized proposals, improving interaction quality through machine learning and emotional feedback adjustments.

JP2024158716APending Publication Date: 2024-11-08SOFTBANK GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023074132
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing emotion identification systems for robots fail to effectively integrate user characteristics, desires, and emotional responses to tailor personalized proposals, leading to suboptimal interaction experiences.

Method used

A control system that includes a diagnosis result acquisition unit, user characteristic acquisition unit, user desire acquisition unit, and proposal content generation unit, utilizing machine learning to generate personalized proposals based on image concept diagnosis, user characteristics, and user preferences, while adjusting content to enhance positive emotional responses.

Benefits of technology

The system enhances user engagement by dynamically adjusting proposals based on emotional transitions, ensuring a more positive interaction experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158716000001_ABST
    Figure 2024158716000001_ABST
Patent Text Reader

Abstract

To provide a control system and a program that output proposal contents to a user on the basis of a result of image consultant diagnosis of the user, feature of the user, and desire of the user.SOLUTION: In a control system, a server 100 includes: a diagnosis result acquisition unit that acquires a diagnosis result of image consultant diagnosis including at least one of color diagnosis, bone structure diagnosis, and face type diagnosis of a user who is talking to an electronic apparatus (robot 200); a user-feature acquisition unit that acquires feature of the user including at least one of voice volume of the user, voice tone of the user, and face expression of the user; a user-desire acquisition unit that acquires desire of the user including at least one of desired occupation of the user and a desired self-image of the user; a proposal content generation unit that generates, on the basis of the diagnosis result, the feature of the user, and the desire of the user, proposal contents for the user; and a control unit that causes the electronic apparatus to output the proposal contents to the user.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a control system and a program. [Background technology]

[0002] Patent Document 1 describes an emotion identification system that identifies the emotion of a robot. [Prior art document] [Patent documents] [Patent Document 1] JP 2017-199319 A Summary of the Invention [Means for solving the problem]

[0003] According to an embodiment of the present invention, a control system is provided. The control system may include a diagnosis result acquisition unit that acquires a diagnosis result of an image consulting diagnosis including at least one of a color diagnosis, a bone structure diagnosis, and a face type diagnosis of a user who is talking to an electronic device. The control system may include a user feature acquisition unit that acquires user features including at least one of the volume of the user's voice, the tone of voice, and a facial expression. The control system may include a user preference acquisition unit that acquires user preferences including at least one of a desired occupation and a self-image of the user. The control system may include a proposal content generation unit that generates a proposal content for the user based on the diagnosis result, the user features, and the user preferences. The control system may include a control unit that causes the electronic device to output the proposal content to the user.

[0004] The control system may include a storage unit that stores learning data that associates suggestions actually made by an image consultant to a target person, a diagnosis result of an image consulting diagnosis of the target person, user characteristics of the target person, and user preferences of the target person. The control system may include a learning execution unit that generates a learning model that receives the diagnosis result of an image consulting diagnosis of a user, the user characteristics of the user, and the user preferences of the user, and outputs a proposal content for the user, by machine learning using a plurality of learning data. The proposal content generation unit may generate the proposal content using the learning model. The learning execution unit may generate the learning model by executing the machine learning in which different weights are applied to the diagnosis result, the user characteristics, and the user preferences.

[0005] Any of the control systems may further include an emotion determination unit that determines a user emotion value indicating an emotion of the user who is conversing with the electronic device. The proposal content generation unit may adjust the proposal content based on a transition of the user emotion value of the user when the control unit is outputting the proposal content to the user. The proposal content generation unit may adjust the proposal content so as to increase the degree of positivity of the user emotion based on a transition of the user emotion value of the user when the control unit is outputting the proposal content to the user.

[0006] In any of the control systems, the suggestion content generation unit may adjust the suggestion content so as to increase the degree of positivity of the user's emotions, further based on the content of a conversation between the electronic device and the user.

[0007] In any of the control systems, the emotion determination unit may further determine a user emotion value of another user listening to the conversation between the electronic device and the user, and the proposal content generation unit may adjust the proposal content based on a transition in the user emotion value of the user when the output control unit is outputting the proposal content to the user and the transition in the user emotion value of the other user.

[0008] Any of the control systems may further comprise the electronics.

[0009] According to one embodiment of the present invention, there is provided a program for causing a computer to function as the control system.

[0010] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0011] [Figure 1] 1 shows an example of the overall configuration of a system 10 according to the present embodiment. [Diagram 2] 1 shows a schematic functional block configuration of a robot 200 and a server 100. [Diagram 3] An example of the hardware configuration of a computer 1200 that functions as a part of the server 100 or the robot 200 is shown in schematic form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0013] FIG. 1 illustrates an example of an overall configuration of a system 10 according to the present embodiment. The system 10 includes a server 100 and a robot 200. The system 10 may include a plurality of robots 200. The robot 200 may be a humanoid robot. The robot 200 may be a non-humanoid robot. The robot 200 may be an example of an electronic device. The server 100 may function as a control system. The robot 200 may be an example of an object to be controlled by the control system. The system 10 may be an example of a control system.

[0014] The robot 200 is placed in an arbitrary location. The user 20 is a person who uses the robot 200.

[0015] The robot 200 is placed, for example, in a home. The user 20 may be a family member of the home in which the robot 200 is placed, or a stranger visiting the home. The robot 200 is placed, for example, in an image consultant salon, a beauty salon, or the like. The user 20 may be a customer of the image consultant salon or the beauty salon. The manner of use of the robot 200 is not limited to these manners of use.

[0016] The server 100 is provided remotely from the robot 200. The server 100 can control the robot 200 through a communication network 50. For example, the server 100 acquires sensor information detected by the robot 200 through the communication network 50, determines an action to be performed by the robot 200 based on the acquired sensor information, and transmits control information through the communication network 50 to instruct the robot 200.

[0017] For example, the robot 200 acquires various types of information detected by sensors, such as the voice and image of the user 20 and information on an external force received by the robot 200, and transmits the information to the server 100. The server 100 generates control information for controlling the robot 200 based on the information acquired from the robot 200. The server 100 generates control information for controlling the robot 200 based on the information acquired from the robot 200, for example, by using a neural network (NN).

[0018] In the system 10, the server 100 may execute processes for processing various information to determine the behavior of the robot 200, and processes for determining the behavior of the robot 200 based on the results of processing the various information. Note that part or all of these processes may be executed by the robot 200. In this case, part of the robot 200 functions as a control system.

[0019] The system 10 according to the present embodiment executes a process of proposing to the user 20 a personal image suited to the user 20. For example, the system 10 analyzes the facial shape, body shape, and so on of the user 20 as well as the facial expression, emotions, and speaking style of the user 20, and provides advice on fashion, makeup, hairstyle, manners, and so on that will make the user 20 look more attractive.

[0020] For example, the system 10 reads the appearance of the user 20 using a camera and a sensor, and proposes a result to the user 20 based on the so-called image consulting theory (color diagnosis, bone structure diagnosis, face type diagnosis, etc.). The system 10 further takes into account the user 20's desires, such as the person he or she wants to be, his or her occupation, and characteristics such as the volume and tone of his or her voice and facial expression, and proposes a result. The system 10 then reads the transition of the user's 20 emotions from the user's facial expressions and conversation in response to the proposal, and makes a proposal that maximizes positive emotions. The system 10 adjusts the way the proposal is made while watching the user's 20 reaction to the proposal. For example, while the robot 200 is talking about the contents of the proposal to the user 20, the impression can be changed by changing the ending of the sentence to "I think," "What do you think," "It is," etc., based on the user's reaction.

[0021] 2 shows an example of a functional block configuration of the robot 200 and the server 100. First, the functional block configuration of the robot 200 will be described. The robot 200 has a sensor unit 210, an information processing unit 220, a control target 230, and a communication unit 240. The information processing unit 220 may be a processor such as an MPU. The communication unit 240 is responsible for communication with the server 100. The communication unit 240 may be a communication device such as a network IF.

[0022] The control target 230 includes a speaker. The control target 230 includes a display unit. The display unit may be disposed at any position of the robot 200, such as the chest of the robot 200. The control target 230 includes LEDs installed in the eyes or the like. The control target 230 includes motors and the like that drive movable parts of the robot 200, such as the limbs and head.

[0023] The sensor unit 210 has various sensors such as a microphone 211, a gyro sensor 212, a motor sensor 213, a camera 214, an infrared sensor 215, and a battery remaining amount sensor 216. It is not essential that the sensor unit 210 has all of these sensors, and the sensor unit 210 may have sensors other than these.

[0024] The microphone 211 acquires surrounding sounds. For example, the microphone 211 acquires the voice of the user 20. The gyro sensor 212 detects the angular velocity of the entire robot 200 and each part of the robot 200. The motor sensor 213 detects the rotation angle of the drive shaft of the motor that drives the movable part of the robot 200. The camera 214 captures images using visible light and generates image information of moving images and still images. The infrared sensor 215 detects surrounding objects using infrared rays. The battery remaining capacity sensor 216 detects the remaining capacity of the battery equipped in the robot 200.

[0025] The sensor unit 210 outputs various sensor data, such as voice data acquired by the microphone 211, an image captured by the camera 214, an angular velocity detected by the gyro sensor 212, a rotation angle detected by the motor sensor 213, a remaining capacity detected by the battery remaining amount sensor 216, and object information detected by the infrared sensor 215, to the information processing unit 220. The information processing unit 220 supplies the acquired sensor signals to the communication unit 240, and transmits them to the server 100. Furthermore, based on the control information acquired from the server 100, the information processing unit 220 causes the speaker of the robot 200 to speak, causes an LED of the robot 200 to emit light, and causes limbs and the like of the robot 200 to operate.

[0026] Next, a functional block configuration of the server 100 will be described. The server 100 has a processing unit 110, a communication unit 120, and a storage unit 130. The processing unit 110 includes a control unit 111, a user state recognition unit 112, a diagnosis result acquisition unit 113, a user characteristic acquisition unit 114, a user preference acquisition unit 115, an emotion determination unit 116, a proposal content generation unit 117, and a learning execution unit 118. The communication unit 120 has an information acquisition unit 122.

[0027] The communication unit 120 is responsible for communication with the robot 200. The communication unit 120 may be a communication device such as a network IF. The information acquisition unit 122 acquires various information. For example, the information acquisition unit 122 acquires information detected by the sensor unit 210 of the robot 200 via the communication network 50.

[0028] The storage unit 130 has a storage medium such as a hard disk device, a flash memory, etc. The storage unit 130 also has a volatile storage device such as a RAM. The storage unit 130 stores program codes and various temporary data that the processing unit 110 reads out at execution time, as well as information required for the processing of the processing unit 110. The storage unit 130 may store so-called Large Language Models (LLM). The storage unit 130 stores information acquired by the communication unit 120 from the robot 200.

[0029] The control unit 111 uses the information stored in the memory unit 130 and the information acquired by the information acquisition unit 122 to determine the behavior of the robot 200, and transmits control information for executing the determined behavior to the robot 200 via the communication unit 120.

[0030] The user state recognition unit 112 recognizes the state of the user 20 based on the information acquired by the information acquisition unit 122. The user state recognition unit 112 may analyze a captured image captured by the camera 214 to recognize the face of the user 20. For example, the storage unit 130 stores face images of a plurality of people, and the user state recognition unit 112 recognizes the face of the user 20 by matching the captured image with the stored face images. The user state recognition unit 112 may analyze a captured image captured by the camera 214 to recognize the facial expression of the user 20. The user state recognition unit 112 analyzes the voice of the user 20 detected by the microphone 211 to recognize the contents of the utterance of the user 20. The user state recognition unit 112 may perform voice emotion recognition by analyzing the voice of the user 20 detected by the microphone 211. For example, the user state recognition unit 112 extracts feature amounts such as frequency components of the voice, and recognizes the emotion of the user 20 based on the extracted feature amounts.

[0031] The user state recognition unit 112 may recognize the behavior of the user 20 based on the information acquired by the information acquisition unit 122. For example, the user state recognition unit 112 stores in advance a neural network that outputs the behavior of the user 20 from the information acquired by the information acquisition unit 122, which is learned using the information acquired by the information acquisition unit 122 and the actual behavior of the user 20 at the time when the information acquisition unit 122 acquired the information as training data, and specifies the user's behavior by inputting the information acquired by the information acquisition unit 122 to the neural network. The neural network may be configured to receive the facial expression of the user 20 and the contents of the user's utterance recognized by the user state recognition unit 112. The method of recognizing the user's behavior is not limited to this, and other known methods may be used.

[0032] The diagnosis result acquisition unit 113 acquires a diagnosis result of the image consulting diagnosis of the user 20 who is conversing with the robot 200. The image consulting diagnosis may include a color diagnosis. The image consulting diagnosis may include a bone structure diagnosis. The image consulting diagnosis may include a face type diagnosis. The image consulting diagnosis may include other diagnosis results.

[0033] The diagnosis result acquisition unit 113 may perform an image consulting diagnosis by itself using the information acquired by the information acquisition unit 122, and acquire a diagnosis result. Information necessary for performing an image consulting diagnosis may be registered in advance and stored in the storage unit 130. The diagnosis result acquisition unit 113 performs a color diagnosis of the user 20, for example, by analyzing a captured image of the user 20. The diagnosis result acquisition unit 113 performs a bone structure diagnosis of the user 20, for example, by analyzing a captured image of the user 20. The diagnosis result acquisition unit 113 performs a face type diagnosis of the user 20, for example, by analyzing a captured image of the user 20.

[0034] When an image consultant is talking to the user 20 together with the robot 200, the diagnosis result acquisition unit 113 may acquire from the image consultant the result of the image consulting diagnosis performed by the image consultant.

[0035] The user feature acquisition unit 114 acquires user features indicating the characteristics of the user 20. The user features may include the volume of the voice of the user 20. The user features may include the tone of the voice of the user 20. The user features may include facial expressions of the user 20. The user features may include other features. The user feature acquisition unit 114 may acquire the user features from information acquired by the information acquisition unit 122 or a user state recognized by the user state recognition unit 112.

[0036] The user preference acquisition unit 115 acquires user preferences indicating the preferences of the user 20. The user preferences may include the occupation desired by the user 20. The user preferences may include the self-image desired by the user 20. The user preferences may include other preferences.

[0037] The user preference acquiring unit 115 may acquire the user preference from the user 20. For example, the control unit 111 elicits the user preference from the user 20 by the robot 200. Also, for example, the user preference acquiring unit 115 acquires the user preference that the user 20 has filled in a diagnosis form or input into a tablet or the like.

[0038] The emotion determination unit 116 determines a user emotion value indicating the emotion of the user 20 conversing with the robot 200. The emotion determination unit 116 may determine the user emotion value of the user 20 using information acquired by the information acquisition unit 122. The emotion determination unit 116 may determine the user emotion value of the user 20 using a user state recognized by the user state recognition unit 112. When voice emotion recognition is executed by the user state recognition unit 112, the emotion determination unit 113 may determine the user emotion value of the user 20 using information acquired by the information acquisition unit 122 and the voice emotion recognition result.

[0039] The user emotion value may include a type of emotion of the user 20, and a value indicating the positive or negative emotion and the degree of the emotion of the user 20. Examples of the types of emotion of the user 20 include "joy", "pleasure", "comfort", "relief", "excitement", "relief", "fulfillment", "anger", "sorrow", "discomfort", "anxiety", "sadness", "concern", and "emptiness". Note that these are merely examples and are not limited to these. The value indicating the positive or negative emotion of the user 20 indicates a positive value if the emotion of the user 20 is a cheerful emotion accompanied by a pleasant feeling or comfort, such as "joy", "pleasure", "comfort", "relief", "excitement", "relief", and "fulfillment", and the brighter the emotion, the larger the value. If the emotion of the user is an unpleasant emotion, such as "anger", "sorrow", "discomfort", "anxiety", "sadness", "concern", and "emptiness", the value indicates a negative value, and the more unpleasant the emotion, the larger the absolute value of the negative value. If the emotion of the user is not any of the above ("normal"), the value of 0 is indicated.

[0040] For example, the emotion determination unit 116 prestores a neural network that outputs a user emotion value of the user 20 from the information acquired by the information acquisition unit 122, the neural network having been trained using the information acquired by the information acquisition unit 122 and a user emotion value indicating the actual emotion of the user 20 at the time the information acquisition unit 122 acquired the information as training data, and acquires the user emotion value by inputting the information acquired by the information acquisition unit 122 into the neural network. The neural network may be configured to accept the facial expression of the user 20 and the vocalizations of the user 20 recognized by the user state recognition unit 112. The method of determining the user emotion value is not limited to this, and other known methods may be used.

[0041] The proposed content generation unit 117 generates proposed content for the user 20. The proposed content generation unit 117 may generate proposed content for the user 20 based on the diagnosis result acquired by the diagnosis result acquisition unit 113, the user characteristics acquired by the user characteristics acquisition unit 114, and the user preferences acquired by the user preference acquisition unit 115. The control unit 111 may cause the robot 200 to output the proposed content generated by the proposed content generation unit 117 to the user 20.

[0042] The proposed content generation unit 117 generates the proposed content by inputting the diagnosis result acquired by the diagnosis result acquisition unit 113, the user characteristics acquired by the user characteristic acquisition unit 114, and the user preferences acquired by the user preference acquisition unit 115 into a learning model that receives as input the diagnosis result of the image consulting diagnosis of the user 20, the user characteristics of the user 20, and the user preferences of the user 20 and outputs the proposed content for the user 20. The learning model may be registered in advance and stored in the storage unit 130.

[0043] Also, the server 100 may generate the learning model. For example, the storage unit 130 stores a large amount of learning data that associates the proposals that the image consultant actually made to the target person, the diagnosis results of the image consulting diagnosis of the target person, the user characteristics of the target person, and the user preferences of the target person. Then, the learning execution unit 118 generates the learning model by machine learning using the multiple learning data stored in the storage unit 130. Note that generating a learning model may include generating a new learning model, and may include updating an existing learning model.

[0044] The learning execution unit 118 may generate a learning model by executing machine learning that applies different weights to the diagnosis result, the user characteristics, and the user preferences. This allows the difference in the influence that the diagnosis result, the user characteristics, and the user preferences have on the proposal content to be reflected in the learning, which contributes to generating more appropriate proposal content.

[0045] The proposed content generating unit 117 may adjust the proposed content based on the transition of the user emotion value of the user 20 when the control unit 111 is outputting the proposed content to the user 20. For example, when the proposed content generating unit 117 determines that the degree of positivity has decreased or that the degree of negativity has increased while the control unit 111 is causing the robot 200 to output the proposed content, the proposed content generating unit 117 adjusts the proposed content so as to soften the degree of the proposal. As a specific example, the control unit 111 adjusts parts of the proposed content that have assertive expressions such as "It would be good to do ~" or "I suggest that you do ~" to softer expressions such as "I think it would be good to do ~" or "How about doing ~". This makes it possible to prevent the user 20 from strengthening negative emotions by making too strong a suggestion at a stage when the user 20 is not yet interested.

[0046] For example, the proposed content generating unit 117 adjusts the proposed content so that the degree of positivity of the user 20 is increased based on the transition of the user emotion value of the user 20 when the control unit 111 is outputting the proposed content to the user 20. As a specific example, when the control unit 111 determines that the degree of positivity has not changed or has decreased while the robot 200 is outputting the proposed content, the proposed content generating unit 117 adds an explanatory content that explains how the user 20 will look more attractive by following the proposal. The explanatory content corresponding to the proposed content may be registered in the storage unit 130 in advance. This can contribute to increasing the degree of positivity of the user 20 and making the user more likely to accept the proposal.

[0047] The proposal content generating unit 117 may adjust the proposal content so that the degree of positivity of the user 20 increases, based on the contents of the conversation between the robot 200 and the user 20. For example, the proposal content generating unit 117 adjusts the proposal content so that, when the user 20 makes a negative statement about the proposal content while the control unit 111 is causing the robot 200 to output the proposal content, the proposal content generating unit 117 adjusts the proposal content so that, when the proposal is changed, the user 20 makes a negative statement about the proposal content, the proposal content generating unit 117 monitors the transition of the user emotion value of the user 20, and when the degree of positivity increases, the proposal content generating unit 117 advances the conversation about the proposal, and when the degree of positivity decreases, becomes negative, or increases, the proposal content is further changed. Also, for example, the proposal content generating unit 117 adjusts the proposal content so that, when the user 20 makes a negative statement about the proposal content while the control unit 111 is causing the robot 200 to output the proposal content, the proposal content generating unit 117 proposes multiple candidates. When multiple candidates are proposed, the proposed content generation unit 117 may adjust the proposed content so as to advance the conversation of the candidate selected from the multiple candidates by the user 20. In this way, the proposed content can be adjusted so as to maximize the degree of positivity of the user 20's feelings.

[0048] When there is another user listening to the conversation between the robot 200 and the user 20, such as when the user 20 is accompanied by another user, the user state recognition unit 112 may recognize the user state of the other user, and the emotion determination unit 116 may identify the user emotion value of the other user. The proposed content generation unit 117 may adjust the proposed content based on the transition of the user emotion value of the user 20 when the control unit 111 is outputting the proposed content to the user 20 and the transition of the user emotion value of the other user.

[0049] For example, the proposed content generating unit 117 does not adjust the proposed content if the degree of positivity of the user 20 and other users increases or there is no change in the degree of positivity while the control unit 111 is causing the robot 200 to output the proposed content, but adjusts the proposed content if the degree of positivity of the user 20 and other users decreases or the degree of negativity increases. The proposed content generating unit 117 may proceed with the proposal without adjusting the proposed content even if the degree of positivity of the user 20 decreases or the degree of negativity increases, or if the degree of positivity of the user 20 simply increases. Thereby, for example, when a relatively large style change such as changing long hair to short hair is proposed to the user 20, the proposal can be proceeded with if the accompanying person feels positive even if the user 20 himself feels uneasy.

[0050] The robot 200 may have emotions. The emotion determination unit 116 may determine a robot emotion value indicating the emotion of the robot 200. The robot emotion value may be an example of an electronic device emotion value. The robot emotion value may include a type of emotion of the robot 200, and a value indicating the positive or negative emotion and the degree of the emotion of the robot 200. Examples of the types of emotions of the robot 200 include "joy", "pleasure", "comfort", "relief", "excitement", "relief", "fulfillment", "anger", "sorrow", "discomfort", "anxiety", "sorrow", "worry", and "emptiness". Note that these are examples and are not limited to these. The value indicating the positive or negative emotion of the robot 200 indicates a positive value if the emotion of the robot 200 is a bright emotion accompanied by a pleasant feeling or comfort, such as "joy", "pleasure", "comfort", "relief", "excitement", "relief", and "fulfillment", and the brighter the emotion, the larger the value. If the emotion of the robot 200 is an unpleasant emotion such as "anger", "sorrow", "discomfort", "anxiety", "sorrow", "worry" or "emptiness", it shows a negative value, and the more unpleasant the emotion, the larger the absolute value of the negative value becomes. If the emotion of the robot 200 is not any of the above ("neutral"), it shows a value of 0.

[0051] The emotion determination unit 116 may determine the robot emotion value based on the information acquired by the information acquisition unit 122. The emotion determination unit 116 may determine the robot emotion value according to a rule for updating the robot emotion value, which is determined in association with the information acquired by the information acquisition unit 122. For example, the emotion determination unit 116 may increase the emotion value of "sadness" of the robot 200 when the remaining battery level of the robot 200 is low or when the room is pitch black. For example, the emotion determination unit 116 may increase the emotion value of "anger" in the case of a user 20 who continues to talk to the robot 200 despite the remaining battery level being low.

[0052] The suggested content generation unit 117 may adjust the suggested content based on the transition of the user emotion value of the user 20 and the transition of the emotion value of the other robot 200, with the other robot 200 being a pseudo companion of the user 20.

[0053] The functions of the server 100 described above may be implemented by one or more computers. At least some of the functions of the server 100 may be implemented by a virtual machine. At least some of the functions of the server 100 may be implemented in the cloud. In the above embodiment, the robot 200 has been mainly given as an example of an electronic device, but this is not limited to this. Examples of electronic devices include smartphones, tablet terminals, PCs (Personal Computers), smart speakers, stuffed toys, vehicles such as automobiles and motorcycles, home appliances, etc. Note that these are merely examples, and electronic devices are not limited to these.

[0054] 3 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as a part of the server 100 or the robot 200. The computer 1200 may function as the server 100, for example. A program installed on the computer 1200 may cause the computer 1200 to function as one or more "parts" of the apparatus according to the present embodiment, or cause the computer 1200 to execute operations or one or more "parts" associated with the apparatus according to the present embodiment, and / or cause the computer 1200 to execute a process or steps of the process according to the present embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0055] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0056] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0057] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227 or the like and provides the programs or data to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0058] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0059] The programs are provided by a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0060] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, the DVD-ROM 1227, or a recording medium such as an IC card, and transmits the read transmission data to a network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0061] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0062] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0063] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0064] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0065] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0066] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0067] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0068] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0069] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0070] 10 system, 20 user, 50 communication network, 100 server, 110 processing unit, 111 control unit, 112 user state recognition unit, 113 diagnosis result acquisition unit, 114 user feature acquisition unit, 115 user preference acquisition unit, 116 emotion determination unit, 117 proposal content generation unit, 118 learning execution unit, 120 communication unit, 122 information acquisition unit, 130 memory unit, 200 robot, 210 sensor unit, 211 microphone, 212 gyro sensor, 213 motor sensor, 214 camera, 215 infrared sensor, 216 battery remaining sensor, 220 information processing unit, 230 control target, 240 communication unit, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 I / O controller, 1222 communication interface, 1224 storage device, 1226 DVD drive, 1227 DVD-ROM, 1230 ROM, 1240 I / O chip

Claims

1. a diagnosis result acquisition unit that acquires a diagnosis result of an image consulting diagnosis including at least one of a color diagnosis, a bone structure diagnosis, and a face type diagnosis of a user who is talking to the electronic device; A user feature acquisition unit that acquires user features including at least one of a volume of the user's voice, a tone of the voice, and a facial expression of the user; a user preference acquiring unit that acquires user preferences including at least one of a desired occupation and a desired self-image of the user; a proposal content generation unit that generates a proposal content for the user based on the diagnosis result, the user characteristics, and the user's desires; a control unit that causes the electronic device to output the content of the proposal to the user; A control system comprising:

2. A storage unit that stores learning data that associates the proposals that an image consultant actually made to a target person, the diagnosis result of the image consultant diagnosis of the target person, the user characteristics of the target person, and the user preferences of the target person; a learning execution unit that generates a learning model by machine learning using a plurality of learning data, the learning model inputting the diagnosis result of the image consulting diagnosis of the user, the user characteristics of the user, and the user preferences of the user, and outputting the contents of a proposal to the user; Equipped with The control system according to claim 1 , wherein the proposal content generator generates the proposal content by using the learning model.

3. The control system according to claim 2 , wherein the learning execution unit executes the machine learning in which different weights are applied to the diagnosis result, the user characteristics, and the user preferences, to generate the learning model.

4. an emotion determining unit for determining a user emotion value indicating an emotion of the user who is conversing with the electronic device; Further equipped with The control system according to claim 1 , wherein the suggestion content generation unit adjusts the suggestion content based on a transition of the user emotion value of the user when the control unit is outputting the suggestion content to the user.

5. The control system according to claim 4 , wherein the suggestion content generation unit adjusts the suggestion content so as to increase the degree of positivity of the user's emotion based on a transition in the user emotion value of the user when the control unit is outputting the suggestion content to the user.

6. The control system according to claim 4 , wherein the suggestion content generator adjusts the suggestion content so as to increase a degree of positivity of the user's emotions, further based on a content of a conversation between the electronic device and the user.

7. The emotion determining unit further determines a user emotion value of another user listening to the conversation between the electronic device and the user; 5. The control system according to claim 4, wherein the suggestion content generation unit adjusts the suggestion content based on a transition of the user emotion value of the user and a transition of the user emotion value of the other user when the control unit is outputting the suggestion content to the user.

8. The control system of claim 1 , further comprising the electronics.

9. A program for causing a computer to function as the control system according to any one of claims 1 to 3.