Control device, control program, and control method

Abstract

To provide users with a more enjoyable dining experience. [Solution] The control device comprises an acquisition unit (311) that acquires information about the user, an estimation unit (312) that estimates the user's state based on the information acquired by the acquisition unit, and a control unit (313) that controls a device for stimulating at least one of the user's five senses according to the state estimated by the estimation unit when the user is eating.

Description

Technical Field

[0001] The present disclosure relates to a control device, a control program, and a control method.

Background Art

[0002] Conventionally, inventions for supporting eating habits have been known (for example, Patent Document 1). Patent Document 1 describes that by adding a lighting effect to a dish, the dish is made to look more delicious and the appetite is enhanced.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the invention described in Patent Document 1 is merely a technique for producing a visual effect and does not consider the state of the user on that day, such as physical condition, etc., so there is room for improvement in providing a more enjoyable dining experience.

[0005] One aspect of the present disclosure aims to realize a control device capable of providing a more enjoyable dining experience by stimulating the five senses according to the state of the user.

Means for Solving the Problems

[0006] In order to solve the above problems, a control device according to one aspect of the present disclosure includes an acquisition unit that acquires information about a user, an estimation unit that estimates the state of the user based on the information acquired by the acquisition unit, and a control unit that controls a device for stimulating at least one of the five senses of the user according to the state estimated by the estimation unit when the user eats.

[0007] To solve the above problems, a control method according to one aspect of the present disclosure is a control method used in a control device, comprising: an acquisition step of acquiring information about a user; an estimation step of estimating the state of the user based on the acquired information; and a control step of controlling a device for stimulating at least one of the user's five senses in accordance with the estimated state when the user is eating.

[0008] Each aspect of the present disclosure may be implemented by a computer, in which case a control program that enables the computer to implement the control device by operating the computer as each part (software element) of the control device, and a computer-readable recording medium on which the program is recorded, also fall within the scope of the present disclosure. [Effects of the Invention]

[0009] According to one aspect of this disclosure, it is possible to provide users with a more enjoyable dining experience. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram illustrates an example of a scenario in which the information processing system disclosed herein is applied. [Figure 2] This diagram illustrates an example configuration of an information processing system according to Embodiment 1. [Figure 3] This is a block diagram showing an example of the server hardware configuration according to Embodiment 1. [Figure 4] This is a block diagram showing an example of the functions of the server processor according to Embodiment 1. [Figure 5] This table explains the relationship between the user's state and sensory stimuli. [Figure 6] This is a sequence chart showing an example of the information processing flow according to Embodiment 1. [Figure 7] This block diagram shows an example of the functions of the server processor according to Embodiment 2. [Figure 8]This is a block diagram showing an example of the functions of the server processor according to Embodiment 3. [Figure 9] This diagram illustrates an example of a dish suggested based on the user's state. [Figure 10] This flowchart shows an example of the processing performed by the server according to Embodiment 3. [Modes for carrying out the invention]

[0011] Each embodiment of this disclosure will be described in detail below with reference to the drawings. In the drawings, identical or substantially identical components will be denoted by the same reference numerals and will not be repeated in the description.

[0012] (Examples of system application) First, an example of the application of the system of this disclosure (the information processing system 100 described later) will be explained with reference to Figure 1. Figure 1 is a diagram illustrating an example of a scene in which the system of this disclosure is applied. As shown in Figure 1, the system of this disclosure is applied, for example, in a scene in which user 10 is about to eat a meal.

[0013] As shown in the left figure of FIG. 1, when the user 10 is about to have a meal, the system of the present disclosure captures the face of the user 10 with the camera 21, and estimates the state of the user 10 (such as physical condition like energetic, tired, etc.) from the obtained face image data. The estimation method is not particularly limited. For example, a well-known algorithm that analyzes feature points of the face (eyes, eyebrows, mouth, nose, etc.) from the face image data to classify the expression and emotion of the subject may be used. Also, as another method, a trained model that has learned the face image data may be used to estimate the state of the user 10 from the face image data. By using such a method, as the state of the user 10, it is possible to estimate physical conditions, moods, mental states, etc. such as happy, glad, excited, at ease, energetic, tired, gloomy, sad, depressed, anxious, listless, feeling stressed, angry, irritable, hungry, not hungry, etc. The installation location of the camera 21 is not particularly limited. For example, the camera 21 may be a camera installed in a restaurant.

[0014] The system of the present disclosure controls various devices so that the user 10 can enjoy the meal according to the estimated state of the user 10 (physical condition, mood, mental state, etc.). Here, the various devices are devices that can stimulate the five senses (vision, hearing, touch, smell, taste) of the user 10. As an example of such a device, as shown in FIG. 1, a chair 11 with a vibrator, a speaker 12, a ceiling light 13, a display 14, an aroma diffuser 15, etc. can be mentioned.

[0015] The chair 11 with a vibrator is a type of device that stimulates the sense of touch, which is one of the five senses, by controlling the magnitude, speed, rhythm, etc. of the vibration.

[0016] The speaker 12 is a type of device that stimulates the sense of hearing, which is one of the five senses, by playing upbeat music, relaxing music, etc.

[0017] The ceiling light 13 is a type of device that stimulates the sense of vision, which is one of the five senses, by controlling the brightness, color, etc.

[0018] The display 14 is a kind of device that stimulates one of the five senses, namely vision, by controlling the video to be displayed, such as exciting video or a relaxing scenery.

[0019] The aroma diffuser 15 is a kind of device that stimulates one of the five senses, namely smell, by controlling the emitted smells and scents, such as the fresh smell of citrus, the soothing smell of cypress, the relaxing smell of lavender, the herbaceous and spicy scents.

[0020] Although illustration is omitted, as a device that stimulates one of the five senses, namely taste, for example, an electronic taste device that reproduces taste by electrical stimulation may be used.

[0021] The system of the present disclosure controls any one, or an arbitrary combination, or all of these vibrating chairs 11, speakers 12, ceiling lights 13, displays 14, aroma diffusers 15, etc. according to the state (physical condition, mood, mental state, etc.) of the user 10, and stimulates at least one, or an arbitrary combination, or all of the five senses of the user 10. This leads to providing a comfortable stimulation space suitable for the physical condition, mood, mental state, etc. of the user 10, and ultimately leads to providing a pleasant dining experience for the user 10. According to such a dining experience, the user 10 can enjoy the cuisine more deliciously and more pleasurably. Hereinafter, the details of the system of the present disclosure will be described. In the following, such a service provided to the user 10 is referred to as a "dining experience service".

[0022] 〔Embodiment 1〕 (Information processing system 100) Referring to Figure 2, an example configuration of the information processing system 100 according to Embodiment 1 will be described. As shown in Figure 2, the information processing system 100 includes a user information acquisition device 20, a server 30, and a five-sensory stimulation device 40. The server 30 is connected to the user information acquisition device 20 and the five-sensory stimulation device 40 via a communication network NW. The communication network NW is not particularly limited, but for example, the Internet may be used.

[0023] (User information acquisition device 20) The user information acquisition device 20 includes, for example, a camera 21 installed in a restaurant, as described in Figure 1. The camera 21 may be installed not only in a restaurant, but also in the user's home, or anywhere the user can eat a meal. For example, the camera 21 may be installed in a vehicle owned by the user 10. The user 10 can eat a meal in the vehicle, and the camera 21 installed in the vehicle can capture images of the user 10's face. In other words, the "meal experience service" of this embodiment can also be applied in a scenario where the user 10 eats a meal in a vehicle.

[0024] In this embodiment, the location where information about user 10 (e.g., facial image data) is acquired and the location where user 10 eats are described as being in the same space (e.g., inside a restaurant), but this is not limited to this. The location where information about user 10 is acquired and the location where user 10 eats may be separate. For example, when user 10 travels to a restaurant by vehicle, facial image data of user 10 may be acquired inside the vehicle, and when user 10 eats at the restaurant, the facial image data acquired inside the vehicle may be used.

[0025] As an example other than the camera 21, the user information acquisition device 20 may include a smartphone 22 owned by the user 10. The smartphone 22 can capture images of the user 10's face by devising a way to fix it in place, or by the user 10 taking a selfie.

[0026] The facial image data of user 10, captured by camera 21 or smartphone 22, is transmitted from user information acquisition device 20 to server 30 via communication network NW. Server 30 analyzes the facial image data of user 10 acquired from user information acquisition device 20 to estimate user 10's state (physical condition, mood, mental state, etc.). The information used to estimate user 10's state is not limited to facial image data. For example, the information used to estimate user 10's state may include, in addition to or instead of facial image data, biometric information of user 10.

[0027] Here, we will explain an example of "biometric information." User 10's biometric information may include information such as user 10's heart rate and heart rate variability. Such biometric information can be acquired, for example, by a smartwatch 23 worn on user 10's wrist. The smartwatch 23 may transmit the acquired biometric information of user 10 to the server 30. In this way, if the user information acquisition device 20 includes a smartwatch 23, the server 30 can acquire user 10's biometric information from the user information acquisition device 20.

[0028] To illustrate one example of how to estimate user 10's state using heart rate, it is known that when stress levels rise, the sympathetic nervous system becomes more active, and heart rate tends to increase. Therefore, if user 10's heart rate is higher than normal, it can be estimated that user 10's mental state is negative. To illustrate another example of how to estimate user 10's state using heart rate variability, the LF / HF (Low Frequency, High Frequency) index, which represents the overall balance between the sympathetic and parasympathetic nervous systems, is known as a stress index based on heart rate variability. A high LF / HF ratio indicates a state where the sympathetic nervous system is dominant, which is said to correspond to a high level of stress. Therefore, if user 10's LF / HF ratio is higher than normal, it can be estimated that user 10's mental state is negative.

[0029] In addition to the camera 21, smartphone 22, and smartwatch 23, the user information acquisition device 20 may also include a monitoring microphone. Such a monitoring microphone can acquire voice data from user 10. A well-known voice recognition system may be used as a method for acquiring voice data. The user information acquisition device 20 may transmit voice data to the server 30 in addition to the user 10's facial image data and biometric information.

[0030] As an example of a method for estimating the state of user 10 using voice data, a well-known algorithm that analyzes features such as voice tone, intensity, speed, and volume to classify the subject's facial expression and emotions may be used. Alternatively, the content of the voice data may be analyzed to estimate whether user 10 is positive or negative.

[0031] As another example, if user 10 posts a message to a Social Networking Service (SNS), that message may be sent from smartphone 22 to server 30. By analyzing the content of the message, it is possible to estimate whether user 10 is positive or negative.

[0032] As another example, the information transmitted to the server 30 may include acceleration data acquired by the smartphone 22. Such acceleration data can be used to analyze the user 10's steps, walking speed, walking rhythm (walking cycle, tempo), and body movements during sleep (turning over, small movements), and it is also possible to estimate the user 10's state from this analysis.

[0033] (Server 30) Server 30 acquires information about user 10 (face image data, biometric information, voice data, messages posted on SNS, acceleration data, etc.) from user information acquisition device 20 via the communication network NW. Server 30 estimates user 10's state (physical condition, mood, mental state, etc.) by analyzing the acquired information. An example of the estimation method has been described above, so it will not be repeated. Based on the estimation results, Server 30 controls the five senses stimulator 40 to stimulate user 10's five senses. Specifically, Server 30 generates a control signal to control the five senses stimulator 40 and transmits the generated control signal to the five senses stimulator 40. The five senses stimulator 40 operates according to the control signal received from Server 30 and stimulates user 10's five senses. In this way, user 10 is provided with a dining experience service tailored to their own state.

[0034] The location of server 30 is not particularly limited, but for example, server 30 may be installed in the management center of a business operator that provides dining experience services. Also, server 30 may be installed in Japan or outside of Japan. Furthermore, server 30 may be a physical server or a cloud server.

[0035] (Five sense stimulation device 40) The five-senses stimulator 40 is a device for stimulating the five senses (sight, hearing, touch, smell, and taste), and as described above, it operates according to control signals received from the server 30 to stimulate the five senses of the user 10. The five-senses stimulator 40 includes a vibration device 41, an acoustic device 42, a lighting device 43, a video device 44, a fragrance device 45, and an air conditioning device 46.

[0036] The vibration device 41 is a device that stimulates the sense of touch, one of the five senses, and an example of this is the vibrator-equipped chair 11 shown in Figure 1. The sound device 42 is a device that stimulates the sense of hearing, one of the five senses, and an example of this is the speaker 12 shown in Figure 1. The lighting device 43 is a device that stimulates the sense of sight, one of the five senses, and an example of this is the ceiling light 13 shown in Figure 1. The video device 44 is a device that stimulates the sense of sight, one of the five senses, and an example of this is the display 14 shown in Figure 1. The fragrance device 45 is a device that stimulates the sense of smell, one of the five senses, and an example of this is the aroma diffuser 15 shown in Figure 1.

[0037] The visual stimulation device 44 is not limited to the display 14; other devices may be used as long as they operate in response to control signals from an external device (e.g., a server 30). For example, a VR (Virtual Reality) headset may be used as the visual stimulation device 44.

[0038] The vibration device 41 that stimulates the sense of touch is not limited to the chair 11 with a vibrator, but other devices may be used as long as they operate in response to control signals from an external device (e.g., a server 30). For example, a haptic feedback device, such as a haptic device, may be used as the vibration device 41 that stimulates the sense of touch. Note that the stimulation of the sense of touch is not limited to vibration. For example, the sense of touch may be stimulated by thermal energy such as heat or cold. As a device that stimulates the sense of touch using thermal energy, for example, tableware with a heat source may be used.

[0039] Although not shown in Figure 1, the air conditioning unit 46 included in the five-senses stimulating device 40 may also be used as a device to stimulate the sense of touch. The air conditioning unit 46 can provide the user 10 with sensations such as "warm" or "cool".

[0040] Although not shown in Figure 2, the five-sense stimulation device 40 may include a device that stimulates the sense of taste. An example of a device that stimulates the sense of taste is the electronic taste device described above.

[0041] Furthermore, the configuration of a device that stimulates the five senses is not limited to a configuration in which one device stimulates one sense. For example, one device may stimulate multiple senses (e.g., sight and hearing). That is, the five-sense stimulation device 40 according to this embodiment may stimulate any combination selected from sight, hearing, touch, smell, and taste. Alternatively, the five-sense stimulation device 40 according to this embodiment may stimulate all of the five senses: sight, hearing, touch, smell, and taste.

[0042] When the "meal experience service" of this embodiment is provided when user 10 eats a meal in the vehicle, equipment installed in the vehicle may be used as the five-sensory stimulation device 40. Specifically, a seat with a vibrator embedded in it may be used as the vibration device 41. Also, a speaker installed in the vehicle may be used as the sound device 42. Also, a lighting device installed on the ceiling may be used as the lighting device 43. Also, a navigation display, meter display, head-up display, etc. may be used as the video device 44. Also, an aroma diffuser installed in the vehicle may be used as the fragrance device 45. Also, an air conditioner installed in the vehicle may be used as the air conditioning device 46.

[0043] (Hardware configuration of Server 30) Next, an example of the hardware configuration of server 30 will be described with reference to Figure 3. Figure 3 is a block diagram showing an example of the hardware configuration of server 30.

[0044] As shown in Figure 3, the server 30 comprises a processor 31, memory 32, storage 33, and a communication interface 34. Each of these components is connected to the others via a bus 35 so that they can communicate with one another.

[0045] The processor 31 reads the program from the storage 33, loads it into the memory 32, and executes processing according to the program. Such a program may be one that causes the computer to execute at least some of the functions described below. The program may also function in combination with other programs already stored in the storage 33, or in combination with other programs implemented in other devices. Furthermore, the program may be delivered to the server 30 via wireless communication. In this case, the processor 31 loads the delivered program into the memory 32 and executes processing. In other words, the program does not necessarily have to be stored in the storage 33.

[0046] The processor 31 is not particularly limited, but can be implemented as, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an MPU (Micro Processor Unit), or an FPGA (Field-Programmable Gate Array). Although Figure 3 shows one processor, it is not limited to this, and multiple processors may be provided.

[0047] Memory 32 is a computer-readable recording medium and consists of at least one of the following: RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Registered Trademark) (Electrically Erasable Programmable ROM), etc. Such memory 32 may also be called registers, cache, main memory, etc.

[0048] Storage 33 is a computer-readable recording medium that stores various types of data. Such storage 33 can be composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. Storage 33 may also be a portable recording medium such as a flexible disk, optical disk, compact disk, or Blu-ray® disc. Furthermore, storage 33 is sometimes referred to as an auxiliary storage device.

[0049] In this embodiment, the storage 33 stores estimation data 331 for estimating the state of user 10 (physical condition, mood, mental state, etc.). The estimation data 331 includes information about user 10 obtained from the user information acquisition device 20 (face image data, biometric information, voice data, messages posted on SNS, acceleration data, etc.).

[0050] The communication interface 34 is an interface for communicating with external devices. The communication interface 34 is implemented, for example, as hardware such as a network adapter, various communication software, or a combination thereof, and is configured to enable wireless or wired communication over a communication network.

[0051] The hardware configuration of server 30 is not limited to the configuration described above. For example, server 30 may have an input / output interface (I / F) as hardware. Such an I / F may include, for example, information input devices such as a keyboard and pointing devices (e.g., mouse, touch panel), audio input devices such as a microphone, and image input devices such as a camera. The I / F may also include image output devices such as a display and audio output devices such as a speaker.

[0052] (Examples of processor 31 functions) Next, an example of the functions of the processor 31 of the server 30 will be described with reference to Figure 4. Figure 4 is a block diagram showing an example of the functions of the processor 31. The processor 31 reads a program from the storage 33 and executes the program using the memory 32 as a working area, thereby functioning as an acquisition unit 311, an estimation unit 312, and a control unit 313. What is described as "~unit" here may be replaced with "~circuit," "~device," or "~equipment," or with "~step," "~procedure," or "~process." In other words, what is described as "~unit" may be implemented by a program stored in the storage 33 as described above, or by hardware such as elements, devices, boards, and wiring only, or by a combination of software and hardware. These functions will be described below.

[0053] The acquisition unit 311 acquires information about user 10 (face image data, biometric information, voice data, messages posted on SNS, acceleration data, etc.) from the user information acquisition device 20 via the communication network NW. The information about user 10 acquired by the acquisition unit 311 is stored in the storage 33 as estimation data 331.

[0054] The estimation unit 312 obtains estimation data 331 for estimating the state of user 10 (physical condition, mood, mental state, etc.) by referring to the storage 33, and estimates the state of user 10 using the estimation data 331. Regarding face image data, the estimation unit 312 may estimate the facial expression and emotions of user 10 by analyzing facial feature points from face image data using a well-known algorithm, or it may estimate the state of user 10 from face image data using a trained model trained on face image data. An example of an estimation method using biometric information, voice data, messages posted on SNS, and acceleration data has been described above, so it will not be repeated.

[0055] The estimation unit 312 may estimate the state of user 10 using the above-mentioned indicators individually, or it may estimate the state of user 10 from a comprehensive perspective by arbitrarily combining multiple indicators.

[0056] The estimation unit 312 outputs the estimation result to the control unit 313.

[0057] The control unit 313 generates a control signal to operate the five-sensory stimulation device 40 based on the estimation results estimated by the estimation unit 312. The control unit 313 transmits the control signal to the five-sensory stimulation device 40 via the communication network NW. By operating according to the control signal, the five-sensory stimulation device 40 can provide stimuli that are appropriate to the state of the user 10.

[0058] (Example of stimuli applied according to the user's state) Here, referring to Figure 5, we will explain an example of controlling the five-sensory stimulation device 40 based on the estimation results, that is, an example of providing stimulation according to the state of the user 10. Figure 5 is a table that explains the relationship between the state of the user 10 and the five-sensory stimulation.

[0059] In Figure 5, let's assume that user 10's state is estimated to be one of the following: "energetic," "tired," "happy," "depressed," "hungry," or "not hungry." As mentioned above, such estimations are possible by analyzing facial image data, biometric information, voice data, messages posted on social media, acceleration data, etc.

[0060] Furthermore, in Figure 5, the five sensory stimuli are classified into "vibration" (touch), "music" (hearing), "lighting" (sight), "images" (sight), and "smell / fragrance" (olfaction). "Vibration" is provided by the vibration device 41. "Music" is provided by the sound device 42. "Lighting" is provided by the lighting device 43. "Images" are provided by the image device 44. "Smell / fragrance" is provided by the fragrance device 45.

[0061] First, let's explain the case where User 10's state is estimated to be "energetic." In this case, User 10 may be given "rhythmic vibrations" as "vibrations." Also, User 10 may be given "upbeat music" as "music." Furthermore, User 10 may be given "bright lighting" as "lighting." Also, User 10 may be given "exciting images" as "visuals." Also, User 10 may be given a "fresh citrus scent" as "smell / fragrance." When User 10's state is estimated to be "energetic," providing such stimuli to User 10 will allow User 10 to enjoy the food more deliciously and more enjoyably.

[0062] Next, we will explain the case where User 10 is estimated to be "tired." In this case, User 10 may be given a "slow, swaying vibration" as a "vibration." Also, User 10 may be given "relaxing music" as "music." Also, User 10 may be given "soft lighting" as "lighting." Also, User 10 may be given a "relaxing landscape" as "visuals." Also, User 10 may be given a "soothing scent of cypress" as "smell / fragrance." When User 10 is estimated to be "tired," providing such stimuli to User 10 will allow User 10 to enjoy the food more deliciously and more enjoyably.

[0063] Next, we will explain the case in which User 10's state is estimated to be "enjoyable." In this case, User 10 may be given "rhythmic vibrations" as "vibrations." Also, User 10 may be given "upbeat music" as "music." Also, User 10 may be given "bright lighting" as "lighting." Also, User 10 may be given "enjoyable images" as "visuals." Also, User 10 may be given a "fresh citrus scent" as "smell / fragrance." When User 10's state is estimated to be "enjoyable," providing such stimuli to User 10 will allow User 10 to enjoy the food more deliciously and more enjoyably.

[0064] Next, let's consider the case where User 10's state is estimated to be "depressed." In this case, User 10 may be given "relaxing music" as "music." Also, User 10 may be given "soft lighting" as "lighting." Also, User 10 may be given "soothing scenery" as "visuals." Also, User 10 may be given the "relaxing scent of lavender" as "smell / fragrance." When User 10's state is estimated to be "depressed," providing such stimuli to User 10 will allow User 10 to enjoy the food more deliciously and more enjoyably.

[0065] Next, let's consider the case where User 10's state is estimated to be "hungry." In this case, User 10 may be given "rhythmic vibrations" as "vibrations." Also, User 10 may be given "upbeat music" as "music." Furthermore, User 10 may be given "bright lighting" as "lighting." Also, User 10 may be given "pleasant images" as "visuals." Furthermore, User 10 may be given "herbal or spicy scents" as "smells / aromas." When User 10's state is estimated to be "hungry," providing such stimuli to User 10 will allow User 10 to enjoy the food more deliciously and more enjoyably.

[0066] Next, let's consider the case where User 10 is estimated to be "not hungry." In this case, User 10 may be given a "slow, swaying vibration" as a "vibration." Also, User 10 may be given "relaxing music" as "music." Also, User 10 may be given "soft lighting" as "lighting." Also, User 10 may be given "a relaxing landscape" as "images." When User 10 is estimated to be "not hungry," providing such stimuli to User 10 will allow User 10 to enjoy the food more deliciously and more enjoyably.

[0067] In Figure 5, the stimuli given to user 10 may be at least one, any combination, or all of them.

[0068] (Process flow) Next, with reference to Figure 6, an example of the processing flow of the information processing system 100 according to Embodiment 1 will be described. Figure 6 is a sequence chart showing an example of the information processing flow between the user information acquisition device 20, the server 30, and the five-sense stimulation device 40.

[0069] In step S101, the user information acquisition device 20 (camera 21) captures a picture of the user 10's face while the user 10 is eating and acquires facial image data. In Figure 6, the user information acquisition device 20 is described as a camera 21, and the manner in which facial image data is acquired as information about the user 10 is described, but it is not limited to this. As mentioned above, the user information acquisition device 20 may be a smartphone 22 or a smartwatch 23. Furthermore, the information about the user 10 is not limited to facial image data, but may also be biometric information, voice data, messages posted on SNS, acceleration data, etc. The user information acquisition device 20 transmits the facial image data of the user 10 to the server 30 via the communication network NW.

[0070] In step S102, the server 30 acquires face image data of user 10 from the user information acquisition device 20 via the communication network NW.

[0071] In step S103, the server 30 estimates the state of user 10 (physical condition, mood, mental state, etc.) from the facial image data acquired in step S102. The server 30 may use a well-known algorithm to analyze facial feature points from the facial image data and estimate the user's expressions and emotions, or it may use a trained model that has been trained on facial image data to estimate the state of user 10 from the facial image data.

[0072] In step S104, the server 30 generates a control signal to operate the five-sense stimulation device 40 based on the estimation result estimated in the processing of step S103.

[0073] In step S105, the server 30 transmits the control signal generated in step S104 to the sensory stimulation device 40 via the communication network NW.

[0074] In step S106, the sensory stimulation device 40 receives a control signal from the server 30 via the communication network NW.

[0075] In step S107, the five-senses stimulator 40 operates according to the received control signal and stimulates the user 10's five senses (see Figure 5).

[0076] As described above, the control method according to Embodiment 1 includes an acquisition step (S102) for acquiring information about the user 10, an estimation step (S103) for estimating the state of the user 10 based on the acquired information, and control steps (S104, S105) for controlling a device (sensory stimulation device 40) for stimulating at least one of the user 10's five senses according to the estimated state when the user 10 is eating.

[0077] Note that the processing flow in the sequence chart shown in Figure 6 is just one example, and steps may be deleted, new steps added, or the processing order rearranged as long as it does not deviate from the main point.

[0078] (Effects and Benefits) As described above, the following effects and advantages can be obtained according to Embodiment 1.

[0079] The server 30 (control device) includes an acquisition unit 311 that acquires information about the user 10 (face image data, biometric information, voice data, messages posted on SNS, acceleration data, etc.), an estimation unit 312 that estimates the user 10's state (physical condition, mood, mental state, etc.) based on the information acquired by the acquisition unit 311, and a control unit 313 that controls a device (sensory stimulation device 40) for stimulating at least one of the user 10's five senses according to the state estimated by the estimation unit 312 when the user 10 is eating.

[0080] As described above, by controlling the five-sensory stimulator 40 according to the user 10's state (physical condition, mood, mental state, etc.) and providing stimulation to the user 10, a comfortable stimulating space tailored to the user 10's physical condition, mood, mental state, etc., can be provided. This leads to a more enjoyable dining experience for the user 10, and such a dining experience allows the user 10 to savor the food more deliciously and enjoyably.

[0081] [Embodiment 2] Next, Embodiment 2 of the present disclosure will be described with reference to Figure 7. Figure 7 is a block diagram showing an example of the functions of the processor 31 according to Embodiment 2. Embodiment 2 differs from Embodiment 1 in that the processor 31 also functions as a determination unit 314. Components that overlap with Embodiment 1 will be referred to by reference numerals and their descriptions will not be repeated. The differences will be described below.

[0082] The determination unit 314 determines whether the dish being eaten by user 10 is the same dish that user 10 has eaten in the past. The determination method is not particularly limited, but for example, the following method may be used. Each time user 10 eats a meal, camera 21 takes an image of the dish, and the image data is transmitted to server 30 and stored in storage 33. Server 30 analyzes the image data and stores in storage 33 a database containing the name of the dish, the date and time the dish was eaten, etc. When user 10 is about to eat a meal, camera 21 takes an image of the dish and transmits the image data to server 30. Server 30 analyzes the image data and refers to the database to determine whether the dish that user 10 is about to eat is the same dish that user 10 has eaten in the past. The term "past" here is not particularly limited, but it may refer to the not-too-distant past, for example, one month ago (first predetermined period ago), two weeks ago (second predetermined period ago), or one week ago (third predetermined period ago). Furthermore, regarding the determination of whether a dish is the same as one previously eaten, it may be determined to be the same if the same dish has been eaten at least once in the past two weeks, or if the same dish has been eaten a predetermined number of times (for example, two or more times) in the past two weeks. As another example, it may be determined to be the same only if the same dish has been eaten every day in the past two weeks.

[0083] If the determination unit 314 determines that the dish that user 10 is about to eat is the same dish that user 10 has eaten in the past, the control unit 313 controls a device (sensory stimulator 40) to stimulate at least one of user 10's five senses according to user 10's state (physical condition, mood, mental state, etc.).

[0084] With this configuration, even in special environments such as outer space, or in situations where only the same dish can be eaten due to a shortage of supplies, the user 10 can enjoy the meal more deliciously and more enjoyably by receiving stimuli tailored to the user 10's state.

[0085] Furthermore, with this configuration, even users 10 who get bored with the same taste all the time can enjoy their meals more deliciously and more enjoyably.

[0086] [Embodiment 3] Next, Embodiment 3 of this disclosure will be described with reference to Figure 8. Figure 8 is a block diagram showing an example of the functions of the processor 31 according to Embodiment 3. Embodiment 3 differs from Embodiment 1 in that the processor 31 further functions as a suggestion unit 315 and a reception unit 316. Components that overlap with Embodiment 1 will be referred to by reference numerals and their descriptions will not be repeated. The differences will be described below.

[0087] The suggestion unit 315 suggests dishes to the user 10 that are appropriate to the user 10's state (physical condition, mood, mental state, etc.) estimated by the estimation unit 312. The scenes in which the suggestion unit 315 suggests dishes to the user 10 are not particularly limited, but for example, when the user 10 visits a restaurant, the suggestion unit 315 may suggest dishes from the menu offered by the restaurant. Now, with reference to Figure 9, an example of a dish suggested according to the user 10's state will be explained.

[0088] (Example dishes suggested based on user 10's state) In the example shown in Figure 9, it is assumed that user 10's state is estimated to be "full of energy and hungry," "accumulating fatigue," or "just starting to catch a cold." As mentioned above, such estimations are possible by analyzing facial image data, biometric information, voice data, messages posted on social media, acceleration data, etc.

[0089] As shown in Figure 9, if the user 10's state is estimated to be "full of energy and hungry," the suggestion unit 315 may suggest a "hearty meat dish" to the user 10.

[0090] Furthermore, if it is estimated that user 10 is experiencing "accumulated fatigue," the suggestion unit 315 may suggest "high-protein meals" to user 10.

[0091] Furthermore, if the user 10's condition is estimated to be "the beginning of a cold," the suggestion unit 315 may suggest "a dish to warm the body" to the user 10.

[0092] If the user 10 accepts the dish proposed by the proposal unit 315, the control unit 313 may control a device (sensory stimulator 40) to stimulate at least one of the user 10's five senses according to the user 10's state when the user 10 eats the dish.

[0093] With this configuration, it is possible to provide meals that satisfy user 10, and furthermore, to provide a pleasant and stimulating environment tailored to user 10's physical condition, mood, mental state, etc., so that user 10 can enjoy the meal more deliciously and more enjoyably.

[0094] The above describes a mode in which the server 30 determines the stimulus to be given to the user 10 according to the state of the user 10, but it is not limited to this. For example, the user 10 may choose the stimulus to receive. The method of selection is not particularly limited, but let's explain using the example of a case where the user 10 visits a restaurant. The user 10 may operate a touch-panel tablet terminal installed in the restaurant to select the stimulus to receive (the stimulus they want to receive). Information indicating the selection result (information indicating the stimulus selected by the user 10) is transmitted from the tablet terminal to the server 30 via the communication network NW.

[0095] In this way, the server 30 receives the selection result from the user 10. This receiving function is the function of the reception unit 316 shown in Figure 8. Based on the received selection result, the server 30 controls the target sensory stimulation device 40 in order to provide the stimulus desired by the user 10.

[0096] With this configuration, user 10 can enjoy not only stimuli determined according to their own state, but also stimuli they have selected themselves, allowing them to savor their meal more deliciously and enjoyably.

[0097] (Process flow) Next, with reference to Figure 10, the processing flow executed by the server 30 according to Embodiment 3 will be described. Figure 10 is a flowchart of an example of the processing executed by the server 30. Note that, unlike Figure 6, the processing by the user information acquisition device 20 and the five-sense stimulation device 40 is omitted in Figure 10. Also, this flowchart illustrates a case where user 10 visits a restaurant as an example.

[0098] When user 10 visits a restaurant, camera 21 installed in the restaurant captures an image of user 10's face.

[0099] In step S201, the server 30 acquires facial image data of user 10 from camera 21 installed in the restaurant via the communication network NW.

[0100] In step S202, the server 30 estimates the user 10's state (physical condition, mood, mental state, etc.) from the facial image data acquired in step S201.

[0101] In step S203, the server 30 suggests dishes to the user 10 from the menu offered by the restaurant, based on the estimation results obtained in step S202 (see Figure 9).

[0102] In step S204, the server 30 generates a control signal to operate the five-sense stimulation device 40 based on the estimation results estimated in the processing of step S202.

[0103] In step S205, the server 30 transmits the control signal generated in step S204 to the five-sense stimulation device 40 via the communication network NW.

[0104] Through the above process, it is possible to provide a meal that satisfies user 10, and furthermore, to provide a pleasant and stimulating environment tailored to user 10's physical condition, mood, mental state, etc., so that user 10 can enjoy the meal more deliciously and more enjoyably.

[0105] Here, as described above, the server 30 can also function as a reception unit 316. That is, in step S206, the server 30 may receive information indicating the selection result of user 10 entered into a touch-panel tablet terminal installed in the restaurant. If the server 30 receives information indicating the selection result from the tablet terminal (YES in step S206), the process returns to step S204, and the server 30 controls the target five-sensory stimulator 40 to provide the stimulus desired by user 10 based on the received selection result. On the other hand, if user 10 does not make a selection, that is, if no information is sent from the tablet terminal (NO in step S206), the process proceeds to step S207.

[0106] In step S207, the server 30 determines whether user 10 has finished eating. The determination method is not particularly limited, but the server 30 may determine that user 10 has finished eating by analyzing images from the camera 21 installed in the restaurant and seeing if the plate is empty. If it is determined that user 10 is still eating (NO in step S207), the process proceeds to step S208, and user 10's state is estimated again. If the estimation result in step S202 and the estimation result in step S208 are different, the server 30 may give user 10 a different stimulus. On the other hand, if it is determined that user 10 has finished eating (YES in step S207), the process proceeds to step S209, and the server 30 stops the five-sensory stimulation device 40.

[0107] Note that the processing flow shown in the flowchart in Figure 10 is just one example, and steps may be deleted, new steps added, or the processing order rearranged as long as it does not deviate from the main point.

[0108] [Other Embodiments] Server 30 may control the sensory stimulation device 40 to provide user 10 with stimuli different from those provided during the previous meal. For example, suppose user 10's state was estimated to be "energetic" during the previous meal, and is estimated to be "energetic" again during the current meal. Also, suppose that only "vibration" (rhythmic vibration) was provided to user 10 during the previous meal (see Figure 5). In this case, during the current meal, user 10 may be given stimuli different from the previous stimulus (vibration), such as "upbeat music," "bright lighting," "exciting visuals," or "fresh citrus scent," or any combination of these, or all of them (see Figure 5). In this way, by providing user 10 with different stimuli than during the previous meal, user 10 can enjoy different stimuli each time they eat, thus obtaining fresh and pleasant stimuli each time they eat, and user 10 can enjoy their meal more deliciously and more enjoyably.

[0109] In the above example, we have used information about user 10 (face image data, biometric information, voice data, messages posted on social media, acceleration data, etc.) as an example of a method for estimating user 10's state (physical condition, mood, mental state, etc.), but we are not limited to this. Information about user 10 may include user 10's past information, and user 10's state may be estimated based on such past information. For example, suppose the past information includes information about travel destinations user 10 has visited in the past. And suppose all of user 10's travel destinations were cold regions. In this case, it is thought that user 10 prefers relatively low temperatures. When such user 10 eats, for example, if the room temperature is relatively high, user 10's state may be estimated to be "sad" or "unhappy". In this way, user 10's state may be estimated from user 10's past information. Note that since the room temperature is relatively high, server 30 may control the air conditioning unit 46 to lower the room temperature. As a result, user 10 changes from a state of "sad" or "unhappy" to a state of "happy" or "joyful," allowing user 10 to enjoy the food more deliciously and more enjoyably.

[0110] In this disclosure, the phrases "based on" or "in accordance with" do not mean "based solely on" or "in accordance with" unless otherwise specified. The phrase "based on" means both "based solely on" and "at least based on." Similarly, the phrase "in accordance with" means both "in accordance with" and "at least based on."

[0111] [Examples of implementation using software] The function of the server 30 (hereinafter referred to as the "device") is a program (control program) that causes the computer to function as the device, and can be realized by a program that causes the computer to function as each control block of the device (particularly the acquisition unit 311, estimation unit 312, control unit 313, determination unit 314, proposal unit 315, and reception unit 316).

[0112] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, each of the functions described in each of the embodiments above is realized.

[0113] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0114] Furthermore, some or all of the functions of each of the above control blocks can also be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of this disclosure. In addition, it is also possible to realize the functions of each of the above control blocks by, for example, a quantum computer.

[0115] Furthermore, each process described in the above embodiments may be performed by AI (Artificial Intelligence). In this case, the AI ​​may operate on the control device described above, or it may operate on other devices (for example, an edge computer or a cloud server).

[0116] This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure. [Explanation of Symbols]

[0117] 100 Information Processing Systems 30 servers 311 Acquisition Department 312 Estimation Department 313 Control Unit 314 Judgment section 315 Proposal Department 316 Reception Department

Claims

1. An acquisition unit that acquires information about the user, Based on the information acquired by the acquisition unit, an estimation unit estimates the user's state, The system includes a control unit that controls a device for stimulating at least one of the user's five senses in accordance with the state estimated by the estimation unit when the user is eating a meal. Control device.

2. The system further includes a determination unit that determines whether the dish being eaten by the user is the same dish the user has eaten in the past. If the determination unit determines that the dish the user is eating is the same dish the user has eaten in the past, the control unit controls a device to stimulate at least one of the user's five senses according to the state. The control device according to claim 1.

3. The system further includes a suggestion unit that proposes a dish to the user that corresponds to the state estimated by the estimation unit, The control unit controls a device for stimulating at least one of the user's five senses according to the state when the user eats the dish proposed by the proposal unit. The control device according to claim 1.

4. The system further includes a receiving unit that receives information indicating the stimulus selected by the user, The control unit controls the device to provide the user with the stimulus selected by the user. The control device according to claim 1.

5. The control unit controls the device to provide the user with a stimulus different from the stimulus given to the user during the previous meal. The control device according to any one of claims 1 to 4.

6. A control program for causing a computer to function as the control device described in claim 1, wherein the computer functions as the acquisition unit, the estimation unit, and the control unit.

7. A control method used in a control device, The acquisition step involves obtaining information about the user, An estimation step to estimate the user's state based on the acquired information, The control step includes controlling a device for stimulating at least one of the user's five senses according to the estimated state when the user is eating, Control method.

Images