Information processing device and cooking system

Abstract

An information processing device according to an embodiment of the present disclosure is provided with: a flavor presenting unit which generates data for presenting, to a user present in a first space, the flavor of a cooking object present in a second space separated from the first space; and a command generating unit which, on the basis of an input from the user, generates a control command for controlling the operations of a cooking robot present in the second space.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and a cooking system. [Background technology]

[0002] In recent years, with the advancement of sensing technology and robot technology, it has been proposed that a user operate a robot located in a remote location and have the robot perform work in the remote location (for example, Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-75301 [Patent Document 2] Japanese Patent Publication No. 2020-75302 Summary of the Invention

[0004] In such cases, the user can feel as if they are actually at the remote location and operating the device themselves, by having various pieces of information acquired by visual sensors, auditory sensors, and force / tactile sensors at the remote location presented to them visually, aurally, and force / tactilely.

[0005] Therefore, when a user remotely controls a robot, it is desirable to present as much sensory information as possible to the user. In particular, in tasks where the quality of the output strongly depends on the user's senses, it is desirable to present information related to other senses in addition to vision, hearing, and force / tactile sensations.

[0006] It is desirable to provide an information processing device and a cooking system that can present a greater amount of sensory information sensed at a remote location to a user operating a robot at a remote location.

[0007] An information processing device according to one embodiment of the present disclosure includes a flavor presentation unit that generates data for presenting the flavor of a cooking object present in a second space separated from the first space to a user present in the first space, and a command generation unit that generates a control command for controlling the operation of a cooking robot present in the second space based on input from the user.

[0008] A cooking system according to one embodiment of the present disclosure includes a sensing unit that senses the flavor of a cooking object present in a second space, a flavor presentation unit that generates data for presenting the sensed flavor of the cooking object to a user present in a first space separated from the second space, a command generation unit that generates a control command based on input from the user, and a cooking robot whose operation is controlled based on the control command and that cooks the cooking object present in the second space.

[0009] In an information processing device and a cooking system according to an embodiment of the present disclosure, data for presenting the flavor of a cooking object present in a second space to a user present in a first space separated from the second space is generated, and a control command for controlling the operation of a cooking robot that cooks the cooking object present in the second space is generated based on an input from the user. This allows, for example, a user present in the first space to grasp the flavor of a cooking object present in the second space separated from the first space. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is an explanatory diagram showing an overview of a cooking system according to a first embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram showing the configuration of the cooking system according to the embodiment. [Figure 3] FIG. 2 is a block diagram showing an example of a more specific configuration of the cooking system according to the embodiment. [Figure 4A] FIG. 10 is a graph showing an example of a trajectory from a current value to a target value in a flavor space. [Figure 4B]FIG. 10 is a graph illustrating an example of a trajectory from a current value to a target value in an operational space. [Figure 5] FIG. 10 is a graph showing an example of visual presentation of the flavor of a cooking object in a modified example. [Figure 6] FIG. 10 is a graph showing another example of visual presentation of the flavor of a cooking object in a modified example. [Figure 7] FIG. 10 is a block diagram showing an example of the configuration of a cooking system according to a second embodiment of the present disclosure. [Figure 8] FIG. 10 is a graph showing an example of visual presentation of the flavor of a cooking object in the cooking system according to the embodiment. [Figure 9] FIG. 2 is a block diagram illustrating an example of an operation of the cooking system according to the embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a cooking system according to a third embodiment of the present disclosure. [Figure 11] FIG. 2 is a block diagram illustrating an example of an operation of the cooking system according to the embodiment. [Figure 12] FIG. 10 is a block diagram showing another example of the operation of the cooking system according to the embodiment. [Figure 13A] FIG. 2 is a schematic diagram showing an example of an arrangement of objects in a first space. [Figure 13B] FIG. 10 is a schematic diagram showing an example of an arrangement of objects in a second space. [Figure 14] FIG. 10 is a block diagram showing an example of a more specific configuration of a cooking system according to a fourth embodiment of the present disclosure. [Figure 15A] FIG. 2 is a schematic diagram illustrating the correspondence of object placement in a first space. [Figure 15B] FIG. 10 is a schematic diagram illustrating the correspondence of object placement in a second space. [Figure 16] FIG. 10 is a block diagram showing an example of a configuration of a modified example of the cooking system according to the embodiment. [Figure 17] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The embodiment described below is a specific example of the present disclosure, and the technology according to the present disclosure is not limited to the following aspects. Furthermore, the arrangement, dimensions, dimensional ratios, etc. of each component of the present disclosure are not limited to the aspects shown in the drawings.

[0012] The explanation will be given in the following order. 1. First embodiment Overview 1.2.Configuration Example 1.3. Variations 2. Second embodiment 2.1.Configuration Example 2.2. Example of operation 3. Third embodiment 3.1.Configuration Example 3.2. Example of operation 4. Fourth Embodiment Overview 4.2.Configuration Example 4.3. Variations 5. Hardware configuration example

[0013] <1. First embodiment> (1.1. Overview) First, an overview of a cooking system according to a first embodiment of the present disclosure will be described with reference to Figures 1 and 2. Figure 1 is an explanatory diagram showing an overview of the cooking system according to this embodiment. Figure 2 is a block diagram showing the configuration of the cooking system according to this embodiment.

[0014] As shown in FIG. 1, the cooking system according to this embodiment is a system in which a user (cook) 1 in a first space remotely controls a cooking robot 200 in a second space, which is a remote location, to cook.

[0015] 2, a user 1 and an information processing device 100 exist in a first space 10, and a cooking robot 200 and an environment 2 exist in a second space 20 separated from the first space 10. The information processing device 100 in the first space 10 and the cooking robot 200 in the second space 20 are connected to each other via a communication path 30 so that they can communicate with each other.

[0016] The user 1 is a chef or the like who remotely controls the cooking robot 200 located in the second space 20 from the first space 10.

[0017] The information processing device 100 generates control commands including the position, speed, and force required for the operation of the cooking robot 200 based on operation input from the user 1, and transmits the generated control commands to the cooking robot 200 via the communication path 30.

[0018] The user 1 can feel as if he or she is in the second space 20 by visually, aurally, and haptically presenting information acquired by the visual sensors, auditory sensors, and force / tactile sensors provided around the cooking robot 200 in the second space 20. For example, the information processing device 100 may present the information acquired by the visual sensors to the user 1 using a head-mounted display, or may present the information acquired by the auditory sensors to the user 1 using headphones. Furthermore, the information processing device 100 may present the information acquired by the visual sensors to the user 1 using a dome-shaped screen, or may present the information acquired by the auditory sensors to the user 1 using surround speakers. In such cases, the information processing device 100 can present the user with more faithfully reproduced auditory and visual information of the second space 20.

[0019] The operation input from the user 1 may be performed using a general input device such as a mouse, keyboard, touch panel, button, switch, or lever. Alternatively, the operation input from the user 1 may be performed by acquiring the user 1's movements using a mechanical mechanism (e.g., an exoskeleton) attached to the user 1. Furthermore, the operation input from the user 1 may be performed by estimating the posture and movements of the user 1 based on an image of the user 1 or a marker attached to the user 1. In other words, the cooking robot 200 may be operated by directly inputting movements from the user 1, or may be operated to trace the movements of the user 1 based on the sensing results of the user 1's movements.

[0020] The environment 2 includes ingredients to be cooked by the cooking robot 200, cooking utensils used by the cooking robot 200, and a kitchen where the cooking is performed by the cooking robot 200.

[0021] The cooking robot 200 can complete a dish by operating cooking utensils included in the environment 2 and cooking ingredients based on control commands transmitted from the information processing device 100. The cooking robot 200 may be, for example, a humanoid robot equipped with legs and arms, or a robot arm device equipped with only arms.

[0022] The cooking robot 200 may include a visual sensor, an auditory sensor, and a force / tactile sensor for sensing the environment 2. The visual information, auditory information, and force / tactile information acquired by the visual sensor, auditory sensor, and force / tactile sensor are transmitted to the information processing device 100 via the communication path 30 and presented to the user 1. The visual sensor, auditory sensor, and force / tactile sensor may be provided outside the cooking robot 200 as long as they can sense information about the environment 2.

[0023] The communication path 30 is a communication path capable of transmitting and receiving data via a wired or wireless communication network such as the Internet communication network, a LAN (Local Area Network), an infrared communication network, a radio wave communication network, or a satellite communication network. The communication path 30 allows data to be transmitted and received between the first space 10 and the second space 20, which are separated from each other.

[0024] In the cooking system according to this embodiment, the information processing device 100 acquires information about flavor, including at least one of the taste, aroma, and texture, of ingredients contained in the environment 2 of the second space 20, ingredients being cooked, or dishes made from ingredients (hereinafter collectively referred to as cooking objects), and presents the acquired information about flavor to the user 1. This allows the user 1 to adjust the flavor of the dish that has been completed through the cooking process more as intended.

[0025] In particular, cooking is a task in which it is difficult to reproduce the same flavor even after the same cooking process due to variations in ingredients. Furthermore, cooking is a task in which the flavors, such as the taste, aroma, and texture, of the completed dish are controlled by the senses of the user 1 operating the cooking robot 200. Therefore, the information processing device 100 presents the flavors of ingredients sensed in the second space 20 to the user 1 presenting in the first space 10, thereby enabling the user 1 to control the cooking robot 200 so that the flavor of the completed dish meets the user 1's intention.

[0026] (1.2. Configuration example) Next, a configuration example of the cooking system according to this embodiment will be specifically described with reference to Fig. 3. Fig. 3 is a block diagram showing a more specific configuration example of the cooking system according to this embodiment.

[0027] As shown in Figure 3, the cooking system of this embodiment includes a user 1, a flavor output unit 131, an AV output unit 132, an input unit 133, and an information processing device 100 on the first space 10 side, and a cooking robot 200 and an environment 2 on the second space 20 side.

[0028] The information processing device 100 and the cooking robot 200 are communicably connected to each other via a network 31. The network 31 may also be connected to a DB server 32 that stores a database to be referenced when the cooking robot 200 cooks.

[0029] (first space 10 side) As described above, the user 1 is a chef or the like. The user 1 can cook using ingredients and the like contained in the environment 2 by operating the cooking robot 200 located in the second space 20 from the first space 10.

[0030] The input unit 133 is a device that receives an operation input from the user 1. The operation input from the user 1 acquired by the input unit 133 is output to the command generation unit 120 of the information processing device 100.

[0031] For example, the input unit 133 may include a general input device such as a mouse, keyboard, touch panel, button, switch, or lever that accepts direct operations from the user 1 to the cooking robot 200. Alternatively, the input unit 133 may include a detection device that detects the movement of the user 1 as an action traced by the cooking robot 200. For example, the input unit 133 may include a detection device that detects the movement of the user 1 using a mechanical mechanism attached to the user 1, or may include an imaging device that detects the movement of the user 1 using a marker or image recognition.

[0032] If the input unit 133 includes a detection device that detects the movement of the user 1 by a mechanical mechanism attached to the user 1, the input unit 133 may be configured to return a reaction force such as force or vibration to the user 1 when receiving an operation input from the user 1. In this case, the input unit 133 can give the user 1 a sensation as if they are actually experiencing the operation of the cooking robot 200, thereby providing the user 1 with a more realistic sensation. Such technology that provides tactile feedback by returning a reaction force to the user 1 is also called haptics technology.

[0033] The AV output unit 132 outputs images and sounds of the environment 2 acquired by the AV sensor unit 212 in the second space 20 to the user 1. In other words, the AV output unit 132 is an output device that presents visual information and audio information acquired in the second space 20 to the visual and audio senses of the user 1.

[0034] For example, the AV output unit 132 may include a display device such as an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an OLED (Organic Light Emitting Diode) display, a hologram, a projector, or an HMD (Head Mounted Display) as a device for presenting visual information acquired by the AV sensor unit 212 of the second space 20 to the vision of the user 1. The AV output unit 132 may also include a sound output device such as a speaker or headphones as a device for presenting auditory information acquired by the AV sensor unit 212 of the second space 20 to the hearing of the user 1. In order to present more realistic visual or auditory information to the user 1, the AV output unit 132 may be configured to include an HMD and surround-sound headphones used in VR (Virtual Reality) technology, etc.

[0035] The flavor output unit 131 outputs the flavor of the cooking object acquired by the flavor sensor unit 211 in the second space 20 to the user 1. Specifically, the flavor output unit 131 may include a taste output unit that reproduces the taste of the cooking object and presents it to the user 1, an aroma output unit that reproduces the aroma of the cooking object and presents it to the user 1, and a texture output unit that reproduces the texture of the cooking object and presents it to the user 1.

[0036] For example, the taste output unit may present the taste of the cooking object to the taste buds of the user 1 by applying an electrical stimulus to the tongue of the user 1 using a gel-like electrode.

[0037] Specifically, the taste output unit includes a first electrode group including electrodes containing sodium chloride for a salty taste, electrodes containing citric acid for a sour taste, electrodes containing magnesium chloride for a bitter taste, electrodes containing glycine for a sweet taste, and electrodes containing glutamic acid for a umami taste, as well as a second electrode that stimulates the taste buds and serves as a counter electrode to the first electrode group. The taste output unit can form an electrical circuit with the user 1 by contacting the first electrode group that stimulates the taste buds with the tongue of the user 1 and the second electrode with the user 1's hand or the like. Therefore, the taste output unit can control the intensities of the salty, sour, bitter, sweet, and umami tastes sensed by the user 1 on the tongue by controlling the amount of current flowing through each electrode of the first electrode group. In this way, the taste output unit can reproduce any taste by combining the intensities of the salty, sour, bitter, sweet, and umami tastes, thereby presenting the taste of the cooking object in the second space 20 to the taste buds of the user 1 in the first space 10.

[0038] For example, the aroma output unit may present the aroma of the cooking object to the user's olfactory sense by spraying a liquid containing an aroma component near the user's nose.

[0039] Specifically, the aroma output unit includes a cylinder containing multiple liquids, each containing a different aroma component. The aroma output unit sprays the liquid containing the aroma component in the form of a fine mist near the nose of the user 1, thereby presenting the aroma component contained in the liquid to the user 1's sense of smell. Therefore, the aroma output unit can control the aroma perceived by the user 1 by controlling the amount of each liquid containing the aroma component sprayed. In this way, the aroma output unit can reproduce any aroma by combining the strengths of each of the aroma components, thereby presenting the aroma of the cooking object in the second space 20 to the user 1's sense of smell in the first space 10.

[0040] For example, the texture output unit may present the texture of the cooking object to the user 1's tactile sense by having the user 1 bite a mechanism that can arbitrarily change the hardness by jamming transition. Specifically, jamming transition is a phenomenon in which powder behaves as if its hardness varies depending on its density. Therefore, according to jamming transition, the hardness of a bag filled with air can be changed by changing the air pressure inside the bag. In this way, the texture output unit can present the texture of the cooking object in the second space 20 to the user 1's tactile sense in the first space 10 by changing the air pressure inside the bag filled with air and having the user 1 bite the bag.

[0041] Alternatively, the food texture output unit may present the food texture of the cooking object to the tactile sense of the user 1 by applying an electrical stimulus to the jaw of the user 1 based on a database that accumulates myoelectric information of the jaw when a person is chewing. Specifically, the food texture output unit may search for myoelectric information corresponding to the food texture of the cooking object from a database that accumulates myoelectric information, and present an electrical stimulus that reproduces the food texture of the cooking object to the jaw of the user 1 based on the searched myoelectric information. In this way, the food texture output unit can present the food texture of the cooking object in the second space 20 to the tactile sense of the user 1 in the first space 10 by using the electrical stimulus.

[0042] However, it goes without saying that the presentation method is not limited to the above, as long as the flavor output unit 131 can present the flavor of the cooking object to the user 1's senses such as taste, smell, or touch.

[0043] The information processing device 100 includes a flavor presentation unit 110, a command generation unit 120, and a converter 140.

[0044] The flavor presentation unit 110 generates data for presenting the flavor of the cooking object to the user 1 based on information about the flavor of the cooking object sensed by the flavor sensor unit 211. The generated data is output to the flavor output unit 131 and presented to the user 1 via the flavor output unit 131.

[0045] Specifically, the flavor presentation unit 110 may generate data for reproducing the taste, aroma, or texture of the cooking object present in the second space 20 in the flavor output unit 131 of the first space 10 and presenting it to the taste, smell, or touch of the user 1. Alternatively, the flavor presentation unit 110 may generate data for expressing the taste, aroma, or texture of the cooking object present in the second space 20 in numerical form and visualizing and presenting it to the user 1.

[0046] The command generation unit 120 generates a control command to control the behavior of the cooking robot 200 based on the operation input from the user 1 acquired by the input unit 133. Specifically, the command generation unit 120 may generate a control command to control the behavior of the cooking robot 200 based on the behavior of the cooking robot 200 input by the user 1, or may generate a control command to control the behavior of the cooking robot 200 so as to trace the movement of the user 1 based on the movement of the user 1. The generated control command is transmitted to the cooking robot 200 in the second space 20 via the network 31.

[0047] The converter 140 standardizes data transmitted and received between the information processing device 100 and the cooking robot 200 via the network 31. Specifically, the converter 140 standardizes the control command generated by the command generation unit 120 and transmits the standardized control command to the cooking robot 200.

[0048] With this configuration of the first space 10, the user 1 can control the cooking robot 200 toward a target goal while passing through subgoals in both the flavor space and the operation space, as shown in Figures 4A and 4B. Figure 4A is a graph showing an example of a trajectory from a current value s to a target value g in the flavor space. Figure 4B is a graph showing an example of a trajectory from a current value S to a target value G in the operation space.

[0049] For example, in the flavor space, as shown in FIG. 4A, the user 1 can control the cooking robot 200 so that the target value g of the flavor of the cooking object is reached from the current value s of the flavor of the cooking object presented by the flavor output unit 131 via multiple subgoals m1...mn.

[0050] However, what the user 1 actually operates is the cooking robot 200. Therefore, in the operational space, as shown in Fig. 4B, the operation of the cooking robot 200 is controlled so as to pass through a trajectory corresponding to the trajectory from the current value s to the target value g in the flavor space (i.e., a trajectory from the current value S to the target value G via multiple subgoals M1...Mn).

[0051] By referring to a database stored in the DB server 32 (described later), it is also possible to estimate a trajectory from the current value S to the target value G in the operational space, which corresponds to the trajectory from the current value s to the target value g in the flavor space, and the subgoals M1...Mn that the cooking robot 200 passes through. In this case, the cooking robot 200 may be controlled to pass through the estimated trajectory from the current value S to the target value G in the operational space and the subgoals M1...Mn that the cooking robot 200 passes through, without any specific operational input from the user 1. This allows the user 1 to control the operation of the cooking robot 200 so that the flavor of the cooking object reaches the target value g, without any specific operational input or by simple feedback.

[0052] (Second space 20 side) As described above, the environment 2 includes ingredients to be cooked by the cooking robot 200, cooking utensils used by the cooking robot 200, and a kitchen where cooking is performed by the cooking robot 200. Based on control commands from the information processing device 100, the cooking robot 200 can complete a dish by cooking the ingredients included in the environment 2 using the cooking utensils and the kitchen.

[0053] The cooking robot 200 includes a converter 240, a flavor sensor unit 211, an AV sensor unit 212, a robot control unit 213, and a robot driving unit 222.

[0054] The flavor sensor unit 211 is a specific example of a sensing unit that acquires the flavors of cooking objects such as ingredients included in the environment 2, ingredients being cooked, and cooked food. Specifically, the flavor sensor unit 211 may include a taste sensor that senses the taste of the cooking object, an aroma sensor that senses the aroma of the cooking object, and a texture sensor that senses the texture of the cooking object.

[0055] For example, a taste sensor can measure the taste of a cooking object by measuring the membrane potential of a lipid membrane that electrostatically and hydrophobically interacts with a tasting substance contained in the cooking object.

[0056] Specifically, first, a reference membrane potential is obtained by immersing the taste sensor in a reference solution. Next, a first membrane potential is obtained, which is changed by the interaction between the taste substance and the lipid membrane, by contacting the taste sensor with a cooking object. Next, the taste sensor is washed with the reference solution, and then the taste sensor is immersed in the reference solution to obtain a second membrane potential. The second membrane potential is different from the reference membrane potential because bitter and astringent substances among the taste substances are adsorbed to the lipid membrane surface. Then, the taste sensor is washed with alcohol or the like to completely remove the taste substances adsorbed to the lipid membrane surface. The taste sensor can determine the taste of the cooking object from the measured first membrane potential and second membrane potential.

[0057] Note that the taste sensors can measure different tastes depending on the type of lipid membrane, etc. Therefore, the taste sensor may measure the taste of the cooking object using multiple sensors and determine the taste of the cooking object based on the outputs from each of the multiple sensors. For example, the taste sensor may include a bitter sensor capable of measuring mainly bitterness, a sweet sensor capable of measuring mainly sweetness, an umami sensor capable of measuring mainly umami, a salty sensor capable of measuring mainly saltyness, a sour sensor capable of measuring mainly sourness, and an astringent sensor capable of measuring mainly astringentness.

[0058] For example, an aroma sensor can measure the aroma of a cooking object by using a MEMS element in which sensor elements, which are adsorbent to various aroma components and whose vibration frequency changes when the aroma components are adsorbed to the sensor element, are arranged in an array. Note that the aroma sensor may detect the adsorption of aroma components to the sensor element by a change in vibration frequency on the array as described above, or may detect it by a change in the resistance value of the semiconductor that makes up the sensor element, the generation of a potential difference due to the adsorption of aroma components to the electrode of the sensor element, or the generation of deflection due to the adsorption of aroma components to the sensor element.

[0059] For example, a food texture sensor can measure information about the texture of a cooking object by pressing the cooking object. Specifically, first, the food texture sensor measures a force signal when the cooking object is pressed, and then, based on a database in which force signals when an ingredient is pressed with the food texture sensor correspond to jaw myoelectric information when a person masticates the same ingredient, jaw myoelectric information when a person masticates the cooking object is derived. After that, by applying electrical stimulation to the jaw muscles of the user 1 based on the derived jaw myoelectric information, the food texture of the cooking object can be presented to the user 1.

[0060] However, it goes without saying that the flavor sensor unit 211 is not limited to the sensing method described above as long as it can sense the flavor of the cooking object.

[0061] The AV sensor unit 212 acquires images and sounds of the environment 2 and the cooking robot 200. Specifically, the AV sensor unit 212 includes an imaging device that captures images of the environment 2 and a microphone that collects sounds generated in the environment 2. As a result, the AV sensor unit 212 can acquire visual information of the environment 2 by sensing the positional relationship, speed, and acceleration of the ingredients, cooking utensils, and kitchen included in the environment 2 using the imaging device, and acquire auditory information of the environment 2 using the microphone.

[0062] The robot control unit 213 controls the robot driving unit 222 based on the control command generated by the command generating unit 120, thereby controlling the operation of the cooking robot 200. Specifically, the robot control unit 213 may use the trajectory of the cooking robot 200 included in the control command as a reference trajectory and control the robot driving unit 222 so that the difference between the actual trajectory of the cooking robot 200 and the reference trajectory becomes smaller. This allows the robot control unit 213 to control the operation of the cooking robot 200 so that the cooking robot 200 operates based on the control command. For example, the robot control unit 213 may control the driving of the arms, effectors provided at the ends of the arms, and legs of the cooking robot 200.

[0063] The robot driving unit 222 is, for example, a motor or actuator provided in each part of the cooking robot 200. For example, the robot driving unit 222 may be a motor or actuator that rotates each joint of the cooking robot 200, or a motor or actuator that drives wheels or legs provided in a movement mechanism that controls the movement of the cooking robot 200. The robot driving unit 222 is controlled by the robot control unit 213 based on the control command generated by the command generation unit 120, and can cause the cooking robot 200 to operate in accordance with the operation input of the user 1.

[0064] Like the converter 140 of the information processing device 100, the converter 240 standardizes data transmitted and received between the information processing device 100 and the cooking robot 200 via the network 31. Specifically, the converter 240 may standardize data related to visual information, such as the movement, positional relationship, speed, and acceleration of the environment 2 and the cooking robot 200, sensed by the AV sensor unit 212; data related to auditory information, such as sound, sensed by the AV sensor unit 212; and data related to flavor information, such as the taste, aroma, and texture of cooking objects (ingredients, ingredients being cooked, and cooked food) contained in the environment 2, sensed by the flavor sensor unit 211. The converter 240 may also standardize data related to the position, speed, and force of each part of the cooking robot 200, sensed by various sensors provided in each robot driving unit 222 of the cooking robot 200, as data related to the movement of the cooking robot 200. The converter 240 transmits the standardized data to the cooking robot 200.

[0065] (Network 31 and DB server 32) The network 31 is a specific example of a communication path 30 that enables mutual transmission and reception of data between the first space 10 and the second space 20. The network 31 may be a wired or wireless communication network such as an Internet communication network, a LAN (Local Area Network), an infrared communication network, a radio wave communication network, or a satellite communication network.

[0066] The DB server 32 is a server that stores a database that accumulates cooking-related data. Specifically, the DB server 32 may store a database that associates image data for image recognition of ingredients with information about the ingredients. The DB server 32 may also store a database that associates the name of a dish, the state of ingredients before and after cooking, and the action of the cooking robot 200 for cooking. The DB server 32 may also store a database that associates the name of a dish, a presentation method, and information indicating where cooked ingredients should be presented. In addition, the DB server 32 may store a database that associates the molecular structure of an ingredient with the taste, aroma, or texture of the ingredient. In such a case, the DB server 32 may further store a database that associates the molecular structure of an ingredient, which changes due to cooking such as heating or mixing of ingredients, with the taste, aroma, or texture of the ingredient.

[0067] When the database described above is stored in the DB server 32, the cooking robot 200 can estimate a trajectory from a current value to a target value in the flavor space or the operation space by referring to the data accumulated in the database. Therefore, when the user 1 sets a target value for a current value in the flavor space or the operation space, the cooking robot 200 can autonomously set an appropriate subgoal and operate autonomously to perform cooking.

[0068] According to the cooking system of this embodiment, when the cooking robot 200 does not operate as intended by the user 1 due to some error, the user 1 can remotely control the cooking robot 200 to resolve the error and operate as intended by the user 1. For example, if the cooking robot 200 deviates from the trajectory from the current value to the target value in the operational space due to an error, the user 1 can remotely control the cooking robot 200 to return it to any subgoal on the trajectory from the current value to the target value.

[0069] If the flavor of the cooking object deviates from the trajectory from the current value to the target value in the flavor space, it may be difficult to return the flavor of the cooking object to any of the subgoals on the trajectory from the current value to the target value. In such a case, the user 1 may resolve the error by resetting the target value in the flavor space. The DB server 32 may also collect data on such error recovery methods by the user 1 so that the cooking robot 200 can autonomously perform error recovery.

[0070] In addition, to prevent the flavor of the cooking object from deviating from the trajectory in the flavor space, the cooking robot 200 may be controlled to gradually adjust the flavor of the cooking object. Specifically, the robot control unit 213 may set more subgoals for the trajectory in the flavor space included in the control command and control the cooking robot 200 to adjust the flavor of the cooking object via more subgoals.

[0071] (1.3. Variations) Next, a modified example of the cooking system according to this embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a graph showing an example of visual presentation of the flavor of a cooking object. Fig. 6 is a graph showing another example of visual presentation of the flavor of a cooking object.

[0072] 5 and 6, the flavor, including the taste, aroma, or texture of the cooking object, may be quantified and visualized in a graph or the like and presented to the user 1. In such a case, the flavor output unit 131, like the AV output unit 132, may be configured with a display device capable of displaying images, such as an LCD, PDP, OLED display, hologram, projector, or HMD.

[0073] For example, as shown in Fig. 5, the taste and aroma (e.g., sourness, sweetness, saltiness, bitterness, aroma 1, aroma 2, aroma 3, and aroma 4) of a cooking object may be visually presented to the user 1 in a radar chart. Note that although the radar chart in Fig. 5 does not show texture data, the texture data may be shown in the same radar chart as the taste and aroma, or may be shown in a radar chart separate from the taste and aroma.

[0074] 6, the tastes of the cooking object (e.g., sweetness, saltiness, bitterness, sourness, and umami) may be visually presented to the user 1 in the form of a histogram. Note that the histogram in FIG. 6 does not show data on aroma and texture, but the data on aroma and texture may be shown in a histogram like the taste, or may be shown in another representation method.

[0075] This allows the user 1 to visually recognize the difference between the sensed value (current value) of the cooking object and the target (target value), making it easier to control the cooking robot 200. Therefore, the user 1 can operate the cooking robot 200 while checking that the current sensed value of the flavor of the cooking object is approaching the target value.

[0076] 2. Second embodiment (2.1. Configuration example) Next, a configuration example of a cooking system according to a second embodiment of the present disclosure will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a block diagram showing an example of the configuration of a cooking system according to this embodiment. Fig. 8 is a graph diagram showing an example of visual presentation of the flavor of a cooking object in the cooking system according to this embodiment.

[0077] As shown in FIG. 7, the cooking system according to the second embodiment differs from the cooking system according to the first embodiment in that a first virtual space 41 is further provided on the first space 10 side.

[0078] The first virtual space 41 is a virtual space that simulates the second space 20 on the first space 10 side. Specifically, the cooking robot 200 and environment 2 of the second space 20 are virtually provided in the first virtual space 41. The user 1 can virtually cook by operating the virtual cooking robot in the first virtual space 41. The first virtual space 41 may be generated, for example, by the command generation unit 120 of the information processing device 100. The command generation unit 120 can generate control commands for the cooking robot 200 located in the second space 20 based on operations performed by the user 1 on the virtual cooking robot in the first virtual space 41.

[0079] That is, by inputting various information from the cooking robot 200 in the second space 20, the first virtual space 41 simulates the state of the environment 2 in the second space 20, as well as the position, speed, acceleration, and force of the cooking robot 200, and presents them to the user 1. The user 1 can operate the virtual cooking robot in the first virtual space 41 to cook the cooking object. The first virtual space 41 and the second space 20 are connected to each other by exchanging control commands to the virtual cooking robot based on operation inputs from the user 1 and sensing results of the cooking robot 200 and the environment 2 in the second space 20 between the first virtual space 41 and the second space 20.

[0080] The first virtual space 41 can be generated, for example, when the first space 10 and the second space 20 are separated by a distance sufficient to cause a communication delay in communication via the network 31. When a communication delay occurs between the first space 10 and the second space 20, the situation of the environment 2 in the second space 20 is presented to the user 1 in the first space 10 with a delay due to the communication delay, and a control command based on an operation input by the user 1 is transmitted to the cooking robot 200 in the second space 20 with a delay due to the communication delay, etc. Therefore, it is considered difficult for the user 1 to control the cooking robot 200 at an appropriate time.

[0081] The first virtual space 41, located on the first space 10 side, is a virtual space that replicates the second space 20 and has no communication delay with the first space 10. Therefore, the user 1 can cook using the virtual cooking robot without considering communication delays by operating the virtual cooking robot in the first virtual space 41. Operation inputs made by the user 1 to the virtual cooking robot in the first virtual space 41 are transmitted as control commands to the cooking robot 200 in the second space 20. This allows the cooking robot 200 in the second space 20 to operate as intended by the user 1.

[0082] However, due to a time lag in sensing, the situation of the environment 2 and the operation of the cooking robot 200 may not completely match between the first virtual space 41 and the second space 20. For example, the flavor of the cooking object after cooking may not completely match between the first virtual space 41 and the second space 20.

[0083] In such a case, for example, when visually presenting the flavor of the cooking object to the user 1, the sensing value of the second space 20 (i.e., the actual current value of the second space 20) may be presented in addition to the target value and the sensing value of the first virtual space 41 (i.e., the apparent current value of the first virtual space 41), as shown in Fig. 8. This allows the user 1 to visually confirm the difference in the flavor of the cooking object between the first virtual space 41 and the second space 20, and therefore allows the cooking robot 200 to be operated so that the flavor of the cooking object in the second space 20 approaches the target value.

[0084] (2.2. Example of operation) Next, an example of the operation of the cooking system according to this embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example of the operation of the cooking system according to this embodiment.

[0085] 9, user 1 issues command 1 (CMD1) by operating the virtual cooking robot in first virtual space 41. Next, user 1 issues command 2 (CMD2), command 3 (CMD3), and command 4 (CMD4) in sequence in response to changes in first virtual space 41 caused by command 1 (CMD1).

[0086] Command 1 (CMD1), command 2 (CMD2), command 3 (CMD3), and command 4 (CMD4) are transmitted to the cooking robot 200 in the second space 20 with a communication delay, and control the cooking robot 200 to have the cooking robot 200 perform cooking.

[0087] The sensing results (e.g., Sense1, Sense2) of second space 20 are transmitted to first virtual space 41 with a communication delay. Therefore, user 1 can confirm the changes in the situation in second space 20 due to command 1 (CMD1), command 2 (CMD2), command 3 (CMD3), and command 4 (CMD4) in first virtual space 41 with a time lag.

[0088] According to the cooking system of this embodiment, no communication delay occurs between the user 1 in the first space 10 and the first virtual space 41. Therefore, the user 1 can operate the cooking robot 200 by eliminating the time lag (i.e., operation delay) between the operation input due to the communication delay between the first space 10 and the second space 20 and the action of the cooking robot 200.

[0089] 3. Third Embodiment (3.1. Configuration example) Next, a configuration example of a cooking system according to a third embodiment of the present disclosure will be described with reference to Fig. 10. Fig. 10 is a block diagram showing an example of the configuration of a cooking system according to this embodiment.

[0090] As shown in FIG. 10, the cooking system according to the third embodiment differs from the cooking system according to the second embodiment in that a second virtual space 42 is further provided on the second space 20 side.

[0091] The second virtual space 42 is a virtual space that simulates the second space 20 on the second space 20 side. Specifically, the cooking robot 200 and the environment 2 in the second space 20 are virtually provided in the second virtual space 42. A control command transmitted from the information processing device 100 is transmitted to the second virtual space 42 and the cooking robot 200 in the second space 20, respectively, while confirming that the second virtual space 42 has been updated based on the sensing results of the environment 2. The second virtual space 42 may be generated, for example, by the robot control unit 213 of the cooking robot 200.

[0092] That is, the control command generated based on the operation input from the user 1 to the virtual cooking robot in the first virtual space 41 is temporarily stored in a cache in the second virtual space 42. After that, the temporarily stored control command is transmitted sequentially to the virtual cooking robot in the second virtual space 42 and the cooking robot 200 in the second space 20 after checking the status of the environment 2 in the second virtual space 42, which has been updated based on the sensing results of the environment 2 in the second space 20.

[0093] Furthermore, the sensing result of the environment 2 of the second space 20 applied to the second virtual space 42 is applied to the second virtual space (mirror) 52 on the first space 10 side and is presented to the user 1 via the second virtual space (mirror) 52. The second virtual space (mirror) 52 may be generated, for example, by the arithmetic unit of the information processing device 100. Accordingly, if an error occurs in the control of the cooking robot 200 and the situation of the environment 2 of the second space 20 becomes unintended by the user 1, the user 1 can clearly determine the difference between the presence or absence of the error by comparing the first virtual space 41 and the second virtual space (mirror) 52 and resolve the error appropriately.

[0094] The second virtual space 42 can be generated, for example, when the first space 10 and the second space 20 are separated by a distance sufficient to cause a communication delay in communication via the network 31. When a communication delay occurs between the first space 10 and the second space 20, a delay occurs between a control command transmitted from the first space 10 and the cooking robot 200 in the second space 20, which can cause a discrepancy between the control command and the operation of the cooking robot 200.

[0095] The second virtual space 42 provided on the second space 20 side is a virtual space with no communication delay between the second space 20 and the cooking robot 200. Therefore, the cooking robot 200 temporarily stores the control command generated by the command generation unit 120 in a cache of the second virtual space 42 and applies it to the control of its operation while checking the situation of the environment 2 in the second space 20, thereby enabling the cooking robot 200 to operate without any discrepancy between the control command and its operation.

[0096] (3.2. Example of operation) Next, an operation example of the cooking system according to this embodiment will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a block diagram showing an example of the operation of the cooking system according to this embodiment. Fig. 12 is a block diagram showing another example of the operation of the cooking system according to this embodiment.

[0097] The operation example of the cooking system shown in FIG. 11 is an operation example when no error occurs on the cooking robot 200 side.

[0098] 11, user 1 issues command 1 (CMD1) by operating the virtual cooking robot in first virtual space 41. Next, user 1 issues command 2 (CMD2), command 3 (CMD3), and command 4 (CMD4) in sequence in response to changes in first virtual space 41 caused by command 1 (CMD1).

[0099] Command 1 (CMD1), Command 2 (CMD2), Command 3 (CMD3), and Command 4 (CMD4) are transmitted to the cache of the second virtual space 42 with a communication delay and temporarily stored. The second virtual space 42 is updated with the sensing results (e.g., Sense1, Sense2, Sense3, Sense4) of the second space 20. Command 1 (CMD1), Command 2 (CMD2), Command 3 (CMD3), and Command 4 (CMD4) are transmitted sequentially from the cache to the virtual cooking robot in the second virtual space 42 and the cooking robot 200 in the second space 20 while checking the sensing results (e.g., Sense1, Sense2, Sense3, Sense4) of the second space 20.

[0100] Furthermore, the sensing results (e.g., Sense1, Sense2, Sense3, Sense4) of second space 20 are reflected in second virtual space 42, and then transmitted to second virtual space (mirror) 52 on the first space 10 side with a communication delay, updating the situation in second virtual space (mirror) 52. Therefore, user 1 can confirm the changes in the situation in second space 20 due to command 1 (CMD1), command 2 (CMD2), command 3 (CMD3), and command 4 (CMD4) in second virtual space (mirror) 52 with a time lag.

[0101] The operation example of the cooking system shown in FIG. 12 is an operation example when an error occurs on the cooking robot 200 side.

[0102] 12, similar to Fig. 11, user 1 issues command 1 (CMD1) by operating the virtual cooking robot in first virtual space 41. Next, user 1 issues command 2 (CMD2), command 3 (CMD3), and command 4 (CMD4) in sequence in response to changes in first virtual space 41 caused by command 1 (CMD1).

[0103] Command 1 (CMD1), Command 2 (CMD2), Command 3 (CMD3), and Command 4 (CMD4) are transmitted to the cache of the second virtual space 42 with a communication delay and temporarily stored. The second virtual space 42 is updated with the sensing result (e.g., Sense1) of the second space 20, and Command 1 (CMD1) is transmitted from the cache to the virtual cooking robot in the second virtual space 42 and the cooking robot 200 in the second space 20 while checking the sensing result (e.g., Sense1) of the second space 20.

[0104] Here, suppose that after the cooking robot 200 in the second space 20 is controlled by command 1 (CMD1), an error occurs and the cooking robot 200 in the second space 20 does not operate normally. In such a case, command 2 (CMD2), command 3 (CMD3), and command 4 (CMD4) following command 1 (CMD1) are not transmitted to the virtual cooking robot in the second virtual space 42 and the cooking robot 200 in the second space 20, and remain temporarily stored in the cache.

[0105] Thereafter, the sensing results of the situation in the second space 20 (e.g., Sense1, Sense5, Sense6, and Sense7) are reflected in the second virtual space 42 and then transmitted to the second virtual space (mirror) 52 in the first space 10 with a communication delay, thereby updating the situation in the second virtual space (mirror) 52. This allows the user 1 to confirm with a time lag in the second virtual space (mirror) 52 that the environment 2 in the second space 20 has changed to an unintended situation after command 1 (CMD1). Furthermore, by checking the difference between the first virtual space 41 and the second virtual space (mirror) 52, the user 1 can understand the action to be taken by the cooking robot 200 in the second space 20 to resolve the error.

[0106] According to the cooking system of this embodiment, no communication delay occurs between the second virtual space 42 and the second space 20. Therefore, the cooking robot 200 can operate by eliminating the time lag (i.e., control delay) between a control command and the operation of the cooking robot 200.

[0107] 4. Fourth Embodiment (4.1. Overview) Next, an overview of a cooking system according to a fourth embodiment of the present disclosure will be described with reference to FIGS. 13A and 13B. In the cooking system according to the fourth embodiment, a kitchen and ingredients are also arranged in the first space 10, and the user 1 actually cooks using the kitchen and ingredients in the first space 10. The cooking system according to the fourth embodiment observes the movements of the user 1 and the movements of ingredients and cooking utensils in the kitchen using sensors and the like, and can cause the cooking robot 200 in the second space 20 to cook based on the observation results. FIG. 13A is a schematic diagram showing an example of the arrangement of objects in the first space 10. FIG. 13B is a schematic diagram showing an example of the arrangement of objects in the second space 20.

[0108] For example, when the user 1 cooks in the first space 10 and operates the cooking robot 200 in the second space 20 to trace the movements of the user 1, the arrangement of objects in the first space 10 may differ from the arrangement of objects in the second space, as shown in Figures 13A and 13B. Specifically, the positions of the kitchen sink, stove, and free space (area 1), the positions of ingredients, and the positions of cooking utensils such as a cutting board and knife may differ between the first space 10 and the second space 20.

[0109] In such a case, it is difficult for the cooking robot 200 to perform cooking in the same way as the user 1, even if it moves its arms or the like to trace the movements of the user 1. In other words, it is important for the cooking robot 200 to understand the meaning of the cooking steps performed by the user 1 in the first space 10, and then perform operations in the second space 20 so as to perform cooking steps with similar meanings.

[0110] In the cooking system according to this embodiment, the information processing device 100 understands the meaning of the user 1's movements in the first space 10 and transmits a control command based on the understood meaning to the cooking robot 200. The cooking robot 200 performs an action in the second space 20 that is equivalent to the meaning of the user 1's movements, thereby being able to perform cooking similar to that performed by the user 1 even if the arrangement of objects is different between the first space 10 and the second space.

[0111] (4.2. Configuration example) Next, a configuration example of the cooking system according to this embodiment will be specifically described with reference to Fig. 14 to Fig. 15B. Fig. 14 is a block diagram showing a more specific configuration example of the cooking system according to this embodiment. Fig. 15A and Fig. 15B are schematic diagrams illustrating the correspondence between the arrangement of objects in the first space 10 and the arrangement of objects in the second space 20.

[0112] As shown in FIG. 14 , the cooking system according to the fourth embodiment differs from the cooking system according to the first embodiment in that a first virtual space 41 and a second virtual space (mirror) 52 are provided on the first space 10 side, and a second virtual space 42 and a first virtual space (mirror) 51 are provided on the second space 20 side. The environments 2 in which cooking is performed are different between the first space 10 and the second space 20, and the first space 10 and the second space 20 are associated with each other based on the meaning of information. The first virtual space 41 may be generated by the command generator 120 or the like, and the second virtual space (mirror) 52 may be generated by a computing device of the information processing device 100. The second virtual space 42 may be generated by the robot control unit 213 or the like, and the first virtual space (mirror) 51 may be generated by a computing device of the cooking robot 200.

[0113] Specifically, as shown in FIG. 15A, each part of the kitchen, ingredients, and cooking utensils (hereinafter collectively referred to as objects) in the first space 10 are imaged by an imaging device 151. The object recognition unit 152 can recognize the names (i.e., meanings) and positions (i.e., coordinates of each vertex, etc.) of the objects in the first space 10 by performing image recognition based on the images captured by the imaging device 151. Similarly, as shown in FIG. 15B, each part of the kitchen, ingredients, and cooking utensils (hereinafter collectively referred to as objects) in the second space 20 are imaged by an imaging device 251. The object recognition unit 252 can recognize the names (i.e., meanings) and positions (i.e., coordinates of each vertex, etc.) of the objects in the second space 20 by performing image recognition based on the images captured by the imaging device 251. In this way, the objects in the first space 10 and the second space 20 are associated with each other according to their names or meanings, and thus differences in their positions can be recognized.

[0114] The object recognition units 152, 252 may recognize the names (i.e., meanings) and positions of each object in the first space 10 and the second space 20 by image recognition using machine learning technology such as a deep neural network. Alternatively, the information processing device 100 may present images of the first space 10 and the second space 20 to the user 1, allowing the user 1 to associate and locate each object in the first space 10 and the second space 20. Furthermore, the information processing device 100 may associate and locate each object in the first space 10 and the second space 20 by using a combination of image recognition by the object recognition units 152, 252 and manual association by the user 1.

[0115] In the cooking system according to this embodiment, first, the user 1 performs actions to perform cooking processes in the first virtual space 41, which is a simulation of the first space 10, in the first space 10. The user 1's actions are transmitted via the network 31 to the first virtual space (mirror) 51, which is a reproduction of the first virtual space 41 in the second space 20. The user 1's actions in the first virtual space (mirror) 51 are then converted into actions of a virtual cooking robot in the second virtual space 42, which is a simulation of the second space 20, in the second space 20, based on the meaning of the actions. The actions of the virtual cooking robot in the second virtual space 42 are then reflected in the actions of the actual cooking robot 200 in the second space 20.

[0116] Furthermore, changes in the environment 2 in the second space 20 due to the operation of the cooking robot 200 are reflected in the second virtual space 42, and then reflected in the second virtual space (mirror) 52 that reproduces the second virtual space 42 in the first space 10. Subsequently, the situation of the environment 2 in the second virtual space (mirror) 52 is converted into the situation of the environment 2 in the first virtual space 41 by association based on meaning.

[0117] According to this, the cooking system of this embodiment allows the user 1 to perform an action on objects that are associated with each other in the first space 10 and the second space 20, and causes the cooking robot 200 in the second space 20 to perform an action with the same meaning as the action of the user 1.

[0118] For example, if user 1 in first space 10 performs the action of "placing the cutting board on area 1," the cooking robot 200 in second space 20 can recognize user 1's action as the following action command and place the "cutting board" in second space 20 on "area 1." 1. If Hand1(Clear) then PickUp(Chopping Board) 2.If Hand1(cutting board) then Move(Hand1, On(area 1)) 3.If Hand1(on(area 1)) then Release(Hand1)

[0119] The reason why confirmations are made for each step, such as "If Hand1 (Clear)," "If Hand1 (Chopping Board)," and "If Hand1 (on (Area 1))," is to ensure that the cooking robot 200 executes the operation only if the preconditions for the operation are met. If the preconditions for the operation are not met, the cooking robot 200 may stop executing the operation and return to the previous step to redo the operation. For example, after rechecking the current positions of "Hand1" and the "Chopping Board," the status of "Hand1" may be set to "Clear," and the above steps of the operation may be redone from the beginning. If the error cannot be resolved using the above methods, the cooking robot 200 may request the user 1 to resolve the error.

[0120] According to the above configuration, the cooking system of this embodiment can control the operation of the cooking robot 200 in the second space 20 by the operation of the user 1 in the first space 10, even between the first space 10 and the second space 20, where the object arrangement is different from each other. Therefore, according to the cooking system of this embodiment, the cooking robot 200 in the second space 20, where the object arrangement is different from that in the first space 10, can perform cooking similar to that of the user 1 in the first space 10.

[0121] (4.3. Variations) Next, a modified example of the cooking system according to this embodiment will be described with reference to Fig. 16. Fig. 16 is a block diagram showing an example of the configuration of the modified example of the cooking system according to this embodiment.

[0122] As shown in FIG. 16, the cooking system according to the modification of this embodiment differs from the cooking system according to the fourth embodiment in that a selector 160 is further provided in the information processing device 100.

[0123] The selector 160 alternates between the first virtual space 41 and the second virtual space (mirror) 52 as the virtual space that the user 1 can view or operate. By using the selector 160, the user 1 can directly view the situation in the second virtual space (mirror) 52, which simulates the second space 20, from the first space 10, and directly operate the virtual cooking robot in the second virtual space (mirror) 52. This allows the user 1, when an error occurs in the cooking robot 200 in the second space 20, to check the cause of the error in the second virtual space (mirror) 52 and directly operate the virtual cooking robot in the second virtual space (mirror) 52, thereby directly resolving the error that occurred in the second space 20.

[0124] <5. Hardware configuration example> Furthermore, the hardware configuration of the information processing device 100 included in the cooking system according to each embodiment of the present disclosure will be described with reference to Fig. 17. Fig. 17 is a block diagram showing an example of the hardware configuration of the information processing device 100.

[0125] The functions of the information processing device 100 according to this embodiment can be realized by cooperation between software and the hardware described below. The functions of the flavor presentation unit 110, the command generation unit 120, the converter 140, and the selector 160 may be executed by, for example, the CPU 901.

[0126] As shown in FIG. 17, the information processing device 100 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 903, and a RAM (Random Access Memory) 905.

[0127] The information processing device 100 may further include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, or a communication device 925. The information processing device 100 may further include an imaging device 933 or a sensor 935, as necessary. The information processing device 100 may have a processing circuit such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) instead of or in addition to the CPU 901.

[0128] The CPU 901 functions as an arithmetic processing device or a control device, and controls operations within the information processing device 100 in accordance with various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 927. The ROM 903 stores programs used by the CPU 901, calculation parameters, etc. The RAM 905 temporarily stores programs used in the execution of the CPU 901, and parameters used during the execution of the programs.

[0129] The CPU 901, ROM 903, and RAM 905 are interconnected by a host bus 907, which is an internal bus such as a CPU bus. The host bus 907 is further connected via a bridge 909 to an external bus 911, such as a PCI (Peripheral Component Interconnect / Interface) bus.

[0130] The input device 915 is a device that accepts input from a user, such as a mouse, keyboard, touch panel, button, switch, lever, or exoskeleton. The input device 915 may also be a microphone that detects the user's voice. The input device 915 may also be, for example, a remote control device that uses infrared rays or other radio waves, or an externally connected device 929 that supports operation of the information processing device 100.

[0131] The input device 915 further includes an input control circuit that outputs an input signal generated based on information input by the user to the CPU 901. By operating the input device 915, the user can input various data to the information processing device 100 or instruct the information processing device 100 to perform processing operations.

[0132] The output device 917 is a device that can visually or audibly present information acquired or generated by the information processing device 100 to a user. The output device 917 may be, for example, a display device such as an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an OLED (Organic Light Emitting Diode) display, a hologram, a projector, or a head-mounted display, a sound output device such as a speaker or headphones, or a printing device such as a printer. The output device 917 can output information acquired by processing by the information processing device 100 as video such as text or an image, or sound such as voice or audio.

[0133] The storage device 919 is a data storage device configured as an example of a storage unit of the information processing device 100. The storage device 919 may be configured, for example, by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 can store programs executed by the CPU 901, various data, various data acquired from the outside, and the like.

[0134] The drive 921 is a device for reading or writing data from or to a removable recording medium 927 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built into or externally attached to the information processing device 100. For example, the drive 921 can read information recorded on the attached removable recording medium 927 and output the information to the RAM 905. The drive 921 can also write data to the attached removable recording medium 927.

[0135] The connection port 923 is a port for directly connecting an external device 929 to the information processing device 100. The connection port 923 may be, for example, a Universal Serial Bus (USB) port, an IEEE 1394 port, or a Small Computer System Interface (SCSI) port. The connection port 923 may also be an RS-232C port, an optical audio terminal, or a High-Definition Multimedia Interface (HDMI) (registered trademark) port. When the connection port 923 is connected to the external device 929, various types of data can be transmitted and received between the information processing device 100 and the external device 929.

[0136] The communication device 925 is, for example, a communication interface configured with a communication device for connecting to the communication network 931. The communication device 925 may be, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 925 may also be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0137] The communication device 925 can transmit and receive signals, for example, via the Internet or other communication devices using a predetermined protocol such as TCP / IP. The communication network 931 connected to the communication device 925 is a wired or wireless network, and may be, for example, the Internet communication network, a home LAN, an infrared communication network, a radio wave communication network, or a satellite communication network.

[0138] It is also possible to create a program that causes hardware such as the CPU 901, ROM 903, and RAM 905 built into a computer to perform functions equivalent to those of the information processing device 100. It is also possible to provide a computer-readable recording medium on which the program is recorded.

[0139] The technology according to the present disclosure has been described above using the first to fourth embodiments and modifications. However, the technology according to the present disclosure is not limited to the above embodiments, and various modifications are possible.

[0140] Furthermore, not all of the configurations and operations described in each embodiment are necessarily essential to the configurations and operations of the present disclosure. For example, among the components in each embodiment, any component not recited in an independent claim showing the highest concept of the present disclosure should be understood as an optional component.

[0141] Terms used throughout this specification and the appended claims should be interpreted as "open-ended" terms. For example, the terms "including" or "including" should be interpreted as "not limited to the manner described as including." The term "having" should be interpreted as "not limited to the manner described as having."

[0142] The terms used in this specification include terms that are used merely for the convenience of description and are not intended to limit the configuration or operation. For example, terms such as "right," "left," "upper," and "lower" merely indicate directions in the drawings to which reference is made. Furthermore, the terms "inner" and "outer" merely indicate directions toward and away from the center of a focused element, respectively. The same applies to similar terms and terms of a similar meaning.

[0143] The technology according to the present disclosure may also be configured as follows. According to the technology according to the present disclosure having the following configuration, a user in a first space can grasp the flavor of a cooking object in a second space separated from the first space. Therefore, an information processing device and a cooking system according to an embodiment of the present disclosure can present more sensory information sensed at a remote location to a user operating a robot at a remote location. The effects achieved by the technology according to the present disclosure are not necessarily limited to the effects described herein and may be any of the effects described in the present disclosure. (1) a flavor presentation unit that generates data for presenting the flavor of a cooking object present in a second space separated from the first space to a user present in the first space; a command generating unit that generates a control command to control an operation of the cooking robot present in the second space based on the input from the user; An information processing device comprising: (2) The information processing device according to (1) above, wherein the cooking object is ingredients before entering the cooking process, the ingredients cooked in the cooking process, or a dish that has been completed after going through all of the cooking processes. (3) The information processing device according to (1) or (2) above, wherein the flavor includes at least one of the taste, aroma, or texture of the cooking object. (4) The information processing device according to any one of (1) to (3) above, wherein the flavor presentation unit generates data for reproducing the flavor and presenting it to the senses of the user. (5) The information processing device according to (4) above, wherein the senses include at least one of taste, smell, and touch. (6) The information processing device according to any one of (1) to (5) above, wherein the flavor presentation unit generates data for visualizing the flavor and presenting it to the user. (7) The information processing device according to (6) above, wherein the flavor presentation unit generates data for superimposing and visualizing the sensed flavor of the cooking object and the target flavor. (8) The information processing device described in any one of (1) to (7) above, wherein the command generation unit generates a first virtual space that simulates the second space on the first space side, and generates the control command that controls the operation of the cooking robot based on the input made by the user to the first virtual space. (9) The information processing device described in any one of (1) to (8) above, wherein the control command controls the operation of the cooking robot in a second virtual space generated on the second space side based on the sensing results for the second space. (10) The information processing device according to any one of (1) to (9) above, wherein the input from the user is an operation instruction for the cooking robot. (11) The information processing device according to any one of (1) to (9) above, wherein the input from the user is data obtained by sensing a cooking operation of the user. (12) The information processing device according to any one of (1) to (11) above, wherein the control command includes a current value, a target value, and a trajectory from the current value to the target value in the operational space of the cooking robot. (13) The information processing device according to (12), wherein the control command further includes an intermediate value set between the current value and the target value. (14) The first space and the second space have different object arrangements, The information processing device according to any one of (1) to (13) above, wherein the cooking robot is controlled in operation by the control command in consideration of a difference in arrangement of the objects in the first space and the second space. (15) The information processing device according to (14) above, wherein the objects in the first space and the second space are associated with each other in meaning by image recognition of objects. (16) The information processing device according to (15) above, wherein the command generation unit grasps the meaning of the input from the user and generates the control command having a meaning corresponding to the meaning of the input. (17) The information processing device according to any one of (1) to (16) above, wherein the control command generated in the first space is transmitted to the cooking robot present in the second space via a communication path. (18) The information processing device according to (17), wherein the first space and the second space are separated from each other by a distance equal to or greater than a distance at which a communication delay occurs in the communication path. (19) a sensing unit that senses the flavor of a cooking object present in the second space; a flavor presentation unit that generates data for presenting the sensed flavor of the cooking object to a user present in a first space separated from the second space; a command generation unit that generates a control command based on the input from the user; a cooking robot whose operation is controlled based on the control command and which cooks the cooking object present in the second space; A cooking system comprising:

[0144] This application claims priority based on Japanese Patent Application No. 2020-188272, filed on November 11, 2020, with the Japan Patent Office, the entire contents of which are incorporated herein by reference.

[0145] Those skilled in the art will recognize that various modifications, combinations, subcombinations, and variations may occur depending on design requirements and other factors, and are intended to be within the scope of the appended claims and their equivalents.

Claims

1. a flavor presentation unit that generates data for presenting a flavor of a cooking object present in a second space separated from the first space to a user present in the first space; a command generating unit that generates a control command to control an operation of the cooking robot present in the second space based on an input from the user; Equipped with The first space and the second space have different object arrangements, and the cooking robot is controlled to operate in accordance with the control command in consideration of the difference in the object arrangements between the first space and the second space. Information processing device.

2. The cooking object is an ingredient before the cooking process, an ingredient cooked in the cooking process, or a dish completed after going through all the cooking processes. The information processing device according to claim 1 .

3. The flavor comprises at least one of the taste, aroma, and texture of the cooking object. The information processing device according to claim 1 .

4. the flavor presentation unit generates data for reproducing the flavor and presenting it to the senses of the user. The information processing device according to claim 1 .

5. The senses include at least one of taste, smell, and touch. The information processing device according to claim 4 .

6. the flavor presentation unit generates data for visualizing the flavor and presenting it to the user. The information processing device according to claim 1 .

7. The flavor presentation unit generates data for superimposing and visualizing the sensed flavor of the cooking object and the target flavor. The information processing device according to claim 6 .

8. the command generation unit generates a first virtual space that simulates the second space in the first space, and generates the control command that controls the operation of the cooking robot based on an input made by the user to the first virtual space. The information processing device according to claim 1 .

9. The control command controls an operation of the cooking robot in a second virtual space generated on the second space side based on a sensing result for the second space. The information processing device according to claim 1 .

10. The input from the user is an operation instruction for the cooking robot. The information processing device according to claim 1 .

11. The input from the user is data obtained by sensing a cooking operation of the user. The information processing device according to claim 1 .

12. The control command includes a current value, a target value, and a trajectory from the current value to the target value in the operational space of the cooking robot. The information processing device according to claim 1 .

13. The control command further includes an intermediate value set between the current value and the target value. The information processing device according to claim 12.

14. The objects in the first space and the second space are associated with each other in meaning by image recognition of the objects. The information processing device according to claim 1 .

15. the command generation unit grasps the meaning of the input from the user and generates the control command having a meaning corresponding to the meaning of the input. The information processing device according to claim 14.

16. The control command generated in the first space is transmitted to the cooking robot located in the second space via a communication path. The information processing device according to claim 1 .

17. The first space and the second space are separated from each other by a distance greater than or equal to a distance at which a communication delay occurs in the communication path. The information processing device according to claim 16.

18. a sensing unit that senses the flavor of a cooking object present in the second space; a flavor presentation unit that generates data for presenting the sensed flavor of the cooking object to a user present in a first space separated from the second space; a command generation unit that generates a control command based on the input from the user; a cooking robot whose operation is controlled based on the control command and which cooks the cooking object present in the second space; Equipped with The first space and the second space have different object arrangements, and the cooking robot is controlled to operate in accordance with the control command in consideration of the difference in the object arrangements between the first space and the second space. Cooking system.

Images