Information processing device and program

Abstract

This information processing device can improve resolution of a taste that can be reproduced in the oral cavity of a person. The information processing device: causes a display unit to display an image which is indicated by image data indicating the image and in which taste data indicating one or more tastes is associated with each of at least some pixels; and outputs, on the basis of an operation received via the image displayed on the display unit, the taste data associated with the pixel designated by the operation.

Description

Information processing device and program

[0001] The present invention relates to an information processing device and a program.

[0002] Research and development is underway on a device that can reproduce the taste of food displayed on a screen in a person's mouth.

[0003] In this regard, a device is known that can reproduce different tastes in the human mouth depending on the playback of a video by accepting and associating the distribution of the five basic tastes for each frame of a loaded video file (see Non-Patent Document 1).

[0004] Yoshiaki Miyashita, "Development of a taste display that reproduces the taste of food displayed on the screen," Proceedings of the 28th Workshop on Interactive Systems and Software (WISS2020), pp. 103-108 (2020).

[0005] Here, the taste of food appearing in a video may vary from part to part. However, because the device described in Non-Patent Document 1 associates the distribution of the five basic tastes with each frame, it is not possible to reproduce the difference in taste of each part of the food contained in the frame in the human mouth. This means that the resolution of the taste of the food appearing in the video is low, which is undesirable.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems of the conventional technology, and provides an information processing device and a program that can improve the resolution of tastes that can be reproduced in the human oral cavity.

[0007] In order to solve the above-mentioned problems, an information processing device according to one embodiment of the present invention displays an image represented by image data showing an image in which taste data indicating one or more tastes is associated with at least some of the pixels on a display unit, and outputs the taste data associated with the pixels specified by an operation based on an operation received via the image displayed on the display unit.

[0008] In addition, in order to solve the above-mentioned problems, a program according to one aspect of the present invention causes a computer to display on a display unit an image represented by image data showing an image in which taste data indicating one or more tastes is associated with at least some of the pixels, and based on an operation received via the image displayed on the display unit, output the taste data associated with the pixel specified by the operation.

[0009] According to the present invention, it is possible to provide an information processing device and a program that can improve the ability to resolve tastes that can be reproduced in the human oral cavity.

[0010] 1 is a diagram showing an example of the configuration of an information processing system 1. FIG. 1 is a diagram showing an example of a mixing ratio indicated by certain taste data. FIG. 2 is a diagram showing an example of a designated target frame displayed on a display unit provided in an information processing device 10. FIG. 3 is a diagram showing an example of a state in which two areas are designated as designated target areas in the designated target frame F1 shown in FIG. 3. FIG. 4 is a diagram showing an example of a file structure of FlavMP4. FIG. 5 is a diagram showing an example of the hardware configuration of an information processing device 10. FIG. 6 is a diagram showing an example of the functional configuration of an information processing device 10. FIG. 7 is a diagram showing an example of a processing flow in which the information processing device 10 outputs taste data by accepting a first operation and a second operation. FIG. 8 is a diagram showing an example of a processing flow in which the information processing device 10 captures a designated target frame as a second taste presentation image from a first taste presentation image, which is a moving image. FIG. 9 is a diagram showing an example of a processing flow in which the information processing device 10 displays a taste palette image and displays the second taste presentation image on the taste palette.

[0011] <Embodiments> Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] <Outline of Information Processing System> First, an outline of the information processing system according to the embodiment will be described.

[0013] An information processing device according to an embodiment includes an information processing device and a taste reproduction device. Here, the information processing device displays an image on a display unit, with taste data indicating one or more tastes associated with at least some of the pixels, and outputs the taste data associated with the pixels specified by an operation received via the image displayed on the display unit. Furthermore, the taste reproduction device reproduces one or more tastes indicated by the taste data output from the information processing device.

[0014] With this configuration, the information processing system according to the embodiment can improve the ability to resolve tastes that can be reproduced in the human oral cavity.

[0015] The configuration of an information processing device included in an information processing system according to an embodiment and the processing performed by the information processing device will be described in detail below.

[0016] <Configuration of Information Processing System> Hereinafter, the configuration of the information processing system according to the embodiment will be described by taking an information processing system 1 as an example of the information processing system according to the embodiment.

[0017] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system 1. As shown in FIG.

[0018] The information processing system 1 displays an image on the display unit, in which taste data indicating one or more tastes is associated with at least some of the pixels. For ease of explanation, the image will be referred to as the first taste presentation image below. The first taste presentation image may be a moving image or a still image. If the first taste presentation image is a moving image, taste data is associated with each frame of the moving image. As an example, the first taste presentation image will be described below. Details of the first taste presentation image will be described later.

[0019] Furthermore, the information processing system 1 outputs taste data associated with pixels designated by an operation received via the first taste presentation image displayed on the display unit, thereby improving the resolution of tastes that can be reproduced in the human oral cavity.

[0020] The information processing system 1 includes an information processing device 10 and a taste reproduction device 20. In the information processing system 1, the information processing device 10 may be configured integrally with the taste reproduction device 20. Alternatively, the information processing system 1 may not include the taste reproduction device 20. In this case, the information processing system 1 is the information processing device 10 itself.

[0021] The information processing device 10 is, for example, an information processing device such as a notebook PC (Personal Computer), a tablet PC, a multi-function mobile phone terminal (smartphone), a mobile phone terminal, a PDA (Personal Digital Assistant), a desktop PC, or a workstation, but is not limited to these.

[0022] The information processing device 10 is communicably connected to the taste reproduction device 20 via a wireless or wired communication network.

[0023] The information processing device 10 displays the first taste presentation image on a display unit. The display unit on which the information processing device 10 displays the first taste presentation image may be a display unit included in the information processing device 10, a display unit of another information processing device connected to be able to communicate with the information processing device 10, a display device connected to be able to communicate with the information processing device 10, or another display unit. Below, as an example, a case will be described where the display unit on which the information processing device 10 displays the first taste presentation image is a display unit included in the information processing device 10.

[0024] The information processing device 10 may be configured to be capable of generating the first taste presentation image, or may be configured to acquire the first taste presentation image from another device.

[0025] The information processing device 10 outputs taste data associated with a pixel designated by an operation received via the first taste presentation image displayed on the display unit, based on the operation. Here, the output destination of the taste data by the information processing device 10 may be the display unit included in the information processing device 10, the taste reproduction device 20, or another device.

[0026] Here, taste data will be described. Taste data is information indicating the mixing ratio of substances that reproduce one or more tastes in the human oral cavity. The one or more tastes may be, for example, the five basic tastes. The five basic tastes are sweet, salty, sour, bitter, and umami. The one or more tastes may include some or all of the five basic tastes plus other tastes such as spiciness or astringency, or may include the other tastes instead of all of the five basic tastes. In the following, as an example, a case will be described in which the one or more tastes are the five basic tastes. In the following, for convenience of explanation, each of the one or more tastes will be referred to as a taste to be mixed.

[0027] A certain target taste to be mixed is associated with one or more substances that reproduce the target taste in the human mouth. For example, if the target taste to be mixed is sour, four substances, malic acid, citric acid, acetic acid, and lactic acid, are associated with the sour taste. In this case, the taste data for sourness is information indicating the mixing ratio of each of the four substances associated with each of the five basic tastes when these substances are mixed. For simplicity's sake, the following description will be given assuming that only one substance is associated with each target taste to be mixed. In this case, for example, sucrose is associated with sweetness. In this case, for example, table salt is associated with salt. In this case, citric acid is associated with sourness. In this case, potassium carbonate is associated with bitterness. In this case, monosodium glutamate is associated with umami. For example, when the ratio of sucrose: salt: citric acid: potassium carbonate: monosodium glutamate is 4:1:2:1:2, and the maximum value of the mixing ratio is 1, the taste data for sucrose is 0.4. In this case, the taste data for salt is 0.1. In this case, the taste data for citric acid is 0.2. In this case, the taste data for potassium carbonate is 0.1. In this case, the taste data for monosodium glutamate is 0.2. In this way, the taste data indicates the mixing ratio of substances associated with each of the tastes to be mixed.

[0028] Fig. 2 is a diagram showing an example of a mixing ratio indicated by certain taste data. The horizontal axis of the graph shown in Fig. 2 indicates each of N tastes to be mixed. N may be any integer equal to or greater than 1. The vertical axis of the graph indicates the mixing ratio whose maximum value is represented by 1. As shown in Fig. 2, the taste data indicates the mixing ratio of substances associated with each of the first to Nth tastes.

[0029] Furthermore, the operations that the information processing device 10 receives via the first taste presentation image include a first operation of designating a desired frame from among frames of a moving image, and a second operation of designating one or more pixels as designation target pixels from among a plurality of pixels of a display unit included in the information processing device 10. Hereinafter, for convenience of explanation, the desired frame will be referred to as a designation target frame.

[0030] The first operation may be any operation that allows one of the frames of the first taste presentation image, which is a moving image, to be selected as the designated frame. For example, the first operation may be an operation that pauses playback of the first taste presentation image to display the desired frame on the display unit as the designated frame until it is no longer necessary to display it, but may instead be another operation.

[0031] The second operation may be any operation capable of designating one or more pixels among a plurality of pixels included in the display unit of the information processing device 10 as the designated target pixel. For example, the second operation is an operation of designating one or more regions on the display unit of the information processing device 10. In this case, the designated target pixel designated by the second operation is one or more pixels among the plurality of pixels when the designated target frame is displayed on the display unit of the information processing device 10, which overlap with the one or more regions designated by the second operation. For convenience of explanation, each of the one or more regions designated by the second operation will be referred to as a designated target region below. When the designated target region is a dot-like region, the pixel overlapping with the designated target region refers to one pixel among the plurality of pixels included in the display unit of the information processing device 10 that overlaps with the dot-like region. When the designated target region is a linear region, the pixel overlapping with the designated target region refers to two or more pixels among the plurality of pixels included in the display unit of the information processing device 10 that overlap with the linear region and are arranged in a line parallel to the linear region. Furthermore, when the specified target area is a planar area, the pixels that overlap with the specified target area are the multiple pixels of the display unit provided in the information processing device 10 that overlap with the planar area and extend in a planar shape parallel to the planar area.

[0032] 3A and 3B are diagrams illustrating an example of a designation target frame displayed on a display unit included in the information processing device 10. The designation target frame F1 shown in FIG. 3A illustrates an example of a designation target frame before a designation target area is designated by a second operation. The designation target frame F1 displays two pasta dishes: pasta P1 with squid ink sauce and pasta P2 with tomato sauce. Therefore, among the multiple pixels of the display unit when the designation target frame F1 is displayed, pixels displaying a portion of the pasta P1 are associated with taste data indicating the mixing ratio of substances associated with one or more target flavors to be mixed to recreate the taste of that portion of the pasta P1. For example, pixels displaying only parsley sprinkled on the pasta P1 are associated with taste data indicating the mixing ratio of substances associated with one or more target flavors to be mixed to recreate the taste of the parsley. Furthermore, among the pixels of the display unit when the designation target frame F1 is displayed, pixels that display a certain portion of the pasta P2 are associated with taste data that indicate the mixing ratio of substances that are associated with one or more target flavors to be mixed to recreate the taste of that portion of the pasta P2. For example, pixels that display only mint sprinkled on the pasta P2 are associated with taste data that indicate the mixing ratio of substances that are associated with one or more target flavors to be mixed to recreate the taste of the mint. Furthermore, among the pixels of the display unit when the designation target frame F1 is displayed, pixels that display portions other than the pasta P1 and pasta P2 (i.e., pixels that show portions other than food) are not associated with taste data, for example. In addition, among the multiple pixels of the display unit when the specified target frame F1 is displayed, pixels displaying parts other than pasta P1 and pasta P2 may be configured to be associated with taste data indicating that the mixing ratio of substances associated with all of the target flavors to be mixed is 0, or may be configured to be associated with taste data indicating the mixing ratio of substances associated with one or more target flavors to be mixed in order to reproduce the taste of the material of that part (e.g., pottery, metal, wood, etc.).

[0033] The designation target frame F1 shown in FIG. 3B is an example of a designation target frame after a designation target region is designated by the second operation. The region W1 shown in FIG. 3B is an example of a designation target region. The region W1 is a rectangular, planar region. The region W1 includes pasta P1. For example, when the region W1 is designated as a designation target region while the designation target frame F1 is displayed on the display unit, the information processing device 10 performs the process described below. The information processing device 10 identifies designation target pixels that overlap with the region W1 and are associated with taste data, and averages the mixture ratios indicated by the taste data associated with each of the identified designation target pixels for each taste. The information processing device 10 then generates new taste data indicating the mixture ratios after averaging for each taste. For convenience of explanation, the new taste data will be referred to as taste data D1 below. After generating the taste data D1, the information processing device 10 outputs the taste data D1. For example, the information processing device 10 displays the taste data D1 on the display unit. In this case, the information processing device 10 displays each of the mixing ratios indicated by the taste data D1 on the display unit for each taste. Furthermore, for example, the information processing device 10 outputs the taste data D1 to the taste reproduction device 20. In this case, the taste indicated by the taste data D1 is reproduced by the taste reproduction device 20. Details of the taste reproduction by the taste reproduction device 20 will be described later. The information processing device 10 performs the above-mentioned processing as processing when the region W1 is designated as the designated target region while the designated target frame F1 is displayed on the display unit. In this way, the information processing device 10 can cause the taste reproduction device 20 to reproduce the taste that a person would feel if they had put all of the pasta P1 in their mouth.

[0034] FIG. 4 illustrates an example of two regions designated as designation target regions in the designation target frame F1 shown in FIG. 3 . In the designation target frame F1 shown in FIG. 4 , in addition to region W1, region W2 is designated as a designation target region. In this case, the information processing device 10 identifies, among the designation target pixels overlapping region W1, the designation target pixels associated with corresponding taste data as first designation target pixels, and identifies, among the designation target pixels overlapping region W2, the designation target pixels associated with corresponding taste data as second designation target pixels. The information processing device 10 averages, for each taste, the mixture ratios indicated by the taste data associated with each of the identified first designation target pixels, and also averages, for each taste, the mixture ratios indicated by the taste data associated with each of the identified second designation target pixels. The information processing device 10 then generates new taste data D1 indicating the mixture ratios for the first designation target pixels after averaging for each taste, and generates new taste data D2 indicating the mixture ratios for the second designation target pixels after averaging for each taste. The information processing device 10 outputs the generated taste data D1 and taste data D2. For example, the information processing device 10 displays the taste data D1 and taste data D2 on a display unit. In this case, the information processing device 10 displays the mixing ratios indicated by the taste data D1 for each taste on the display unit, and also displays the mixing ratios indicated by the taste data D2 for each taste on the display unit. Furthermore, for example, the information processing device 10 outputs the taste data D1 and taste data D2 to the taste reproduction device 20. In this case, the taste indicated by the taste data D1 and the taste indicated by the taste data D2 are reproduced by the taste reproduction device 20. In this way, the information processing device 10 can cause the taste reproduction device 20 to reproduce the taste perceived by a person who has put all of the pasta P1 in their mouth and the taste perceived by a person who has put all of the pasta P2 in their mouth.

[0035] When regions W1 and W2 are designated as target regions in the target frame F1, the information processing device 10 may be configured to average the mixing ratios indicated by the taste data D1 and the mixing ratios indicated by the taste data D2 for each taste and output new taste data indicating the resulting averaging. In this case, the information processing device 10 can cause the taste reproduction device 20 to reproduce the taste perceived by a person who has both the entire pasta P1 and the entire pasta P2 in their mouth. This allows the information processing system 1 to reproduce the taste that a person would experience if they were to eat multiple foods at the same time. For example, the information processing system 1 can simulate the pairing of foods such as wine and cheese. This is useful because it encourages the development of new flavors.

[0036] The taste reproduction device 20 may be any device capable of reproducing the taste indicated by the taste data based on the taste data acquired from the information processing device 10. For example, the taste reproduction device 20 may be a device capable of dispensing, as liquid, substances corresponding to each of the tastes to be mixed (in this example, sucrose, salt, citric acid, potassium carbonate, and monosodium glutamate) into a single container while adjusting the amount so as to realize each of the mixing ratios indicated by the taste data. In this case, a user can enjoy the taste indicated by the taste data by drinking the liquid in which the substances are mixed in the single container. Candidates for devices that can be used as such a taste reproduction device 20 include, for example, TTTV3 (Transform The Taste and reproduce Varieties; see Miyashita Yoshiaki, Murakami Takato, Otomo Chihiro, and Fukaike Miku, "TTTV3 (Transform The Taste and reproduce Varieties): A seasoning mechanism that reproduces differences in origin and variety and taste expression using LLM," Entertainment Computing Symposium Proceedings, Vol. 2023, 2023), Norimaki Synthesizer (see International Publication No. 2021 / 215468), etc.

[0037] <Specific Example of the Configuration of the First Taste Presentation Image> Specific examples of the configuration of the first taste presentation image are described below. The first taste presentation image can be generated, for example, by editing video data in MP4 format. For ease of explanation, the following description will refer to MP4 video data with added taste data as FlavMP4. Figure 5 is a diagram showing an example of the file structure of FlavMP4. As shown in Figure 5, the file structure of FlavMP4 is based on the ISO Base Media File Format. Since the ISO Base Media File Format is well known, a detailed description of it will be omitted. Like MP4, FlavMP4 manages information on a box-by-box basis. Therefore, in FlavMP4, information is recorded in boxes configured in a nested structure. In the example shown in Figure 5, the taste data is stored in a moov box as a trak (Taste), one of the trak boxes.

[0038] In this way, the first taste presentation image can be configured in FlavMP4, which is an improved version of MP4 format video data. In this case, the information processing device 10 can display the first taste presentation image by executing an application program capable of playing FlavMP4. Note that the first taste presentation image may also be configured in other formats of video data.

[0039] <Hardware Configuration of Information Processing Apparatus> The hardware configuration of the information processing apparatus 10 will be described below with reference to Fig. 6. Fig. 6 is a diagram showing an example of the hardware configuration of the information processing apparatus 10.

[0040] The information processing device 10 includes, for example, a processor 11, a memory unit 12, an input receiving unit 13, a communication unit 14, a display unit 15, and a sound output unit 16. These components are connected to each other via a bus so that they can communicate with each other. The information processing device 10 also communicates with the taste reproduction device 20 via the communication unit 14.

[0041] The processor 11 is, for example, a CPU (Central Processing Unit). Note that instead of a CPU, the processor 11 may be another processor such as an FPGA (Field Programmable Gate Array). The processor 11 executes various programs stored in the storage unit 12.

[0042] The storage unit 12 includes, for example, a hard disk drive (HDD), a solid state drive (SSD), an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a random access memory (RAM), etc. Note that instead of being built into the information processing device 10, the storage unit 12 may be an external storage device connected via a digital input / output port such as a universal serial bus (USB). The storage unit 12 stores various types of information, images, programs, etc. to be processed by the information processing device 10.

[0043] The input receiving unit 13 is an input device, and is, for example, a touch panel that is configured integrally with the display unit 15. Note that instead of the touch panel, the input receiving unit 13 may be another input device that is configured separately from the display unit 15, such as a keyboard, a mouse, or a touchpad.

[0044] The communication unit 14 includes, for example, a digital input / output port such as a USB, an Ethernet (registered trademark) port, an antenna for wireless communication, and the like.

[0045] The display unit 15 is an example of a display unit included in the information processing device 10. The display unit 15 is a display device and includes, for example, a liquid crystal display.

[0046] The sound output unit 16 is an audio output device, and includes, for example, a speaker.

[0047] <Functional Configuration of Information Processing Apparatus> The functional configuration of the information processing apparatus 10 will be described below with reference to Fig. 7. Fig. 7 is a diagram showing an example of the functional configuration of the information processing apparatus 10.

[0048] The information processing device 10 includes a storage unit 12 , an input receiving unit 13 , a communication unit 14 , a display unit 15 , a sound output unit 16 , and a control unit 17 .

[0049] The control unit 17 controls the entire information processing device 10. The control unit 17 includes an output control unit 171 and a processing unit 172. These functional units included in the control unit 17 are realized, for example, by the processor 11 executing various programs stored in the storage unit 12. Some or all of the functional units may be hardware functional units such as large scale integration (LSI) and application specific integrated circuits (ASIC).

[0050] The output control unit 171 generates an image in response to the received operation. The output control unit 171 displays the generated image on the display unit 15. The output control unit 171 also displays frames of a moving image on the display unit 15 in a predetermined order. That is, the output control unit 171 displays frames of the first taste presentation image on the display unit 15 in a predetermined order. When displaying frames of the moving image on the display unit 15, the output control unit 171 also outputs sounds associated with each frame of the moving image to the sound output unit 16. Therefore, the output control unit 171 plays the moving image.

[0051] In response to the received operation, the processing unit 172 executes each process performed by the information processing device 10. For example, when the output control unit 171 is displaying a first taste presentation image on the display unit 15, the processing unit 172 outputs taste data by receiving a first operation and a second operation.

[0052] <Process for outputting taste data by accepting a first operation and a second operation> Hereinafter, a process in which the information processing device 10 outputs taste data by accepting a first operation and a second operation will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the flow of a process in which the information processing device 10 outputs taste data by accepting a first operation and a second operation. As an example, the following describes a case in which the information processing device 10 starts playing a first taste presentation image, which is a moving image, at a timing before the process of step S110 shown in Fig. 8 is performed.

[0053] After the reproduction of the first taste presentation image starts, the processing unit 172 waits until the first operation is received (step S110).

[0054] When the processing unit 172 determines that the first operation has been received (step S110—YES), the output control unit 171 pauses the playback of the first taste presentation image (step S120). As a result, the playback of the first taste presentation image in the information processing device 10 is paused while displaying the frame of the first taste presentation image, which is a moving image, that was displayed on the display unit 15 at the time the processing of step S120 was performed. This allows the information processing device 10 to designate that frame as the designation target frame.

[0055] Next, the processing unit 172 waits until a second operation is received (step S130).

[0056] If the processing unit 172 determines that a second operation has been received (step S130—YES), it identifies one or more designation target areas according to the received second operation. Then, the processing unit 172 identifies one or more designation target pixels for each of the identified one or more designation target areas (step S140). In FIG. 8 , the processing of step S140 is indicated by “designation target pixel identification.” Note that the methods for identifying one or more designation target areas and one or more designation target pixels have already been described. Therefore, detailed descriptions of these methods will be omitted here.

[0057] Next, for each of the one or more designated target regions identified in step S140, the processing unit 172 averages, for each flavor, the mixture ratios indicated by the flavor data associated with each of the one or more designated target pixels identified in the designated target region (step S150). In Fig. 8, the processing of step S150 is indicated by "averaging flavor data."

[0058] Next, the processing unit 172 generates new taste data indicating the mixture ratio averaged in step S150 for each of the one or more designated target regions identified in step S140 (step S160). In Fig. 8, the processing of step S160 is indicated by "generation of taste data after averaging."

[0059] Next, the processing unit 172 outputs the new taste data generated in step S160 for each of the one or more designated target regions identified in step S140 (step S170). After that, the processing unit 172 ends the processing of the flowchart shown in FIG.

[0060] As described above, the information processing device 10 displays on the display unit 15 a first taste presentation image in which taste data indicating one or more tastes is associated with at least some of the pixels, and outputs taste data associated with a designated target pixel designated by an operation received via the first taste presentation image displayed on the display unit 15. This allows the information processing device 10 to improve the resolution of tastes that can be reproduced in the human oral cavity.

[0061] <First Variation of the Embodiment> The following describes a first variation of the embodiment. In the first variation of the embodiment, the information processing device 10 stores still images generated by capturing each of one or more designation target frames, and displays an image displaying each of the stored still images on the display unit 15. Hereinafter, for convenience of explanation, this image will be referred to as a taste palette image. That is, the taste palette image displays one or more still images, each pixel of which has taste data representing various tastes, similar to paints of various colors placed on a palette. A user can select a desired still image from one or more still images on the taste palette image displayed on the display unit 15 and cause the taste reproduction device 20 to reproduce the taste represented by the taste data associated with the selected still image. Hereinafter, for convenience of explanation, the designation target frame captured as a still image from the first taste presentation image will be referred to as a second taste presentation image. Note that the second taste presentation image may be a still image in which taste data is associated with at least some of the pixels using image editing software or the like, instead of the designation target frame captured as a still image from the first taste presentation image. In this case, the user does not need to use the first taste presentation image to generate the second taste presentation image.

[0062] 9 is a diagram showing an example of a process flow in which the information processing device 10 captures a designated target frame as a second taste presentation image from a first taste presentation image, which is a moving image. Hereinafter, as an example, a case will be described in which the process of step S120 shown in FIG. 8 is executed and the designated target frame is displayed on the display unit 15 at a timing before the process of the flowchart shown in FIG. 9 is executed.

[0063] After the designated frame is displayed on the display unit 15, the processing unit 172 waits until a predetermined third operation is performed (step S210). Here, the third operation may be any operation that allows capturing of the frame displayed on the display unit 15.

[0064] When it is determined that the third operation has been performed (step S210-YES), the processing unit 172 captures the designated target frame displayed on the display unit 15 as a still image (step S220).

[0065] Next, the processing unit 172 stores the still image captured in step S220 as a second taste presentation image in the storage unit 12 (step S230). After that, the processing unit 172 ends the processing of the flowchart shown in FIG.

[0066] As described above, the information processing device 10 stores the designated target frame from the first taste presentation image, which is a moving image, as the second taste presentation image, which is a still image. This allows the information processing device 10 to easily store the still image to be displayed on the taste palette image.

[0067] The information processing device 10 may be configured to capture a part of the designated frame as a single still image by the third operation.

[0068] 10 is a diagram showing an example of a processing flow performed by the information processing device 10 to display a taste palette image and a second taste presentation image on the taste palette. Below, as an example, a case will be described in which one or more second taste presentation images are stored in the storage unit 12 at a timing before the processing of step S310 shown in FIG. 10 is performed. Also, below, as an example, a case will be described in which the information processing device 10 receives a taste palette image display operation to display a taste palette image on the display unit 15 at that timing.

[0069] After receiving the taste palette image display operation, the output control unit 171 generates a taste palette image (step S310). Here, the method for generating the taste palette image in step S310 may be a known method or a method to be developed in the future.

[0070] Next, the output control unit 171 displays the taste palette image generated in step S310 on the display unit 15 (step S320). Here, the taste palette image is an image that does not include the second taste presentation image.

[0071] Next, the processing unit 172 waits until it receives an operation via the taste palette image displayed on the display unit 15 in step S320 (step S330).

[0072] When the processing unit 172 determines in step S320 that an operation has been received via the taste palette image displayed on the display unit 15 (step S330—YES), the processing unit 172 determines whether the received operation is a predetermined fourth operation (step S340). Here, the fourth operation may be any operation that allows a user to select a desired second taste presentation image from one or more second taste presentation images pre-stored in the storage unit 12.

[0073] If the processing unit 172 determines that the received operation is the fourth operation (step S340—YES), the output control unit 171 displays the second taste presentation image selected by the received fourth operation on the taste palette image (step S370). The method for displaying the second taste presentation image on the taste palette image in step S370 may be a known method or a method to be developed in the future. The display format of the second taste presentation image on the taste palette image may be a reduced version of the second taste presentation image, like a thumbnail, or any other display format. After displaying the second taste presentation image on the taste palette image, the processing unit 172 transitions to step S330 and waits again until an operation is received via the taste palette image.

[0074] On the other hand, if the processing unit 172 determines that the received operation is not the fourth operation (step S340—NO), it determines whether the received operation is a display deletion operation that causes the information processing device 10 to delete the display of the taste palette image from the display unit 15 (step S350). In Fig. 10, the processing of step S350 is indicated by "Delete display of taste palette image?"

[0075] If the processing unit 172 determines that the received operation is a display deletion operation (step S350-YES), the output control unit 171 deletes the display of the taste palette image from the display unit 15 (step S360) and terminates the processing of the flowchart shown in Figure 10.

[0076] On the other hand, if the processing unit 172 determines that the received operation is not a display deletion operation (step S350—NO), it performs processing corresponding to the received operation (step S380). Here, in step S380, the processing unit 172 performs, for example, the processing of steps S130 to S170 shown in FIG. 8 as the processing corresponding to the operation. As a result, the information processing device 10 can, for example, specify a target pixel by specifying a target area for the second taste presentation image displayed on the taste palette image. As a result, the information processing device 10 can output taste data for the second taste presentation image. That is, even when the information processing device 10 uses a still second taste presentation image, the resolution of the tastes that can be reproduced in the human oral cavity can be improved. After performing the processing of step S380, the processing unit 172 transitions to step S330 and waits again until it receives an operation via the taste palette image.

[0077] <Second Modification of the Embodiment> A second modification of the embodiment will be described below. In the second modification of the embodiment, when the regions W1 and W2 are designated as designation target regions in the designation target frame F1 shown in FIG. 4 , the information processing device 10 may be configured to subtract, for each flavor, the mixing ratio indicated by one of the flavor data D1 and the flavor data D2 from the mixing ratio indicated by the other of the flavor data D1 and the flavor data D2. In this case, the information processing device 10 can, for example, subtract the flavor of pasta from the flavor of pasta with squid ink sauce, generate flavor data indicating the flavor of the squid ink sauce alone, and output the generated flavor data.

[0078] <Variation 3 of the embodiment> Variation 3 of the embodiment will be described below. In Variation 3 of the embodiment, the taste reproduction device 20 may be configured to be able to print a designated target frame. In this case, for example, for each pixel of the designated target frame, the taste reproduction device 20 ejects a liquid of the taste indicated by the taste data associated with the pixel at a position corresponding to the pixel on a print medium on which the designated target frame is printed. In this way, the taste reproduction device 20 can output a print medium that allows the user to taste the taste of the food displayed in the designated target frame. This may also be referred to as a taste print.

[0079] The above-described items may be combined in any manner.

[0080] <Notes> [1] An information processing device that displays, on a display unit, an image in which taste data indicating one or more tastes is associated with at least some pixels, and outputs the taste data associated with the pixels specified by an operation based on an operation received via the image displayed on the display unit. [2] The information processing device described in [1], wherein the taste data is information indicating a mixture ratio of substances that reproduce each of the one or more tastes in the human oral cavity. [3] The information processing device described in [1] or [2], wherein the image is a moving image. [4] The information processing device described in [3], wherein the operation includes a first operation for designating a first frame among frames of the moving image, and a second operation for designating one or more pixels among a plurality of pixels included in the display unit as target pixels. [5] The information processing device described in [4], wherein the second operation is an operation for designating one or more regions on the display unit, and the target pixels designated by the second operation are the one or more pixels that overlap with the one or more regions designated by the second operation among the plurality of pixels when the first frame designated by the first operation is displayed on the display unit. [6] The information processing device according to [5], wherein the taste data is information indicating a mixing ratio of substances that reproduce each of the one or more tastes in the human oral cavity, and when the first frame specified by the first operation is displayed on the display unit, the mixing ratios indicated by each of the taste data associated with each of the one or more pixels that overlap with the one or more regions specified by the second operation are averaged for each taste, and the taste data indicating the mixing ratio after averaging for each taste is output to the other device. [7] The information processing device according to [1] or [2], wherein the image is a still image. [8] A program that causes a computer to display on a display unit an image indicated by image data indicating an image in which taste data indicating one or more tastes is associated with at least some pixels, and to output the taste data associated with the pixels specified by the operation based on an operation received via the image displayed on the display unit.

[0081] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and may be changed, replaced, deleted, etc., without departing from the gist of the present invention.

[0082] Furthermore, a program for implementing the functions of any of the components of the above-described devices (e.g., the information processing device 10, the taste reproduction device 20, etc.) may be recorded on a computer-readable recording medium and then loaded and executed by a computer system. Note that the term "computer system" as used herein includes hardware such as an operating system (OS) and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and compact disks (CDs)-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0083] The above-mentioned program may also be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The above-mentioned program may also be intended to realize some of the above-mentioned functions. Furthermore, the above-mentioned program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0084] 1... Information processing system, 10... Information processing device, 11... Processor, 12... Memory unit, 13... Input receiving unit, 14... Communication unit, 15... Display unit, 16... Sound output unit, 17... Control unit, 20... Taste reproduction device, 171... Output control unit, 172... Processing unit

Claims

1. An information processing device that displays an image on a display unit in which taste data indicating one or more tastes is associated with at least some of the pixels, and outputs the taste data associated with the pixels specified by an operation received via the image displayed on the display unit.

2. The information processing device according to claim 1, wherein the taste data is information indicating a mixture ratio of substances that reproduces each of the one or more tastes in the human oral cavity.

3. The information processing device according to claim 1, wherein the image is a moving image.

4. The information processing device according to claim 3, wherein the image is a moving image represented by moving image data in MP4 format to which the taste data is added.

5. The information processing device of claim 3, wherein the operations include a first operation for designating a first frame among the frames of the moving image, and a second operation for designating one or more pixels among the plurality of pixels of the display unit as pixels to be designated.

6. An information processing device as described in claim 5, wherein the second operation is an operation of specifying one or more areas on the display unit, and the specified target pixels specified by the second operation are one or more pixels that overlap with the one or more areas specified by the second operation among the multiple pixels when the first frame specified by the first operation is displayed on the display unit.

7. The information processing device described in claim 6, wherein the taste data is information indicating a mixture ratio of substances that reproduce each of the one or more tastes in the human oral cavity, and when the first frame specified by the first operation is displayed on the display unit, the mixture ratio indicated by each of the taste data associated with each of the one or more pixels that overlap with the one or more areas specified by the second operation is averaged for each taste, and the taste data indicating the mixture ratio after averaging for each taste is output to the other device.

8. The information processing device according to claim 1, wherein the image is a still image.

9. A program that causes a computer to display on a display unit an image represented by image data showing an image in which taste data indicating one or more tastes is associated with at least some of the pixels, and based on an operation received via the image displayed on the display unit, output the taste data associated with the pixel specified by the operation.

Images