Information processing device, printing system, information processing method, and program

Abstract

To provide an information processing device for determining an appearance of an edible article matching a food or drink.SOLUTION: An information processing device is provided, comprising an image acquisition unit for acquiring an image of a food or drink, an appearance determination unit configured to determine an appearance of an edible article to be added to the food or drink on the basis of a color of the food or drink and / or nutrients contained in the food or drink, and an output unit for outputting appearance information indicative of the determined appearance of the edible article.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a printing system, an information processing method, and a program.

Background Art

[0002] Supplements are widely used to assist in the intake of nutrients. In order to take supplements appropriately, it is necessary to grasp the nutrients lacking in the diet. Patent Document 1 describes a system that proposes supplements based on an image of the diet consumed by a user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] A food printer that produces edible food by printing is known. Using the food printer, it is desired to create edible food that matches the appearance of the food and drink consumed by the user or supplements the nutrients lacking in the food and drink.

[0005] The present invention has been made to solve the above problems, and an object thereof is to provide an information processing apparatus, a printing system, an information processing method, and a program for determining the appearance of edible food that matches food and drink.

Means for Solving the Problems

[0006] An information processing apparatus according to an embodiment of the present invention includes an image acquisition unit that acquires an image of food and drink, an appearance determination unit that determines the appearance of edible food added to the food and drink based on the color of the food and drink and / or the nutrients contained in the food and drink, and an output unit that outputs appearance information indicating the determined appearance of the edible food.

[0007] The appearance determination unit preferably determines at least one of the color, shape, and pattern of the edible food as the appearance of the edible food.

[0008] The appearance determination unit preferably calculates a feature amount related to the color of the food and drink from an image of the food and drink, and determines the color of the edible food based on the feature amount.

[0009] The output unit preferably outputs appearance information to a printing device that produces the edible food by printing.

[0010] The information processing apparatus further includes a storage unit that associates and stores a plurality of raw materials used in the printing device, the color of each raw material, and the types of nutrients contained in each raw material. The appearance determination unit preferably identifies the types of nutrients lacking in the food and drink based on an image of the food and drink, and determines the appearance of the edible food based on the colors of the raw materials containing the identified types of nutrients.

[0011] The appearance determination unit preferably further identifies the amount of nutrients lacking in the food and drink based on an image of the food and drink, and determines the appearance of the edible food based on the amount of the raw material corresponding to the identified amount of nutrients.

[0012] The information processing apparatus further includes a storage unit that stores the user's preference regarding the appearance of the edible food. The appearance determination unit preferably determines the appearance of the edible food further based on the user's preference.

[0013] The appearance determination unit determines a plurality of candidates for the appearance of the edible food, determines the candidate selected by the user from among the plurality of candidates as the appearance of the edible food, and the information processing apparatus preferably further includes a setting unit that sets the user's preference based on the history of the user's selection.

[0014] The information processing apparatus further includes an arrangement determination unit that determines the position of the edible food relative to the food and drink and / or the size of the edible food. The output unit preferably further outputs arrangement information indicating the determined position and / or size.

[0015] An information processing apparatus according to an embodiment of the present invention includes a food and drink information acquisition unit that acquires information on food and drink that a user prepares, ingests, or has prepared or ingested, an appearance determination unit that identifies nutrients contained in the food and drink based on the food and drink information and determines the appearance of an edible item added to the food and drink based on the nutrients contained in the food and drink, and an output unit that outputs appearance information indicating the determined appearance of the edible item.

[0016] The information processing apparatus further includes a skin information acquisition unit that acquires skin information indicating the skin condition of a user who ingests food and drink, and it is preferable that the appearance determination unit identifies nutrients recommended for the user based on the nutrients contained in the food and drink and the skin information and determines the appearance of an edible item added to the food and drink based on the nutrients recommended for the user.

[0017] An information processing apparatus according to an embodiment of the present invention includes a skin information acquisition unit that acquires skin information indicating the skin condition of a user who ingests food and drink, an appearance determination unit that identifies nutrients recommended for the user based on the skin information and determines the appearance of an edible item added to the food and drink based on the nutrients recommended for the user, and an output unit that outputs appearance information indicating the determined appearance of the edible item.

[0018] An information processing apparatus according to an embodiment of the present invention includes a food and drink information acquisition unit that acquires information on food and drink that a user prepares, ingests, or has prepared or ingested, an image acquisition unit that acquires an image of the food and drink generated based on the food and drink information, an appearance determination unit that determines the appearance of an edible item added to the food and drink based on the color of the food and drink in the image, and an output unit that outputs appearance information indicating the determined appearance of the edible item.

[0019] A printing system according to an embodiment of the present invention includes the information processing apparatus according to the embodiment and a printing apparatus that produces an edible item by printing based on the appearance information output from the information processing apparatus.

[0020] The information processing method according to an embodiment of the present invention is an information processing method executed by an information processing apparatus, including acquiring an image of food and drink, determining an appearance of an edible item added to the food and drink based on the color of the food and drink and / or nutrients contained in the food and drink, and outputting appearance information indicating the determined appearance of the edible item.

[0021] The program according to an embodiment of the present invention causes a computer to function as the information processing apparatus according to the embodiment.

Advantages of the Invention

[0022] The information processing apparatus, printing system, information processing method, and program according to the present invention determine an appearance of an edible item that matches the food and drink.

Brief Description of the Drawings

[0023]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0024] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the technical scope of the present invention is not limited to those embodiments, and extends to the invention described in the claims and equivalents thereof.

[0025] [First Embodiment] FIG. 1 is a diagram showing an example of the schematic configuration of a printing system 1 according to the first embodiment of the present invention. The printing system 1 includes a user terminal 2, a food printer 3, and a server 4. The user terminal 2, the food printer 3, and the server 4 communicate with each other via a network N which is the Internet or an intranet. The food printer 3 is an example of a printing device, and produces an edible product by printing based on the appearance information output from the server 4. In the present embodiment, it is assumed that the food printer 3 produces an edible product by printing an image on an edible sheet. The server 4 is an example of an information processing device.

[0026] The user terminal 2 is an information processing terminal such as a PC (Personal Computer), a smartphone, a mobile phone, or a tablet terminal possessed by the user.

[0027] FIG. 2 is an exemplary hardware configuration diagram of the food printer 3. The food printer 3 includes a memory 31, a communication unit 32, a printing unit 33, a processor 34, and a cartridge 35.

[0028] The memory 31 is an example of a storage unit and stores data and programs. The memory 31 is, for example, a semiconductor memory or a magnetic disk. The memory 31 stores an operating system program, a driver program, an application program, and various types of data used for processing by the processor 34.

[0029] The communication unit 32 enables the hood printer 3 to communicate with other devices. The communication unit 32 is a communication interface circuit such as a wireless LAN (Local Area Network) or a wired LAN. The communication unit 32 transmits data supplied from the processor 34 to other devices and supplies data transmitted from other devices to the processor 34.

[0030] The printing unit 33 produces an edible object by printing. The printing unit 33 has a print head and a drive mechanism. The print head ejects droplets of an edible raw material 351 housed in a cartridge 35. The drive mechanism includes, for example, a guide rail, an actuator, and a control circuit. The guide rail and the actuator move the print head. The control circuit controls the print head to eject the raw material 351 onto the print head to print an image on an edible sheet. The printing unit 33 prints an image based on the image data supplied from the processor 34.

[0031] The processor 34 is an example of a processing unit and comprehensively controls the operation of the hood printer 3. The processor 34 is a circuit such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). The processor 34 executes various types of processing based on the programs stored in the memory 31.

[0032] In the present embodiment, the cartridge 35 includes a plurality of containers that each contain a plurality of raw materials 351. For example, the plurality of raw materials 351 are edible inks. The plurality of raw materials 351 are different from each other in terms of at least one of the color and the nutrients contained in the raw material. A single raw material 351 may contain a single type of nutrient or may contain a plurality of types of nutrients. The raw material 351 accommodated in the cartridge 35 is ejected from the print head of the printing unit 33.

[0033] In the present embodiment, an example of the hood printer 3 has been described by taking as an example the case where the hood printer 3 includes a single cartridge 35 and a plurality of types of raw materials 351 are accommodated in the single cartridge 35. However, the hood printer 3 is not limited to the present embodiment. In other embodiments, the hood printer 3 may include a plurality of cartridges 35. In other embodiments, a single cartridge 35 may accommodate a single type of raw material 351.

[0034] FIG. 3 is an exemplary hardware configuration diagram of the server 4. The server 4 has a memory 41, a communication unit 42, and a processor 43.

[0035] The memory 41 is an example of a storage unit and stores data and programs. The memory 41 is, for example, a semiconductor memory or a magnetic disk. The memory 41 stores an operating system program, a driver program, an application program, and various types of data used for processing by the processor 43.

[0036] The communication unit 42 enables the server 4 to communicate with other devices. The communication unit 42 is, for example, a communication interface circuit such as a wireless LAN or a wired LAN. The communication unit 42 transmits data supplied from the processor 43 to other devices and supplies data transmitted from other devices to the processor 43.

[0037] The processor 43 is an example of a processing unit and comprehensively controls the operation of the server 4. The processor 43 is a circuit (such as a CPU, DSP, LSI, ASIC, or FPGA). The processor 43 executes various processes based on a program stored in the memory 41.

[0038] The processor 43 has an image acquisition unit 431, a determination unit 432, a setting unit 433, and an output unit 434 as functional blocks. Each of these units is a functional module realized by the processor 43 that executes a program stored in the memory 41. Each of these units may be implemented in the server 4 as a dedicated processing circuit. The determination unit 432 is an example of an appearance determination unit and an arrangement determination unit.

[0039] FIG. 4 is a diagram showing an example of the data structure of the raw material table T1 stored in the memory 41. The raw material table T1 stores information related to a plurality of raw materials 351 accommodated in the cartridge 35, associating information such as the name, color of the raw material 351, and nutrients contained in the raw material 351 with each other. In the example shown in FIG. 4, as information related to the nutrients contained in the raw material 351, information indicating the type and amount of nutrients contained in a unit amount of the raw material 351 is stored.

[0040] The raw material table T1 is set in advance according to the raw material 351 accommodated in the cartridge 35.

[0041] FIG. 5 is a diagram showing an example of the data structure of an image table T2 stored in the memory 41. The image table T2 stores, as information regarding a plurality of images printed on an edible sheet by the hood printer 3, identification information of the image data, information regarding the color of the image, information regarding the shape of the image, information regarding the target age of a person (sometimes referred to as a user) who eats or drinks the food or drink to which the printed matter with the image is added, the type of raw material used for printing the image, the type and amount of the raw material, etc., in association with each other. The plurality of images are different from each other. For example, the plurality of images may be images in which at least one of color, shape, and pattern is different from each other. In the example shown in FIG. 5, as the color of the image, the color most contained in the image, the second most contained color, and the ratio occupied by each color in the image are stored.

[0042] The image table T2 is set in advance.

[0043] FIG. 6 is a diagram showing an example of the data structure of a user table T3 stored in the memory 41. The user table T3 stores, as information regarding the user, the identification information, attribute information, and preferences of the user in association with each other. The attribute information of the user may include the user's age, gender, height, weight, etc. The preferences of the user may include preferences regarding the user's color, color scheme, and eating habits, etc. The preference regarding the color scheme is a preference regarding the relationship between the color of the food or drink and the color of the edible item. For example, as the preference regarding the color scheme, whether the user prefers an edible item of a color in a homologous color relationship with the color of the food or drink, or an edible item of a color in a complementary color relationship with the color of the food or drink is stored. The preference regarding the eating habits stores whether the user is on a diet and wants to reduce the amount of food intake.

[0044] The user table T3 is set in advance based on the attribute information and preferences provided by each user. The information regarding the preferences of the users in the user table T3 is updated in the printing method described later.

[0045] FIG. 7 is a sequence diagram showing an example of the flow of a printing method executed by the printing system 1. The printing method is realized by the processors of the respective devices constituting the printing system 1 cooperating with the hardware. For example, the printing method is executed in response to the user operating the user terminal 2.

[0046] First, the image acquisition unit 431 of the server 4 acquires an image of food and drink from the user terminal 2 (step S11). For example, the image acquisition unit 431 may acquire an image of food and drink from the user terminal 2 by receiving it via the communication unit 42. In this case, the image of food and drink may be generated by the user terminal 2 photographing the food and drink, or may be acquired by other methods. The image of food and drink may be an image representing the food and drink that the user intends to consume, that is, the food and drink prepared for consumption. In this case, the user may operate the user terminal 2 to photograph the prepared food and drink to generate an image of the food and drink. The image of food and drink may also be an image representing the food and drink that the user intends to prepare or the food and drink that the user has consumed. In this case, the user may operate the user terminal 2 to acquire an image of the food and drink from a website or the like that introduces the food and drink. In the following description of the printing method, the food and drink represented in the image is assumed to be a bento box including a staple food, a main dish, and a side dish.

[0047] Next, the determination unit 432 determines the appearance of the edible sheet to be added to the food and drink based on the color of the food and drink (step S12). The edible sheet is an edible sheet on which an image is printed by the food printer 3. For example, the edible sheet on which an image is printed is added to the food and drink by being placed on the food and drink. The determination unit 432 specifies the color of the food and drink based on the image of the food and drink, and determines the appearance of the edible sheet based on the color of the food and drink.

[0048] The determination unit 432 may specify a food and drink area corresponding to the food and drink in the image of the food and drink, and specify the color of the food and drink based on the colors of the pixels included in the specified area.

[0049] For example, the determination unit 432 identifies the food and drink area by applying template matching to the image of the food and drink. Not limited to such an example, the determination unit 432 may identify the food and drink area using a learned model. When a learned model is used, the determination unit 432 identifies the food and drink area by inputting the image of the food and drink into the learned model that has been trained to output the food and drink area corresponding to the food and drink when an image is input. For example, the learned model is an object detection model such as R-CNN (Regional Convolutional Neural Network).

[0050] The determination unit 432 identifies the color of the food and drink by calculating a feature quantity related to the color of the food and drink based on the pixels included in the food and drink area. The feature quantity related to the color of the food and drink may be the ratio of the pixels representing a predetermined color among the pixels included in the food and drink area. For example, the feature quantity is the ratio of the pixels representing brown and the ratio of the pixels representing green among the pixels included in the food and drink area. The feature quantity may be a representative value such as the average value or the most frequent value of the pixel values of the pixels included in the food and drink area.

[0051] The determination unit 432 may identify the color of the food and drink without calculating the feature quantity related to the color of the food and drink. For example, the determination unit 432 may identify as the color of the food and drink the color that is associated in advance with the type of the food and drink output from the template used for template matching or the learned model.

[0052] When the image includes a plurality of foods and drinks, the determination unit 432 may identify a plurality of food and drink areas corresponding to the plurality of foods and drinks respectively, and identify the color of the food and drink for each food and drink area. For example, the determination unit 432 identifies three food and drink areas corresponding to the staple food, the main dish, and the side dish included in the bento, and calculates the feature quantity related to the color individually for each of the three food and drink areas. Not limited to such an example, the determination unit 432 may identify only one food and drink area corresponding to the bento.

[0053] The determination unit 432 determines the appearance of the edible food based on the image of the food or drink. For example, the determination unit 432 determines the appearance of the edible food by selecting, as the image to be printed on the edible sheet, any one of the plurality of images stored in the image table T2 based on the color of the food or drink. The appearance of the edible food is at least one of the color, shape, pattern, and size of the edible food. For example, the determination unit 432 determines at least one of the color, shape, pattern, and size of the edible food by selecting, as the image to be printed on the edible sheet, any one of the plurality of images stored in the image table T2, in which at least one of the color, shape, pattern, and size is different from each other, based on the color of the food or drink.

[0054] The determination unit 432 may determine the color of the edible food based on the feature amount related to the color of the food or drink. When the feature amount is the ratio of the pixels representing a predetermined color, the determination unit 432 determines the color of the edible food by comparing the ratio indicated by the feature amount with a preset reference ratio. For example, the determination unit 432 may determine the color of the edible food to be green when the ratio of the pixels representing green is lower than the reference ratio. The determination unit 432 selects, as the image to be printed on the edible sheet, an image including the determined color of the edible food from among the plurality of images stored in the image table T2.

[0055] The determination unit 432 may further determine the color of the edible food based on the user's preference regarding the appearance of the edible food. For example, the determination unit 432 selects, as the image to be printed on the edible sheet, an image including the first color determined by the above method based on the feature amount related to the color of the food or drink and the second color determined based on the user's preference. The user's preference may be a preference regarding the color combination relationship of the user. In this case, the determination unit 432 refers to the user table T3 to obtain the preference regarding the user's color combination. The determination unit 432 determines, as the second color, a color that is in a relationship of the same color system or complementary color with the color of the food or drink from among the plurality of images stored in the image table T2 based on the preference regarding the user's color combination. The user's preference may be a preference regarding the user's eating tendency. For example, when the user is on a diet, the determination unit 432 determines blue, which reduces appetite, as the second color.

[0056] The determination unit 432 may determine at least one of the pattern, shape, and size of the edible object based on the user's physical information. For example, the determination unit 432 selects, as the image to be printed on the edible sheet, an image associated with a target age suitable for the user's age from among a plurality of images including the color of the determined edible object. The determination unit 432 may also select, as the image to be printed on the edible sheet, an image printed in a size corresponding to the user's height or weight from among a plurality of images including the color of the determined edible object. Not limited to the above-described examples, the determination unit 432 calculates an evaluation value for each of the plurality of images based on whether each of the plurality of images includes the color of the edible object, whether it is associated with a target age suitable for the user's age, whether it is printed in a size corresponding to the user's height or weight, etc., and may select the image having the maximum evaluation value as the image to be printed on the edible sheet.

[0057] The determination unit 432 may also determine at least one of the pattern, shape, and size of the edible object based on the type of food and drink. For example, the determination unit 432 selects, as the image to be printed on the edible sheet, an image having a predetermined relationship with the type of food and drink represented in the image from among a plurality of images including the color of the determined edible object. When the plurality of images stored in the image table T2 represent food and drink, the predetermined relationship is that the food and drink represented in the image acquired in step S11 does not overlap with the food and drink represented in the images stored in the image table T2. As a result, since food and drink not included in the image of the food and drink is printed on the edible sheet, an impression is given that the variety of ingredients is rich.

[0058] The determination unit 432 may determine a plurality of candidates for the appearance of the edible item, and determine, as the appearance of the edible item, the one selected by the user from among the plurality of candidates. In this case, the determination unit 432 determines a plurality of candidates for the appearance of the edible item based on the color of the food and drink. For example, the determination unit 432 determines a plurality of candidates by selecting a plurality of candidate images from among the plurality of images stored in the image table T2 based on the color of the food and drink. The determination unit 432 transmits the plurality of candidate images to the user terminal 2, and receives, from the user terminal 2, the candidate image selected by the user from among the plurality of candidate images. The determination unit 432 determines the appearance of the edible item by selecting the candidate image selected by the user as the image to be printed on the edible sheet.

[0059] Next, the setting unit 433 sets the user's preference regarding the appearance of the edible item (step S13).

[0060] The setting unit 433 may set the user's preference based on the user's evaluation of the determined appearance of the edible item. For example, the setting unit 433 transmits information regarding the determined appearance of the edible item to the user terminal 2, and receives, from the user terminal 2, information indicating the user's evaluation of the appearance of the edible item displayed on the user terminal 2, thereby receiving the user's evaluation. The setting unit 433 sets the user's preference by updating the user table T3 based on the information indicating the user's evaluation. For example, when an image that is in the relationship of complementary colors with the color of the food and drink and includes green is selected as the image to be printed on the edible sheet and the user evaluates it as favorable, the setting unit 433 stores green as the color preferred by the user and stores complementary colors as the color combination relationship preferred by the user in the user table T3.

[0061] When a plurality of candidates for the appearance of the edible object are determined in step S12, the setting unit 433 may set the user's preference based on the history of the user's selection of any one of the plurality of candidates. For example, when a candidate image that is in the complementary color relationship with the color of the food or drink and includes green is selected by the user, the setting unit 433 stores green as the color preferred by the user and stores complementary colors as the color matching relationship preferred by the user in the user table T3. The setting unit 433 may set the user's preference based on a plurality of candidate images selected by the user within a predetermined period. For example, the setting unit 433 may store, in the preference table T3, as the color preferred by the user, the color common to the plurality of candidate images selected by the user within a predetermined period.

[0062] Next, the output unit 434 outputs appearance information indicating the appearance of the edible object (step S14). For example, the output unit 434 outputs by transmitting the appearance information to the hood printer 3 via the communication unit 42. The appearance information may be data of an image printed on the edible object.

[0063] Next, the processor 34 of the hood printer 3 produces the edible object by printing based on the appearance information (step S15). For example, the processor 34 controls the printing unit 33 based on the appearance information, and ejects droplets of the raw material 351 of the color indicated in the image data to a predetermined position on the edible sheet to print the image indicated by the appearance information. Thus, the printing process ends.

[0064] As described above, in the present embodiment, the server 4 includes a determination unit 432 that determines the appearance of the edible object added to the food or drink based on the color of the food or drink, and an output unit 434 that outputs appearance information indicating the appearance of the edible object. Therefore, the server 4 can determine the appearance of the edible object that matches the color of the food or drink.

[0065] Further, the determination unit 432 determines the color of the edible object based on the feature amount related to the color of the food or drink. Therefore, the server 4 can improve the appearance of the food or drink and the edible object by giving a sense of unity to the colors of the food or drink and the edible object, or by making the food or drink and the edible object colorful.

[0066] Further, the output unit 434 outputs appearance information to the food printer 3 that produces edible food by printing. Therefore, the server 4 can produce edible food having an appropriate appearance.

[0067] The server 4 further includes a memory 41 that stores the user's preference regarding the appearance of the edible food, and the determination unit 432 determines the appearance of the edible food based on the user's preference. Therefore, the server 4 can determine the appearance of the edible food that conforms to the user's preference.

[0068] The determination unit 432 determines a plurality of candidates for the appearance of the edible food, and determines, as the appearance of the edible food, the candidate selected by the user from among the plurality of candidates. The server 4 has a setting unit 433 that sets the user's preference based on the history of the user's selection. Therefore, the appearance of the edible food that conforms to the user's preference can be determined.

[0069] In the above-described embodiment, the food printer 3 produces edible food by printing an image on an edible sheet. However, the present invention is not limited to such an example, and the food printer 3 may be a three-dimensional food printer that produces edible food having a desired three-dimensional shape by molding raw materials. In this case, the memory 41 stores information regarding the shape of the edible food produced by printing instead of the image table T2. Note that, in the present specification, the production of a three-dimensional object by a three-dimensional food printer may be referred to as printing.

[0070] In the above-described embodiment, the printing system 1 includes the user terminal 2, the food printer 3, and the server 4. However, the present invention is not limited to such an example, and the functions of some of the devices among the user terminal 2, the food printer 3, and the server 4 may be realized by other devices. For example, at least a part of the functions of the server 4 may be realized by the user terminal 2 or the food printer 3. At least a part of the functions of the user terminal 2 and the server 4 may be realized by the food printer 3.

[0071] In the above-described embodiment, the printing system 1 was described in detail by taking as an example the case where an edible sheet printed with edible ink is produced and the edible sheet is applied to food or drink. However, the printing system 1 is not limited to this embodiment. In other embodiments, the printing system 1 may produce an object for adjusting the composition of nutrients contained in the food or drink by being added to the food or drink. For example, the printing system 1 may produce an edible or inedible object that absorbs water or a specific type of nutrient. By adding the above object to the food or drink, the composition of nutrients contained in the food or drink can be appropriately adjusted.

[0072] [Modification Example 1 of the First Embodiment] In the above-described embodiment, the determination unit 432 determines the appearance of the edible object based on the color of the food or drink. Not limited to such an example, the determination unit 432 may determine the appearance of the edible object based on the nutrients contained in the food or drink. For example, the determination unit 432 may identify the types of nutrients lacking in the food or drink based on an image of the food or drink, and determine the appearance of the edible object based on the color of the raw material containing the identified types of nutrients.

[0073] In this case, in step S12 of the printing method, the determination unit 432 may identify the type of food and drink represented in the image of the food and drink, and identify the types of nutrients lacking in the food and drink based on the identified type. For example, the determination unit 432 applies template matching to the image of the food and drink to identify the type of food and drink represented in the image. The determination unit 432 may also identify the type of food and drink represented in the image using a learned model. When a learned model is used, the determination unit 432 identifies the food and drink area using a learned model that has been trained to output the type of food and drink represented in the image when the image is input. For example, the learned model is an object detection model such as R-CNN. The determination unit 432 identifies the types of nutrients lacking in the food and drink by comparing the types and amounts of nutrients associated in advance with the identified type of food and drink with the required nutrients of the user. The required nutrients of the user may be a default value, or may be a value set based on the user's physical information such as age, gender, height, or weight stored in advance in the memory 41.

[0074] In addition, the determination unit 432 may determine the color of the edible food based on the color of the raw material containing the type of nutrient lacking in the food and drink, and determine the appearance of the edible food by selecting the image of the determined color as the image to be printed on the edible sheet. For example, the determination unit 432 refers to the raw material table T1 to identify the raw material containing the type of nutrient lacking in the food and drink, and determines the color of the identified raw material as the color of the edible food. The determination unit 432 determines the appearance of the edible food by selecting, from among the plurality of images stored in the image table T2, the image containing the determined color of the edible food as the image to be printed on the edible sheet. Not limited to the above example, the determination unit 432 may determine the color obtained by mixing a plurality of raw materials containing the type of nutrient lacking in the food and drink as the color of the edible food.

[0075] As described above, in this modification, the determination unit 432 determines the appearance of the edible food based on the color of the raw material containing the type of nutrient lacking in the food and drink. Therefore, the server 4 can determine the appearance of the edible food that conforms to the food and drink by supplementing the nutrients lacking in the food and drink.

[0076] Not limited to the above examples, the determination unit 432 may specify the amount of nutrients lacking in the food and drink based on the image of the food and drink, and determine the appearance of the edible food based on the amount of raw materials corresponding to the specified amount of nutrients.

[0077] In this case, the determination unit 432 specifies the type of the food and drink represented in the image of the food and drink, and specifies the type and amount of nutrients lacking in the food and drink based on the specified type. The determination unit 432 specifies the type and amount of nutrients lacking in the food and drink by comparing the type and amount of nutrients associated in advance with the specified type of food and drink with the user's required nutrients.

[0078] Further, the determination unit 432 determines the appearance of the edible food by selecting, as the image to be printed on the edible sheet, an image in which the types and amounts of raw materials corresponding to the types and amounts of nutrients lacking in the food and drink are used. For example, the determination unit 432 refers to the raw material table T1 and the image table T2, and specifies the types and amounts of nutrients contained in the raw materials used for printing each of the plurality of images. The determination unit 432 determines the appearance of the edible food by selecting, as the image to be printed on the edible sheet, an image in which the types and amounts of nutrients contained in the raw materials used for printing satisfy the types and amounts of nutrients lacking in the food and drink. The types and amounts of nutrients contained in the raw materials used for printing the image may be stored in the memory 41 in advance.

[0079] Thus, in this modification example, the determination unit 432 may determine the appearance of the edible food based on the amount of raw materials corresponding to the types and amounts of nutrients lacking in the food and drink. In this case, the server 4 can determine the appearance of the edible food that conforms to the food and drink by sufficiently supplementing the nutrients lacking in the food and drink.

[0080] [Modification Example 2 of the First Embodiment] In the above-described embodiment, the determination unit 432 determines the appearance of the edible food based on the image of the food and drink. Not limited to such examples, the determination unit 432 may further determine the arrangement of the edible food with respect to the food and drink based on the image of the food and drink.

[0081] The determination unit 432 may determine the arrangement of the edible food with respect to the food and drink based on the color of the food and drink represented in the image. For example, the determination unit 432 determines the position where the edible food is to be arranged as the position of the one with the lowest color saturation among the staple food, main dish, and side dish. Thereby, since the food and drink with low saturation is hidden by the edible food, the food and drink as a whole becomes colorful.

[0082] The determination unit 432 may also determine the arrangement of the edible food with respect to the food and drink based on the type of the food and drink represented in the image. For example, the determination unit 432 determines the position where the edible food is to be arranged as the position of the side dish among the staple food, main dish, and side dish. In this case, the determination unit 432 identifies the type of the food and drink that is the side dish using template matching or a learned model. The determination unit 432 selects, as the image to be printed on the edible sheet, the image representing the side dish of the specified type from among the plurality of images stored in the image table T2. For example, when the side dish is pickles, the determination unit 432 selects the image representing pickles as the image to be printed on the edible sheet. Thereby, the user can grasp that the edible food should be arranged on the food and drink represented in the image printed on the edible sheet.

[0083] The determination unit 432 may determine the arrangement of the edible food based on the user's preference. For example, the user's preference used for determining the arrangement of the edible food is the preference regarding the user's color combination relationship. In this case, the determination unit 432 determines, based on the preference regarding the user's color combination relationship, the position where the edible food is to be arranged as being on the food and drink of a color that is in the same color series or complementary color relationship as the color of the edible food. The user's preference used for determining the arrangement of the edible food may be the preference regarding the user's food and drink. In this case, the determination unit 432 determines the position where the edible food is to be arranged as being on the food and drink that the user does not like. Thereby, since the disliked food and drink is hidden by the edible food, the appetite is enhanced.

[0084] [Modification Example 3 of the First Embodiment] In the above-described embodiment, the determination unit 432 determines the appearance of the edible material added to the food or drink based on the color of the food or drink. However, not limited to such an example, the determination unit 432 may determine the appearance of the edible sheet by inputting the image of the food or drink into a learned model that has been learned to output an image to be printed on the edible sheet when an image is input. In this case, the learned model may be learned using teacher data in which an image of the food or drink posted on the website is associated with identification information of an image representing the food or drink to be added to the image. The learned model may further receive the user's attribute information as an input. Thereby, an edible material suitable for the user's attributes is produced.

[0085] [Modification Example 4 of the First Embodiment] The determination unit 432 may determine the appearance of the edible material based on the relationship between a plurality of types of food or drink. When the image of the food or drink includes a plurality of types of food or drink, the determination unit 432 may select, as the image to be printed on the edible sheet, an image indicating the order in which each type of food or drink should be consumed. For example, the determination unit 432 selects an image indicating that the main dish should be consumed first, the staple food second, and the side dish third. Thereby, it becomes possible to guide a child's eating method. Further, the determination unit 432 may generate an image displaying the nutrients contained in the food or drink, and select the generated image as the image to be printed on the edible sheet. Thereby, the user can grasp the nutrients contained in the food or drink.

[0086] [Modification Example 5 of the First Embodiment] Further, the hood printer 3 may print an image on any one of a plurality of types of sheets. In this case, the determination unit 432 may determine the sheet on which the image is to be printed based on the type of the food or drink. For example, when the food or drink represented in the image contains a large amount of moisture, salt, and / or fat, the determination unit 432 determines the sheet that easily absorbs moisture, salt, and / or fat as the sheet on which the image is to be printed. The above sheet may be an edible sheet or a non-edible sheet.

[0087] The determination unit 432 analyzes the image to identify food and drink items for which it is desirable to reduce moisture, salt, and / or fat content. The determination unit 432 may select an image similar to the color and / or pattern of the identified food and drink items as the image to be printed on the sheet. By attaching the printed matter to the identified food and drink items, the moisture, salt, and / or fat content contained in the food and drink items can be adjusted. Salt and fat may be an example of specific nutrients. Also, since the appearance of the printed matter is similar to the appearance of the food and drink items, when the manufacturer of the food and drink items arranges the printed matter on the food and drink items, the manufacturer can easily recognize the location where the printed matter is arranged.

[0088] In another embodiment, the determination unit 432 may determine the edible sheet on which the image is to be printed based on the user's preference. For example, if the preference regarding the user's eating tendency indicates that the user is on a diet, the determination unit 432 determines the edible sheet with a color that reduces appetite or the edible sheet that stimulates the satiety center by electrical stimulation as the edible sheet on which the image is to be printed. When the edible sheet that stimulates the satiety center by electrical stimulation is selected, the hood printer 3 may charge the edible sheet on which the image has been printed after step S15 of the printing method. Thereby, an edible product that conforms to the user's preference is produced.

[0089] [Second Embodiment] FIG. 8 is an exemplary hardware configuration diagram of the server 5 according to the second embodiment of the present invention. The server 5 differs from the server 4 in that the processor 43 has a food and drink information acquisition unit 531 and an image acquisition unit 532 instead of the image acquisition unit 431. Since the other configurations of the server 5 are the same as the corresponding configurations of the server 4, the same reference numerals are given and the description thereof is omitted.

[0090] The food and drink information acquisition unit 531 and the image acquisition unit 532 are functional modules realized by the processor 43 that executes the program stored in the memory 41. Each of these units may be implemented in the server 5 as a dedicated processing circuit.

[0091] FIG. 9 is a sequence diagram showing an example of the flow of a printing method executed by a printing system having a server 5. The printing method is realized by a processor of each device constituting the printing system cooperating with hardware. For example, the printing method is executed in response to a user operating the user terminal 2.

[0092] First, the food and drink information acquisition unit 531 of the server 5 acquires food and drink information regarding food and drink from the user terminal 2 (step S21). The food and drink information may be information regarding food and drink consumed or prepared by the user, or information regarding food and drink that the user intends to consume or prepare. The food and drink information may be character information including a name of the food and drink or a phrase describing characteristics such as the appearance of the food and drink, which is input to the user terminal 2 by the user.

[0093] Next, the image acquisition unit 532 acquires an image of the food and drink generated based on the food and drink information (step S22). The image acquisition unit 532 may generate and acquire an image of the food and drink using a learned model. In this case, the image acquisition unit 532 generates an image by inputting the food and drink information to a learned model that has been trained to output an image when character information is input. The learned model may be an image generation AI (Artificial Intelligence) such as a GAN (Generative Adversarial Network) or a diffusion model.

[0094] The processing of steps S23 - S26 is the same as the processing of steps S12 - S15 of the processing method according to the first embodiment, and thus the description thereof is omitted.

[0095] As described above, in the present embodiment, the server 5 includes an image acquisition unit 532 that acquires an image of food and drink generated based on food and drink information, a determination unit 432 that determines the appearance of edible food added to the food and drink based on the color of the food and drink represented in the image, and an output unit 434 that outputs appearance information indicating the appearance of the edible food. Therefore, even when the user cannot generate or acquire an image of food and drink, the server 5 can determine the appearance of edible food that matches the color of the food and drink.

[0096] [Third Embodiment] FIG. 10 is an exemplary hardware configuration diagram of a server 6 according to the third embodiment of the present invention. The server 6 differs from the server 4 in that the processor 43 has a food and drink information acquisition unit 631 and a determination unit 632 instead of the image acquisition unit 431 and the determination unit 432. Since the other configurations of the server 6 are the same as the corresponding configurations of the server 4, the same reference numerals are given and the description thereof is omitted.

[0097] The food and drink information acquisition unit 631 and the determination unit 632 are functional modules realized by the processor 43 that executes a program stored in the memory 41. Each of these units may be implemented in the server 6 as a dedicated processing circuit.

[0098] FIG. 11 is a sequence diagram showing an example of the flow of a printing method executed by a printing system having the server 6. The printing method is realized by the processors of the respective devices constituting the printing system cooperating with the hardware. For example, the printing method is executed in response to the user operating the user terminal 2.

[0099] First, the food and drink information acquisition unit 631 of the server 6 acquires food and drink information regarding food and drink from the user terminal 2 (step S31). The food and drink information may be information regarding food and drink ingested or prepared by the user, or information regarding food and drink that the user intends to ingest or prepare. The food and drink information may be character information including a statement describing features such as the name or appearance of the food and drink input to the user terminal 2 by the user.

[0100] Next, the determination unit 632 determines the appearance of the edible food added to the food or drink based on the food or drink information (step S32). The determination unit 632 may identify the nutrients contained in the food or drink based on the food or drink information, and determine the appearance of the edible food based on the nutrients contained in the food or drink.

[0101] The determination unit 632 may identify the type of the food or drink based on the food or drink information, and identify the types of nutrients lacking in the food or drink based on the identified type of the food or drink. For example, the determination unit 632 may identify the type of the food or drink by extracting the name of the food or drink from the food or drink information. The determination unit 632 identifies the types of nutrients lacking in the food or drink by comparing the types and amounts of nutrients associated in advance with the identified type of the food or drink with the user's required nutrients.

[0102] The determination unit 632 may determine the color of the edible food based on the color of the raw material containing the type of nutrient lacking in the food or drink, and determine the appearance of the edible food by selecting the image of the determined color as the image to be printed on the edible sheet. For example, the determination unit 632 refers to the raw material table T1 to identify the raw material containing the type of nutrient lacking in the food or drink. The determination unit 632 determines the color when the identified raw materials are mixed as the color of the edible food. The determination unit 632 determines the appearance of the edible food by selecting the image associated with the color of the edible food in the image table T2 as the image to be printed on the edible sheet.

[0103] Not limited to the above example, the determination unit 632 may identify the amount of nutrients lacking in the food or drink based on the food or drink information, and determine the appearance of the edible food based on the amount of the raw material corresponding to the identified amount of nutrients.

[0104] Since the processes of steps S33 - S35 are the same as the processes of steps S13 - S15 of the printing method according to the first embodiment, the description thereof is omitted.

[0105] As described above, in the present embodiment, the server 6 includes a determination unit 632 that identifies nutrients contained in food and drink based on the food and drink information, and determines the appearance of the edible food added to the food and drink based on the nutrients contained in the food and drink, and an output unit 434 that outputs appearance information indicating the appearance of the edible food. Therefore, even when the user cannot generate or acquire an image of the food and drink, the server 6 can determine the appearance of the edible food that matches the food and drink by supplementing the nutrients lacking in the food and drink.

[0106] [Fourth Embodiment] FIG. 12 is an exemplary functional block diagram of a server 7 according to a fourth embodiment of the present invention. The server 7 is different from the server 4 in that the server 7 includes a food and drink information acquisition unit 731, a skin information acquisition unit 732, and a determination unit 733 instead of the image acquisition unit 431 and the determination unit 432. Since other configurations of the server 7 are the same as the corresponding configurations of the server 4, the same reference numerals are given and the description is omitted.

[0107] The food and drink information acquisition unit 731, the skin information acquisition unit 732, and the determination unit 733 are functional modules realized by a processor 43 that executes a program stored in the memory 41. Each of these units may be implemented in the server 7 as a dedicated processing circuit.

[0108] FIG. 13 is a sequence diagram showing an example of the flow of a printing method executed by a printing system having the server 7. The printing method is realized by the cooperation of the processors of the respective devices constituting the printing system with the hardware. For example, the printing method is executed in response to the user operating the user terminal 2.

[0109] First, the food and drink information acquisition unit 731 of the server 7 acquires food and drink information regarding food and drink from the user terminal 2 (step S41). The food and drink information may be information regarding food and drink that the user has consumed or prepared, or information regarding food and drink that the user intends to consume or prepare. The food and drink information may be character information including a statement describing features such as the name of the food and drink or the appearance of the food and drink, which is input to the user terminal 2 by the user.

[0110] Next, the skin information acquisition unit 732 acquires skin information indicating the state of the user's skin who ingests food and drink from the user terminal 2 (step S42). The skin information may include an image representing the user's skin generated by the user terminal 2 photographing the user's skin.

[0111] Next, the determination unit 733 determines the appearance of the edible added to the food and drink based on the food and drink information and the skin information (step S43). The determination unit 733 may determine the appearance of the edible based on the nutrients contained in the food and drink specified based on the food and drink information and the nutrients recommended for the user specified based on the skin information.

[0112] The determination unit 733 may specify the type of the food and drink based on the food and drink information, and specify the types of nutrients contained in the food and drink based on the specified type of the food and drink. For example, the determination unit 733 may specify the type of the food and drink by extracting the name of the food and drink from the food and drink information. The determination unit 733 specifies the types of nutrients associated in advance with the specified type of the food and drink as the types of nutrients contained in the food and drink.

[0113] The determination unit 733 may analyze the skin information to specify the nutrients recommended for the user. For example, the determination unit 733 may generate numerical data indicating the degree of spots, wrinkles, etc. of the user's skin by applying a known image analysis method to the image representing the user's skin. The determination unit 733 may specify the nutrients recommended for the user by referring to the correspondence between the numerical data stored in advance in the memory 41 and the nutrients recommended for the user.

[0114] The determination unit 733 may specify the types of nutrients lacking in the food and drink based on the nutrients contained in the food and drink and the nutrients recommended for the user. For example, the determination unit 733 may correct the necessary nutrients of the user based on the nutrients recommended for the user. The determination unit 733 may specify the types of nutrients lacking in the food and drink by comparing the nutrients contained in the food and drink with the corrected necessary nutrients.

[0115] The determination unit 733 may determine the color of the edible food based on the color of the raw material containing the nutrient lacking in the food and drink, and determine the appearance of the edible food by selecting the image of the determined color as the image to be printed on the edible sheet.

[0116] Not limited to the above example, the determination unit 733 may specify the amount of nutrients lacking in the food and drink based on the food and drink information and the skin information, and determine the appearance of the edible food based on the amount of the raw material corresponding to the specified amount of nutrients. Since the processing of steps S44 - S46 is the same as the processing of steps S13 - S15 of the printing method according to the first embodiment, the description thereof is omitted.

[0117] As described above, in the present embodiment, the server 7 includes a determination unit 733 that determines the appearance of the edible food added to the food and drink based on the food and drink information and the skin information, and an output unit 434 that outputs appearance information indicating the appearance of the edible food. Therefore, even when the user cannot generate or acquire an image of the food and drink, the server 7 can determine the appearance of the edible food that suits the food and drink by supplementing the nutrients lacking in the food and drink to keep the user's skin healthy.

[0118] In the above-described embodiment, the determination unit 733 is assumed to determine the appearance of the edible food added to the food and drink based on the food and drink information and the skin information. Not limited to such an example, the determination unit 733 may determine the appearance of the edible food without relying on the food and drink information. In this case, step S41 of the printing method may be omitted. Also, in step S43, the determination unit 733 may specify the types of nutrients recommended for the user based on the skin information, determine the color of the edible food based on the color of the raw material containing the specified types of nutrients, and determine the appearance of the edible food by selecting the image of the determined color as the image to be printed on the edible sheet. In this case, the server 7 can determine the appearance of the edible food that suits the food and drink from the perspective of keeping the user's skin healthy.

[0119] In the above-described embodiment, the skin information was assumed to include an image representing the user's skin. However, the skin information is not limited to such an example, and may include numerical data indicating the degree of freckles, wrinkles, etc. of the user's skin, which is generated by analyzing an image representing the user's skin or by applying other measurement methods.

[0120] The above-described embodiments and modifications may be implemented in appropriate combinations within the scope of the present invention. A part of the components of the above-described specific embodiment or modification may be added to the other above-described embodiments or modifications, or may be replaced with a part of the components of the other embodiments or modifications. When the above components are added and / or replaced, other components may be deleted. Also, a part of the components of the above-described specific embodiment, modification or combination may be deleted. The various processes described above may be executed in different orders as appropriate within the scope of the present invention.

Explanation of Reference Numerals

[0121] 1 Printing system 3 Hood printer 4 Server 431 Image acquisition unit 432 Determination unit 433 Setting unit 434 Output unit

Claims

1. An image acquisition unit that acquires an image of food and drink; An appearance determination unit that determines the appearance of edible food added to the food and drink based on the color of the food and drink and / or the nutrients contained in the food and drink; An output unit that outputs appearance information indicating the determined appearance of the edible food; An information processing apparatus having the above.

2. The appearance determination unit determines at least one of the color, shape, and pattern of the edible food as the appearance of the edible food. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1.

3. The appearance determination unit calculates a feature amount related to the color of the food and drink from the image of the food and drink, and determines the color of the edible food based on the feature amount. The information processing apparatus according to Claim 2. The information processing apparatus according to Claim 2.

4. The output unit outputs the appearance information to a printing apparatus that produces the edible food by printing. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1.

5. Further comprising a storage unit that stores in association a plurality of raw materials used in the printing apparatus, the color of each raw material, and the type of nutrients contained in each raw material. The appearance determination unit identifies the type of nutrients lacking in the food and drink based on the image of the food and drink, and determines the appearance of the edible food based on the color of the raw material containing the identified type of nutrients. The information processing apparatus according to Claim 4. The information processing apparatus according to Claim 4. The information processing apparatus according to Claim 4.

6. The appearance determination unit further identifies the amount of nutrients lacking in the food and drink based on the image of the food and drink, and determines the appearance of the edible food based on the amount of the raw material corresponding to the identified amount of nutrients. The information processing apparatus according to Claim 5. The information processing apparatus according to Claim 5.

7. Further comprising a storage unit that stores the user's preference regarding the appearance of the edible food. The appearance determination unit determines the appearance of the edible food further based on the user's preference. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1.

8. The appearance determination unit determines a plurality of candidates for the appearance of the edible food, determines, as the appearance of the edible food, the candidate selected by the user from among the plurality of candidates, and further comprises a setting unit that sets the user's preference based on the history of the user's selection. The information processing apparatus according to Claim 7. The information processing apparatus according to Claim 7. The information processing apparatus according to Claim 7.

9. Further comprising an arrangement determination unit that determines the position of the edible food with respect to the food and drink and / or the size of the edible food. The output unit further outputs arrangement information indicating the determined position and / or size. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1.

10. A food and drink information acquisition unit that acquires food and drink information regarding food and drink prepared or consumed by the user, or prepared or consumed by the user; A nutrient identification unit that identifies nutrients contained in the food and drink based on the food and drink information; An appearance determination unit that determines the appearance of the edible food added to the food and drink based on the nutrients contained in the food and drink; An output unit that outputs appearance information indicating the determined appearance of the edible food; An information processing apparatus having the above.

11. Further comprising a skin information acquisition unit that acquires skin information indicating the state of the user's skin, The appearance determination unit identifies nutrients recommended for the user based on the nutrients contained in the food and drink and the skin information, and determines the appearance of the edible food based on the nutrients recommended for the user. The information processing apparatus according to claim 10.

12. A skin information acquisition unit that acquires skin information indicating the state of the skin of a user who consumes food and drink; An appearance determination unit that identifies nutrients recommended for the user based on the skin information, and determines the appearance of the edible food added to the food and drink based on the nutrients recommended for the user; An output unit that outputs appearance information indicating the determined appearance of the edible food; An information processing apparatus having the above.

13. A food and drink information acquisition unit that acquires food and drink information regarding food and drink prepared or consumed by the user, or prepared or consumed by the user; An image acquisition unit that acquires an image of the food and drink generated based on the food and drink information; An appearance determination unit that determines the appearance of the edible food added to the food and drink based on the color of the food and drink in the image; An output unit that outputs appearance information indicating the determined appearance of the edible food; An information processing apparatus having the above.

14. The information processing apparatus according to any one of claims 1-13; A printing apparatus that produces the edible food by printing based on the appearance information output from the information processing apparatus; A printing system having the above.

15. An information processing method executed by an information processing apparatus, Acquiring an image of food and drink, Determining the appearance of the edible food added to the food and drink based on the color of the food and drink and / or the nutrients contained in the food and drink, Outputting appearance information indicating the determined appearance of the edible food; An information processing method including the above.

16. A program for causing a computer to function as the information processing apparatus according to any one of claims 1-13.

Images